skysight-inc/HackerNews huggingface dataset:
Ask HN: Ship logs to a central Elasticsearch from many remote locations
hi,

I need to ship logs to a single central Elasticsearch over the internet, from a bunch of different locations, in some of them connection can be unstable. I plan to use filebeat installed in every location with reasonably large queue configured, so any connection glitches would not stop logs from eventually being shipped to Elastic.

A question - had you ever worked with scenario like this? Do you know anything better that just putting filebeat with large queue in every remote location? Any thoughts/ideas?

Any feedback/suggestion is very welcome.

Thank U!!!
That link says two years, actually. 2016 dropped from 2015, 2017 and 2018 increased, nothing is specified for 2019. Are you saying there was another increase in 2019?

I assume that coal plants become unprofitable once they need considerable maintenance because they're not being built any more, which I assume means that no investors consider them a worthwhile investment. Further, because the price of solar and wind power is dropping. If the price curves continue as in the past decade, then in a few years, building a new solar plant becomes cheaper than operating an existing power plant in the first parts of the world. At that point, what coal plant owner will spend money on signfificant maintenance?
Ask HN: Companies that offer SF/BayArea salaries for full time remote position?
Hi guys, I'm looking to gather a list of companies that offer a competitive salary (at Bay Area levels) for full-time remote positions in software development.
I thought this was a decent opinionated post about some of the UI fundamentals. I doubt many developers really have the patience to dig deep into UX stuff after all.

You got any good resources you'd recommend to read about the visual hierarchy concept?
Deeply concerning that the same megalomaniac threatening nuclear war on Twitter also broadcasts claims that climate change is a Chinese hoax.
As a collector of maps, thank you so much for this! The layout is so creative and informative, and the styling is great!
True, but in this case (apparently) there is the added wrinkle that this roundabout has better throughput than other designs, so what’s the downside? I guess it forces people to think harder than they’d normally have to while driving?

I definitely find myself taking slightly longer routes that let me avoid stressful situations, e.g. turning left onto a busy road.
My cubicle at my last job was surrounded by a mountain of expensive printers because they would loose some funding if they didn't spend all of their budget each year.

If we ordered a server it would show up in 6 to 12 months and sometimes would be sent to the wrong place. The people in charge of ordering would replace SSDs with spinning disk and we wouldn't know about the change until the wrong parts arrive months later. Fun times.
As a former longtime contractor I totally agree. One thing often overlooked is this 'Silver Tsunami' that will wreak havoc on the old guard (existing giant contracting companies). The existing workforce is aging, the new blood is uninterested in the old ways (cruft, old tech) and frankly there's tremendous opportunity for a 'Space-X' style small contractor to get a serious foothold.

Some of these companies need to replace 40% of their workforce in the next decade. Who would a new grad choose?
I doubt it's the "lowest it has ever been" when you have the US president bragging about having a "big button" on twitter...
> By what right does google set the search engine to begin with? They use their browser's popularity for a competitive advantage and that's legal.

Yeah, but the solution to that is regulation and not an arms race! Google should (and does) allow the user/installer/organization to change the search engine, precisely to allow for competition in the industry.

Browser vendors need to respect that choice. It would be equally bad for Google to be surreptitiously change the Chrome (or Edge) search engine to Google from Bing! It's just bad. Respect user preferences, period.
The user is the business that administers the machines. They want this. If not, they would disable it.
> The world is sleepwalking its way through a newly unstable nuclear landscape. The arms control boundaries that have helped prevent nuclear catastrophe for the last half century are being steadily dismantled.

There is so much ignorance in this piece. Almost nobody in the real world abides by these arms controls. Russia ignored them. So did Iran. So will the next country. Why would the US abide by a treaty that’s being overtly flouted?

Also, how does recognizing this reality make things more dangerous like the article suggests?
There absolutely is a middle ground. It starts with actual audits of the DoD, which requires no reduction of capabilities and have never even been completed once.
Let the mansplaining begin
We outsource the dirty stuff. The important thing is that there is a market for our bullets and bombs.
Honestly, some of the comments here (if you show dead ones) just prove the need for a community like this.

@mariedm - congrats! Love the design.

Edit: One suggestion - a personal peeve is when a site flashes up a "subscribe" modal when I've only been on the site for a few seconds. Not sure what the conversions are like on that, but personally I'd be more likely to do that if I'd had a chance to look around.
If you think that the only thing the U.S. military does is "assault" then you haven't really done any research at all, have you? Read up on force-projection, specifically as it relates to soft-power.
> We put databases on Oracle that should be handled by Postgres or MariaDB since DoD prefers Oracle.

I mean, if you’ve already budgeted the CapEx for some additional Oracle licenses, the OpEx efficiencies of having unified tooling and a unified ops doctrine are no joke.

I haven’t worked with an Oracle DBMS, but I think this is analogous: I’d sure hate to have to manage a cloud infrastructure where parts were on AWS, parts on GCP, and parts on Azure. Sure, there are generic tools that treat all three the same, or over-layers like K8s that don’t care about substrate—but what if the projects on each platform were taking advantage of that platform’s specialties? What if I was using SNS on AWS, or BigQuery on GCP?

To bring that back through the analogy, what if our Oracle projects were tuned using Oracle-specific query-planner hints, while our Postgres projects did their ETL using PG-specific Foreign Data Wrapper connectors?

In both cases, the only real solution is hiring and retaining O(N) specialized ops headcount, one team for each stack. And that cost gets a lot higher than just paying for another darn Oracle license.
> Understandably, many on the left believe the problem is simply the Republican Party, and if only Democrats could win decisive majorities, American democracy would work better.

I was waiting for a paragraph that started "On the other hand, many on the right believe the problem is the Democratic party," or something to that effect. The author says the problem is the two party system. I think this article shows that the problem is that half the country pretends the other half doesn't exist, and doesn't even bother addressing them, let alone convincing them to change their minds.
More importantly, we're diluting the gene pool by allowing stupid people to procreate. Let's stop this thing at the source.
"The cloud" is such a scam - of course a "war cloud" - why not, just triple the security billing invoice and collect obscene cash and power. Every sucker buys into "the cloud" when running a server yourself is elementary. We are doomed.
this is from 2017, should likely be noted in the title.
That's the more important point here. Figuring out what gives you enjoyment and motivation is HARD and we shouldn't beat ourselves up if we haven't figured it out yet. The process of figuring it out is messy and chaotic but so long as we're making some progress we should be kind to ourselves.
While I'm in total agreement, and think that this is likely a bad idea, doesn't the same problem exist with mail-in voting? An abusive party can hover over your shoulder while filling it out, and then walk you to the mailbox.
71 detainees were classified as too dangerous to be transferred from Guantanamo, even though there was no evidence to justify laying charges against them.

Mohammed Ahmad Ghulam Rabbani was one of the 71 individuals deemed too innocent to charge, but too dangerous to release.
This is probably a silly question, but why can't they sue in civil court _whilst_ being imprisoned?
But it's clearly not accurate even with respect to itself. The world is closer to nuclear annihiliation now than in any year since 1953, including multiple proxy wars between the US and USSR along with the Cuban Missile Crisis? I don't see any plausible way to believe that.
This is still the case. I've had ongoing neurological and digestive problems for years now, and each primary physician or neurologist I see has been hesitant to run tests let alone attempt to diagnose what's going on. So they labeled it as "stress" which made me feel like I was crazy for a very long time. Paradoxically this caused me a great deal of stress. I ended up seeing two different mental health specialists - one with experience in somatic disorders - that concluded (after months of therapy) it's not a psychological problem.

There's so many experiences reported in MS and other neuro-based communities online where people were given the stress and anxiety label only to be diagnosed months, years, and sometimes decades later with an actual illness like MS. I feel truly bad for people that have the extraordinarily rare diseases because chances are slim they'll find someone to actually investigate beyond the standard textbook procedures.
We had that back like 10 years ago when google provided a tool (I forgot it's name, Google desktop search?) which scanned and indexed files locally. It was accessible through a shortcut, I laid mine to double-Ctrl and upon invocation it prompted a search bar

It was so great in search quality, number of formats it could index and performance that at one day I index the whole company file share. People showed up next to me and I rediscovered files which would have been lost otherwise.

This tool also integrated after configuration with the browser which issued an invocation to localhost to integrate local search results. As this was widely regarded as a terrible security feature to potentially tell google all about local files, reception of that feature was poor. This feature was the only incentive for google to provide the tool and therefore the tool got quickly axed.
Please don't post unsubstantive and/or flamebait comments to HN. It only makes this place even worse.
The statistics of those are also public record and show that US surveillance is nowhere close to Chinese.
What do you not get from, say, Hacker News or StackOverflow that you're trying to fill with WomenMake? Genuine question, not a snark.
The Zettelkasten Method
I think people are missing a key point here . This is included with the pro plus installer which is exclusively a business product. My company installs a bunch of tracking and other crap software on my windows machine in the name of data privacy, security and other bullshit. All it does is makes my i7 laptop with 16 gigs of ram behave like a slow crappy laptop.
So if you have an issue with this complain to your work organization because somebody higher up made a decision to go with Pro plus .
The cloud is cheap, it's when you need so many clouds you can't see sunlight anymore it becomes expensive.
I never thought about how you could be coerced into voting a specific way like that and it does seem to be a big problem to me. But is that already possible with mail-in voting?
What's hysterical to me is that Microsoft was actually making this point in their proposal because they very much expected for AWS to solely receive go-ahead with this contract.
What's so "surely" about it?
You’re right it’s not a great fit. What I meant was “something we as a society reflexively say” (which shibboleth kinda captures) “which is obsolete” (which shibboleth does not).
I agree, the "100 seconds to midnight" feels a little silly, but:

1. These scientists know that a good headline is necessary in today's world to become part of popular discourse.

2. Regardless of the headline, the actual post IMO is a very good and thoughtful overview of how huge resources put towards technological propaganda are undermining societal institutions around the world and raising the risk that "cooler heads" will not prevail.
Like I said the first reason why I created Women Make is because I couldn’t find fellow women makers. There are several reasons for this: women are less visible in general, and also tend to less promote themselves. Gathering us is an opportunity to find allies, but also models, people from who you can inspire from and find the strength to do what you want to do. That being said, it doesn’t have to be exclusive. I am myself part of several communities, and most of them are not emphasizing on women. But I’m glad I have a place like Women Make for certain topics and to know I’m not alone in this.
> I'm really prone to letting my thoughts get off on the worst tangents, and it will take a while to get myself back on track

Sometimes I'll be listening to an audiobook and I'll start thinking about something else. By rewinding the audiobook I can figure out how long the distraction was (could be over a minute). It also amazes me how I have no recollection of the portions of the audiobook that were playing while I was distracted.
How is this different from voting by mail, then? e.g. basically everyone in Oregon votes by mail only.
Could you explain a bit more about the specific problems / challenges your policies (e.g. women-first moderation) are meant to address?

I'm a man working in the tech industry, and I feel like I still don't 100% understand the concerns or frustrations that motivate a project like yours. I'd like to better understand where you (and presumably some other women) are coming from.
Might be unpopular opinion, but I honestly think nuclear weapons are fine.

Nuclear weapons are a 70 y/o technology, and I don't like my taxpayer dollars going to innovation in WMDs.

Changing to something else would result in additional spend by military-industrial complex for systems that might not be as ready or safe, faster promotions by career types (vs. career death when you get shunted into nuclear weapons work), and reinventing the wheel in terms of diplomatic protocol, public understanding of risk/reward, and potential international crisis.

I just don't think it's worth getting rid of nukes before changing the fundamental properties of human nature. Until that happens, I'd rather keep the nukes rusting away in their silos to keep the saber-rattlers happy and headaches away.
You're right, I just gut-reacted to the first comments.
My first "real computer" (not counting the "computer" in the Milton Bradley Big Trak - though it was a form of Logo programming, I suppose) was a TRS-80 Color Computer 2 - in 1984.

My parents got it for me after I showed an interest in the Apple IIe computer that was wheeled around my elementary school at the time (yes - one computer for the whole school, each teach got it "checked out" once a month - it ended up being so popular, that the teachers later gave up their break room for the kids to have a "computer room" to learn in, completely outfitted by Apple and donations).

Anyhow - I had a TV for a monitor, cassette tape for storage, and a few cartridges (Canyon Climber and Reactoid were my favorite - though later in my teenage years I would discover Dungeons of Daggorath).

I typed in tons of BASIC programs from magazines and books, and yes - finding things incompatible was both frustrating, but informative. I took to learning how to convert between BASICs - GWBasic, PCjr BASIC, and AppleSoft Basic were all fairly easy - for the most part - to convert from so as to run on the Color Computer.

Forget anything for the C=64 or Vic-20, as most of them used tons of POKEs and the special sprite hardware to get anything done, and were (for a kid) impossible to understand what was going on or being done.

I also learned how to convert some code from Fortran to BASIC - I had found a book of graphics code in Fortran, and wanted to run the programs myself...so I learned.

Later, I found a book that detailed how to convert BASIC programs from one system to another; that also assisted me. But ultimately, it was a case of trial and error, tons of reading about the different systems graphics modes, etc.

I still own all of my old computers; I'll keep 'em until I die, I suppose.
The end result of your stance is essentially arguing that the only problem with American intervention in the Middle East is that it isn't efficient enough.
Connecting founders looking for a side gig and companies looking to hire them
You can't unopen Pandora's box. It worked well enough for the few years only the US had WMDs, but that could never last.
> [If] Apple were in fact creating their own key pairs on their server and sending users the key pair, don’t you think someone would have discovered.

You once again misunderstand the vulnerability. The vulnerability is that China does this because China controls the keyservers in China.

As far as anybody discovering this, that would be very difficult because Apple does not let you install your own apps on the device and would not approve an app designed to detect this.

But even more, why would they bother? People who care about their privacy will simply avoid closed source software and especially closed systems like Apple's instead of trying to use a known compromisable system safely.

>But since it’s in a Wikipedia article, I guess that kind of closes the case.

I was pointing you to a place where you could learn about cryptography because you seem not to understand the basic concepts. The Wikipedia article does not describe this particular vulnerability.
Amazon Engineer Leaked Private Encryption Keys. Outside Analysts Discovered
Since you asked, here’s some (old) science predicting the future (current state) of civilization:

In 1980, the American Petroleum Institute correctly predicted the current environmental crisis would happen now.

From the report:

"Timescale for significant impact, very roughly 50 years"
"1°C Rise (2005): Barely noticeable"
"2.5°C Rise (2038): Major economic consequences, Strong regional dependence"
"5°C Rise (2067): Globally Catastrophic effects"

They also predict that climate problems will cause global economic growth to halt in about 2025.

There’s no reason to think the trend lines in the report are suddenly incorrect. In fact, current events are well within the error bars of their model.

Averting it requires billions of people to agree to change course, and so far our leaders are doing nothing to make that happen.

It seems much worse than the nuclear scare to me.

Here’s a HN discussion of the API report:

https://news.ycombinator.com/item?id=20325359

Here’s a report from the same organization walking through the math:

https://insideclimatenews.org/sites/default/files/documents/...

Of course, much progress has been made in climate modeling since then, and now we’re more sure we’re screwed. On the other hand, since the report is 40 years old, it’s trivial to go back and confirm their predictions were correct so far.
So if I am running both Nextdns and adblocker locally, is there any point ? Which one is blocking first ?
Propotional representation is not with out its problems, but I think given that our current system has a bicameral legislature where one body (the Senate) basically just represents entrenched power interests and is badly divorced from the will of the people, and grossly unrepresentative (since it's 2 per state where states have differing populations), I think we could gain a lot by making that body proportional.

Would it be perfect? Nah. Would it be -a lot- better than what we have? Yes.

With the Senate proportional party, and the House still geographically representative, and then if you add in ranked choice voting, you could break the two party system and have a much, much more representative government.
AWS container security survey 2019 results
Mmm enjoy the boredom and think about it as a time for you to attentively choose what to do with your spare time?
Note I said most cases, which, in English, means "not all cases, but most of them."

Obviously, the U.S. Military does slightly more than assault. Proportionally not much more, but a bit more. Like football.

https://en.wikipedia.org/wiki/Army_Black_Knights_football
A Natural Log: Our Innate Sense of Numbers Is Logarithmic, Not Linear (2008)
Something to do with the fact that he is lazy loading images using JavaScript I believe.
Phishing campaign leads to UPS Store data breach
For the FT specifically, I get the impression that a pretty large portion of subscriptions are corporate/paid for by employers so I'm not sure how true that really is.
I haven't used Mint, so I went to take a look...

Am I missing something, or is Mint asking me to hand over the credentials to all my other financial accounts? That strikes me a horrendously awful idea.
Twitter demands AI company stops 'collecting faces'
Also this is completely ignoring that in China the government would have picked Microsoft over Amazon, and there would be zero legal recourse because there is no rule of law there. And as Chinese subjects we would not be allowed to say anything critical of the government because it threatens the "national dignity." Even if it's obviously cronyism perpetrated by the leadership, like if Microsoft in this case were owned by one of the party officials or a relative. We would be expected to trust our government explicitly even in cases where we personally know that it's wrong.

So sure, you can get a lot done in those cases, but it might not be the right things to do, and there's zero process for remediation except to hope that another party leader takes over and changes it for you.

So when anyone says China is totalitarian this is what is meant and why the US is ideologically opposed to China and Chinese rule.
Ahh, ok. That explains it. I was wondering what I was missing. Thank you.

edit: and that I can live with.
> Only agreement 1 can be guaranteed to save humanity.

That doesn't work either—some country will defect and develop WMDs in secret and then take over the world. Of the choices you present, the only reasonable option is ubiquitous access to WMDs (option 3), with the understanding that anyone found using them for any purpose other than a proportional response to a existential threat will be ruthlessly annihilated by everyone else.
The historian Dan Snow recently did a podcast with the author of a new book about Pilecki. Definitely worth a listen.

https://www.historyhit.com/podcasts/

(2nd one down on the left)

Book: https://www.amazon.co.uk/Volunteer-Mission-Underground-Ausch...
> It only makes this place EVEN worse.

@dang, I believe you need a day off :)

But seriously, just in case you're feeling a bit down about HN lately, I really appreciate the work you guys do to make this a good place for discussion. I'm sure I'm not alone in that. Well done.
It does seem bad; it's hard to say if it's even an improvement on net. https://www.electionscience.org/library/approval-voting-vers...
> I absolutely think there is a such thing as ADHD and these people were prescribed the medication to manage it

Then why do you claim they were "high"? No, they're more toward "normal" than they would otherwise be.

Claiming ADHD people are high, while you are "sober" is a pejorative which diminishes the fact that ADHD exists and needs treatment, and is usually intended to embarrass and denigrate those that have it.
How did a comment that must be from 2015 leap into this thread?