Does HTTPS actually protect your data? - Page 1
Forum section: electronics / general

author: IanMacdonald
author_rank: Frequent Contributor
timestamp: October 10, 2017, 11:31:55 am
content: Been doing a bit of research into this, and the results are just slightly worrying.https://iwrconsultancy.co.uk/blog/httpsIt strikes me that the security researchers who've put videos on YT about this, have been 'cherry picking' scenarios where SSL does protect. Which it does in some cases. It's the number of instances where it doesn't that are the concern though. Especially as people will tend to assume that they are always protected from password theft if it is active.

author: grumpydoc
author_rank: Super Contributor
timestamp: October 10, 2017, 11:45:00 am
content: Implemented correctly it protects data "in flight" so, for example, a malicious employee at your ISP could not sniff all traffic to pull out user name/site/password combos.I suppose its value depends on how much of a threat you think the above is compared with, say, exploiting poor programming and sanitising of user input to suck all of the user name/password combos from the remote database.It also prevents ISPs modifying data in transit in other ways such as inserting ads - so I suppose it does have its uses.

author: hermit
author_rank: Frequent Contributor
timestamp: October 10, 2017, 11:50:34 am
content: It's better than nothing but you have full time criminals looking for ways around it. Do you keep all of your money under your mattress because sometimes banks get robbed? Lots of compromises still happen because people click on links or open attachments they shouldn't. Sure, someone just mistakenly picked you out for a huge refund you don't have coming. Just click on the attachment to claim it.......

author: Red Squirrel
author_rank: Super Contributor
timestamp: October 10, 2017, 02:15:57 pm
content: The way I like to see encryption is that it does not stop eavesdropping 100%, but it slows it down. The big 5 eyes networks store every packet that crosses their listening posts, if they can't decrypt it with today's tech they will be able to with tomorrow's, but they may also have discard the data by then if they feel it's not important to them or them finding out the info may not be as critical by then.

author: tggzzz
author_rank: Super Contributor
timestamp: October 10, 2017, 02:26:19 pm
content: Quote from: IanMacdonald on October 10, 2017, 11:31:55 amIt protects in the same way as an armoured van protects its contents during the journey from here to there. If the employees are crooked or the buildings are tin shacks, the value of an armoured van is moot.

author: T3sl4co1l
author_rank: Super Contributor
timestamp: October 10, 2017, 05:28:22 pm
content: Is it enough, on its own? No, of course not.Is it better than nothing? Absolutely!I even set up HTTPS on my website (even though I have no active content there..). Was kind of a pain, but it's behaving well enough. I use Let's Encrypt's certification.Tim

author: Jr460
author_rank: Regular Contributor
timestamp: October 10, 2017, 06:21:46 pm
content: The biggest problem with HTTPS is that until recently and still by default it will try to use some very poor encryption.Many places have polices and scan their network to find web servers that will accept poor ciphers.New browsers will also disable weak ciphers.The other problem is the key management for the negotiated cipher. It is sent over the connect encrypted by private key of the server's certificate. So like other have said, if you record all traffic, and then later get the certificate either hook/crook, poor management of the web server, or FBI with a national security letter, then all past traffic can be decrypted.Much new versions of TLS/SSL allow for Diffie-Hellman key negotiation where the key is never sent over the wire. This is what SSH has done since day one. It doesn't matter if you record the traffic, the keying material is never sent or kept.Now, try to find web servers that only use DH for keys..... I don't think it is very popular at this time.

author: chriswebb
author_rank: Unknown
timestamp: October 10, 2017, 07:27:17 pm
content: One way to test your HTTP server is to use something like the ssl lab's tester:https://www.ssllabs.com/ssltest/Gives you a pretty detailed look at what your server does, and if it is doing something poor, it gives great details on how to fix. Went from a D to an A+ in a few minutes with that site.

author: bd139
author_rank: Super Contributor
timestamp: October 10, 2017, 07:35:16 pm
content: Let’s just add that it’s a cat and mouse game. Protocols are evolving. Ciphers are evolving. Key exchange methods are evolving. People are finding viable attacks against these bits of mathematics all the time.Fundamentally TLS is secure but only if you configure it properly and keep on top of protocol evolution.There is a possibility that the cat will catch up with the mouse suddenly. At this point we’re shit out of luck and society may fall overnight. No joke.In the mean time it all stacks on honesty of the PKI which is debatable, whether or not implementations are flawed or have been made flawed by security service operators (look at openbsd IPSec for an example of this) and whether or not either peer is trustworthy. If you’re using an old TLS version to talk to Equifax from a crapware infested shitbox then it’s game over.Ugh it’s a mess.SSH is much better when it comes to key management.(This is the day job by the way)

author: taydin
author_rank: Frequent Contributor
timestamp: October 10, 2017, 07:50:50 pm
content: https is a good defense against individual or organized crooks, or government agencies of less developed countries. But I am assuming that it is absolutely zero defense against cryptoanalysis by the NSA and related organizations. So, is it safe or not for YOU? Well, it depends on how critical your secret is

author: lundmar
author_rank: Frequent Contributor
timestamp: October 10, 2017, 07:53:05 pm
content: Once we get powerful quantum computers all current ciphers will be breakable and no archived data from the past will be secure. It will be a game changer.My point is, whatever you do, don't use the internet until we have post-quantum cryptography!Best to play it safe - I'll see you guys again in 2030.

author: IanMacdonald
author_rank: Frequent Contributor
timestamp: October 10, 2017, 08:09:44 pm
content: Some of the responders seem to have missed the point. It's not a question of whether the SSL cipher is strong or weak, or whether it is better than nothing. It's a very simple issue of whether the product does what it says on the tin. It is supposed to prevent a MITM from stealing sensitive data such as passwords. If it does not do that to a reasonable degree, then it is snake oil.Your call as to what a reasonable degree is, but I'd have said that over 75% success rate might qualify, under 50% not. After all you don't expect AV software to trap every virus, or a spamfilter every spam. You expect a serviceable performance, though. Especially if you have to pay for the product. Which you do, where proper certificates are concerned.The lowdown is:HTTPS prevents your ISP from acting as a MITM. Success.HTTPS does not prevent third party advertisers from acting as a MITM. Fail.Each visit to a large website involves one ISP, but anything up to 40 connections to advertisers and other third parties.That is a2.5% success ratein performing its claimed function. Give or take.Even with just one advertiser on a page, that's still only a 50% success rate.There is no way, but no way, that a product with a 2.5% success rate can be considered fit for purpose.If anyone can challenge these findings with a demo proving that adsites cannot read passwords typed anywhere in the host page, then by all means show me. With the complexity of these subjects I realise that it's easy to be wrong. However, I've been quite thorough in researching this.

author: Ash
author_rank: Regular Contributor
timestamp: October 11, 2017, 03:27:52 am
content: Hi Ian,I've been developing web applications since the mid 90's (yes, the very beginning). I think there are a number of problems with your argument which basically come from a lack of experience with this stuff. This is a complex topic, and I will only give simple answers below, if you want to develop a web site, it is your responsibility to learn this in depth (or employ someone that has) - and honestly most people get it wrong at least a few times. The developer must work with the tools the browsers supply to ensure security.Internet security is built on a number of layers, TLS (the encryption behind HTTPS) is only one on them, the browser implements many layers of defence in depth that I think you should go and read about. Basically it is all layers of trust, starting at the user deciding to visit your site and trusting you, the developer/owner, that you are doing things properly. And yes, this means that it is easy to have untrustworthy sites - protect yourself with knowledge; but without this web of trust, the Internet wouldn't exist, period.TLS / HTTPS has exactly 2 jobs. Authentication of the Server to the Client browser (it is possible to also authenticate the client to the server, but this is almost never done), and ensuring the data over the wire is secure and tamper proof. That's it. nothing more.Yes, older cipher suites are now considered insecure and are deprecated in any good browser. This is normal and was built into the negotiation process from the beginning. Also look at certificate pinning where it is possible to detect that a certificate has been substituted (this can be done in a corporate environment for instance where a new root authority can be created).I believe you are miss-understanding the same-origin-policy.This is enforced very rigorously by the browser at the DOCUMENT level - that is a HTML document that you load. The supposed MITM attach you demonstrate is invalid - you, as the site developer - have SPECIFICALLY GRANTED TRUST to your supposed evil site by including it into the document. This is EXACTLY how the web is supposed to work, it is a very useful and powerful thing. But as you demonstrated - if you aren't careful (as a site developer), you will get pwned, often..If you don't trust a site, you MUST NOT include any resources from it, especially scripts because once you do that script is considered part of your document because you have explicitly trusted it (by writing the script tag).If you DON'T trust the site, but you still want to use the "content" they offer (eg Ads) you must IFRAME it into your site. This creates a brand new document / security scope with the same-origin-policy set up for their domain. Your document tells the browser where to place that document, and that is it. There is no way that iframe document (excluding browser bugs, and some very carefully designed message passing schemes that require cooperation between both ends) can access information in your document.To address issues you raised on your article about HTTPS "failing" to protect a password at the ends - yes, correct it doesn't. Given its role, it makes absolutely no sense to assume that it would.Protecting passwords is your job as a developer. Hint - NEVER EVER EVER store passwords or simple hashes of passwords anywhere. The incoming password should immediately be securely hashed (https://crackstation.net/hashing-security.htm) and securely erased from memory before using this hash to verify identity and issuing a session level identity token. Read about OAuth for instance. If done at scale the "normal" web site doesn't handle passwords at all, only tokens, a separate very carefully audited and protected site manages the user authentication process. Basically these tokens are stored as cookies with various settings to ensure they are only ever sent over encrypted connections to a specific host (or domain related to the host) - the site works with the tools the browser supplies to ensure this token which represents your identity can't be compromised.I should also point out to those reading this, that you shouldn't trust ANY web site too much. Get a secure password manager (I use KeePass, but there are others) and have a different large randomly generated password for each web site. Protect this with a long, strong password in your password manager - a long phrase or sentence is a good start. This way when the inevitable security breach occurs (because developers are people and make mistakes) it is just that one site that is compromised.For instance, I've dealt with local service company that is clearly storing plain text passwords, and even better presenting these passwords to their support people - I've been on the phone to one and the lady commented "oh, wow, that's a weird password" when she opened my profile.. (it was a large random string) WTF.. that is why I use password managers.Hope that help clarify the issues, not start a flame war.I'm happy to discuss the specifics, but it probably doesn't belong on EEVBlog..Actually, if anyone here develops embedded stuff with web sites maybe it does.. These are often insecure (wifi routers, I'm looking at you)Ash.

author: IanMacdonald
author_rank: Frequent Contributor
timestamp: October 11, 2017, 06:25:01 am
content: Well then, Ash, I've been programming longer than you. I've also written programs in industry. You need to see this from the end user's perspective.The end user is not expected to understand every line of the code. The end user does have a right to expect that the code will do what is claimed of it.This also extends to situations where a software product could work under some circumstances, but is being promoted for use in circumstances where it will self-evidently not achieve its goals.A promotional drive is in effect, to forceallwebsites to use HTTPS. The same promoters are advising webmasters NOT to use alternatives such as client side password hashing, because (they say) such approaches are inferior to SSL.This places a degree of product liability on the promoters. It requires that HTTPS should protect passwords on ALL websites. I emphasis the all.If webmasters were left to decide whether the product is suitable for any given role, that would be a different legal situation. They are not.I have demonstrated that it is ineffective in protecting passwords on sites with third party advertising.The end user is meanwhile told by the browser that SSL is doing the job of:1: Authenticating where (all of) the data comes from2: Preventing MITM attacks.I have demonstrated that on sites with ads, it does neither.I can tell you from experience of advising on such cases that legal counsel take a pragmatic view of such matters. They are not interested in 'Ah but you don't have the knowledge to see why the product couldn'tactuallybe expected to do that, even though our promotional material says that it does.' They are only interested in the broad nature of the claim made, and whether the product does the basic job it is sold for.

author: Ash
author_rank: Regular Contributor
timestamp: October 11, 2017, 02:55:54 pm
content: Ian,You certainly have not demonstrated that it is possible to MITM https, or that sites with adsmustexpose credentials or other details to the advertisers.You have demonstrated that it is possible for people to design terribly insecure web sites, especially if they include code from 3rd party sites. You have also demonstrated that sites are able to collude with evil 3rd parties to expose your personal information.This is not the fault of the tool, this is the fault of the owner of the site, the content producer. If you turn off https, your "attack" still works.Your argument is very much like blaming:- an MP4 video codec if the video contains miss-information or lies.- a telephone if a phone scammer convinces you to part with money over the phoneIt isn't the protocol, it is the content producer that is at fault.The drive to get people to use https on all their web sites is good and important:People should not send any credentials or personal information to a non-https web site.If it is sent in the clear, anyone can read it on the network. You claim that this doesn't happen often, I'd say it just doesn't make the news because it isn't normally a company to "blame", just some poor person having their identity or credit cards details stolen because they used the free open wifi at the local shopping mall..You simply can not implement a secure site asking for any personal information or credentials with out encryption over the link. This is what the drive to https is about - ensuring that users KNOW not to send personal information to a insecure web site.If you own a web site that isn't just static text, sorry but tough. Get a certificate and implement https. In fact, enforce https only so that the user is redirected properly to https before interacting with the site (and also ensure all cookies are https only).Is it still possible to attack a web site with https, yes (Security is hard). Does that make https or the drive to move everything to https wrong. no.Your proposal to provide a "hash this input value" in the browser could mask the actual password text sure, but what about my name, date of birth, address, email address that a site asks for? It can't help with that. What about the information leaking about what articles I read, or interests I have, or comments I made?But there are a big problems with client side hashing - it simply does not fix the issue. This is the same if it is done in Javascript or in the browser code itself (your augmented input box). (BTW: doing it in Javascript is a terrible idea, for even more reasons than below).How do I prevent someone reading the hash as it goes over the wire/air? Then simply using it again (replay attack)?For it to work, it must always hash to the same thing so I can check it against my database. The hash is now effectively my password (I don't need the original text, just the hash to sign in), and unless you are using https, you just let anyone sign in as me. Back to square 1. And even worse, now my database contains hashes that can be directly used to sign into the site so if that data is compromised it is again all over.What about the next request? The browser has to either send the hash again, or the server issues you with a session token of some kind. What is to stop someone again on the same network catching that session token and again, replaying it.No encryption means there is no possibility of security. This is what these campaigns are trying to teach people.Ash.

author: IanMacdonald
author_rank: Frequent Contributor
timestamp: October 13, 2017, 05:45:00 pm
content: I have demonstrated that it is possible to MITM HTTPS. I have done so, using two HTTPS sources. Also that it is trivially easy to do so. Jawdroppingly easy. I suspected such for a while, but hadn't gotten round to testing it out.There may be ways of mitigating the attack, and it is possible that sites with advertising are employing them. The issue here is, how do you tell if any given site is using them? If a given site is NOT, then it is not safe to log in at that site.However, you have no way of determining that.Which really puts us back to square one, that we don't know if any given site is safe.Regardless of that, it is unacceptable for a browser to be claiming that all page content came from an authenticated source, when that is patently untrue.The 'Hash becomes the password' rebuttal is one which crops-up in numerous places. It seems to be in the nature of a 'canned response' the promoters of HTTPS have advised their supporters to use. It is BS. Hashes are normally salted They do not become the password.We *KNOW* that passwords get stolen from Web and database servers, and in vast numbers. That, is an incontrovertible fact. Meanwhile, as a justification for the HTTPS rollout it issurmisedthat passwords are being stolen by ISPs, purely because that is technically possible. Being possible, however, does not mean it is taking place.For that matter, my ISP istheoreticallyable to debit any amount they want from my bank card. Being possible, does not of course mean that they do. However, applying the HTTPS argument would require that some way of preventing this activity MUST be implemented, purely on the basis of it being a theoretical risk. I don't see that happening. Why not?Likewise there is nothing to stop my neighbour from picking up her kitchen knife, coming round here and killing me with it. Thetheoreticalpossibility exists, bur I do not regard theprobabilityof that happening as being sufficient to demand any special security measures. If we start trying to cover every possibletheoreticalrisk, the situation soon gets out of hand.On the other hand I have a dashcam because I know there is adocumentedrisk of fraudulent insurance claims, road rage incidents and so on. The risk I am covering is not theoretical.The pragmatist in me says that all this time and money spent on HTTPS is time and money wasted that should be spent on fixingdocumentedsecurity issues. Rather thanhypotheticalones.

author: madires
author_rank: Super Contributor
timestamp: October 13, 2017, 06:15:35 pm
content: A lot of "security" appliances perform SSL/TLS MITM to ensure compliance, to prevent leakage of business secrets or to perform malware scanning. That's also the reason why TLS 1.3 is delayed. Those boxes simply block TLS 1.3 while the IETF working group tries to keep the improved security in 1.3. A fallback option to 1.2 would render the security benefits of 1.3 useless.

author: IanMacdonald
author_rank: Frequent Contributor
timestamp: October 13, 2017, 09:28:30 pm
content: This is true, and I believe that was the issue behind the Lenovo/Superfish scandal.Which just got me thinking: If a proxy of some kind sets up a SSL relay on the main site, that would be apparent to the end user because the 'padlock' info would show the proxy's cert, not that of the visited site. So, easy to detect. However, if it proxied some or alladvertisingconnections but was careful not to touch the main site's data stream, would the user be aware of that? Probably not, because the browser doesn't display the certs for adsites.If so, then there is nothing to stop your ISP from selectively proxying known advertising connections, for example doubleclick. You won't know that the ads are being relayed to you under your ISP's cert instead of the adsite's cert. The proxied ads could be a vehicle for various kinds of MITM nastiness.In which case, if the purpose is to stop your ISP from snooping, then that protection is very easily circumvented.Which would underline that the only dependably secure HTTPS connection is a single origin one.

author: rich
author_rank: Frequent Contributor
timestamp: October 13, 2017, 10:01:49 pm
content: Quote from: IanMacdonald on October 13, 2017, 05:45:00 pmNo, as Ash has already stated, you've only demonstrated that a page which embeds script served from a 3rd-party is vulnerable to attack from that 3rd-party script - Either directly by the 3rd party script or by being compromised and serving up a rouge script. This isn't news.Quote from: IanMacdonald on October 13, 2017, 05:45:00 pmYou don't. Even if the site didn't (or was prohibited) from loading 3rd party scripts you still don't know if it is 'safe'. If you really want to know how secure the client portion of a site is, you'll need to security review/test it for your given definition of 'safe'. But, as users, we don't do that because life is too short. Instead we form a trust relationship with the original vendor. But how do you know whether the page you're viewing is the real site or an imposter (either because MITM or DNS attack)? Perhaps if there was some form of certificate the site could present which would allow the user agent to validate the site's identity?Quote from: IanMacdonald on October 13, 2017, 05:45:00 pmThat isn't what the browser padlock is claiming. It's validating the cert for the original page. It's up to you if you trust that vendor to follow appropriate security practices in all aspects of that site. The 3rd party script issue you seem keen to highlight is only one of many vectors.

author: bd139
author_rank: Super Contributor
timestamp: October 13, 2017, 10:37:34 pm
content: Also with 3rd party scripts, you can set the content security policy headers as perhttps://www.w3.org/TR/CSP/which eliminates that possibility entirely.Set up TLS properly, get an EV cert, pin your certs, set up CSP, set up HSTS.You have to trust both peers either way. The remote peer you have to assume that the CA is valid and did their job. Looking at the fuck ups recently with verisign and symantec, it's all bollocks. The local peer you have to assume that your cert store is valid and that the software is trustworthy.Typically you can MITM ANYTHING easily if you have access to the cert store on the local machine. If you have a trusted CA root and issue a cert on it then sign this, the browsers will trust it (Google are the only people walking away from this quickly and issuing revocations). At that point any intermediate node with that CA authority can issue a cert with the CN set to the host name and MITM your traffic. That's how corporate firewalls work and some insidious malware.SSH is the only way to do this properly. It doesn't have a PKI that requires a third party authority. You can choose to trust them not delegate that trust to an anonymous band of crackpots lie symantec. Problem is it's inconvenient doing key distribution this way and most idiots just click yes to a host key prompt so you're screwed anyway.Ugh it's all a mess and we're going to die one day so everyone just does what they can to tick the Qualys SSL Labs and pen testers boxes, then goes home and rocks in the corner crying.

author: apelly
author_rank: Unknown
timestamp: October 13, 2017, 11:10:26 pm
content: Quote from: IanMacdonald on October 10, 2017, 11:31:55 amNo they aren't.HTTPS protects your communication from eavesdropping and manipulation while browsing the web. Well.https://security.stackexchange.com/questions/5126/whats-the-difference-between-ssl-tls-and-https#5127Troll much?

author: IanMacdonald
author_rank: Frequent Contributor
timestamp: October 14, 2017, 08:51:06 am
content: 1: Almost all ads are served by javascript. You can't serve ads by javascript and not run javascript.If not you'd have to insert the ads directly into your page before it's sent. You could do that statically, but only for a fixed set of ads. In theory, PHP can grab the contents of a foreign page and then inject the HTML into the current page. In doing so it could grep for malicious code and remove it. That would be a lot safer, but it would require a lot more preparatory work by the webmaster. Which is why it's not so common. It would also require that the main site's host act as a gateway for the ad traffic, which would probably not go down too well.2: CSP is so convoluted a topic that few people understand it. Which means, few webmasters will implement it. In any case, I don't think it can control the actions of a script once it is running in the context of the current page. If a page is to allow javascript ads, then it must relax the CSP to allow that. If the ads were run in a frame then it might be possible to sandbox that frame with CSP. Not sure about that.Normally, frames and popups opened from the same site can access theopener's properties, thus running ads in a frame or popup of some kind is not in itself a protection against them spying on the main page. Doing the login in a popup is a protection from ads on the main page, but only because js cannot enumerate windows. Not so with a frame, because theframesetcan be enumerated.The main point though is that since the end user has no way of telling if a CSP is effect, that implies that the 'padlock' does not indicate a safe site.The objective here is to tell the user if the site is safe.3: Quoting propaganda posted on stackexchange proves nothing. If repetition made things true, we'd all be running everything on magnet motors.

author: bd139
author_rank: Super Contributor
timestamp: October 14, 2017, 09:21:02 am
content: CSP is easy. Just RTFM

author: Ash
author_rank: Regular Contributor
timestamp: October 14, 2017, 11:33:27 am
content: Quote from: IanMacdonald on October 13, 2017, 05:45:00 pmOk, lets get this clear - a Man-in-the-middle attack is when a third part can impose themselves in between the server and the client and view all traffic with neither the client or the server knowing.This is only possible in https if a new trusted root certificate authority public key is installed on the client computer. The MITM will then be able to issue a new certificate in the name of the web site signed by the (bogus) new key. The computer will then trust that the fake certificate is legitimate.To do this requires either a corporate network where the machine joins the domain and is issued with these certificates so that the (trusted) corporate filewall can do this to inspect traffic, or a piece of malware has installed the certificate (this includes stupid OEMs installing dumb ass software tools).Quote from: IanMacdonald on October 13, 2017, 05:45:00 pmAgain, you have not demonstrated any MITM attack.You have shown that a browser will, when instructed to by the code on your site under your control include a script from another source. This is the absolutely correct behaviour and how may things on the internet work.The included script from the other site WAS authenticated using its own site's https certificate etc. If this failed it would not have been loaded.An important point here, is that a browser will either refuse to load or warn the user if you ask it to include a script from a non-secure (http) source in a secure (https) page.Quote from: IanMacdonald on October 13, 2017, 05:45:00 pmUm.. this rebuttal that you so glibly put down as a canned response is reality, sorry if it doesn't fit your view of the world.Lets take it step by step.A (secure) hash takes the password and applies either a key or salt to the plain text resulting in the "Hash". Let us assume that this is done in the browserThe browser now submits this hash along my by user name (or similar) to the server over a HTTP connectionThe server receives the hash and user name, and looks this up in its databaseThe server can't do anything with the hash, it doesn't know the password, it has to use what it gotIt therefore can only store what was given (sure it could hash it again, but that doesn't add any security to the password), so it does a check against what it storedUser is "signed in"Sadly for this user, they were using an open WiFi connection at a local restaurant... I'm sitting on the same WiFi network running some tools that allow be to dump WiFi traffic in promiscuous mode.. Hum.. interesting a signing request to insecurebank.com, I wonder...I take the plain text http request I saw on the wire and play it again. What happens?Server sees request, server looks at user name and hash, server validates hash. Bingo. I can be you.The hash is the passwordThe hash can not change. You can't change the salt/key because the server can't "recompute" the hash to verify it. You can't adjust the iteration count over time to maintain a suitable computation challenge (see PKDBF2) Basically proper hashing does a lot of work (designed to take noticable time on a modern computer, but this threshold should be adjustable. If the server is securely passed the password, it can check it against is current hash, then upgrade is internally stored hash to the new configuration to mitigate risks of passwords being broken should they be stolen.If you are sending the hash to the server, what do you do? if you choose to just save the hash in your database, then you have failed. because now If I get your database, all the details I need to directly log in are in that file. If you hash the hash, you are in exactly the same position as you would be if you sent the server the password over a secure channel, so why all the extra work. And the secure channel tells the client and end user that they are talking to who they think they are talking too.Quote from: IanMacdonald on October 13, 2017, 05:45:00 pmOk, ISP are not the problem. They are a business and are protecting their interests - what would happen if it got out that they were doing this? And if your bank lets you sign in without https, you need a new bank, yesterday.You have to worry about the guy in the corner on your free wifi running Kali Linux sniffing all the wifi traffic for fun and profit.Ash.

author: madires
author_rank: Super Contributor
timestamp: October 14, 2017, 11:41:58 am
content: Quote from: IanMacdonald on October 13, 2017, 09:28:30 pmThe problem of "fake" certificates of MITM proxies and rogue CAs can be solved by DANE.

author: Ash
author_rank: Regular Contributor
timestamp: October 14, 2017, 11:51:51 am
content: Quote from: IanMacdonald on October 14, 2017, 08:51:06 amI'd say all of them are.That doesn't mean that you go shoving their javascript into your site with out verifying it. If you don't trust your ad provider, why are you using them?The script doesn't need to be put into the page. You should be able to set up an iframe. The document in that iframe will be provided by them and it will be in their domain as far as the same origin policy works.Quote from: IanMacdonald on October 14, 2017, 08:51:06 amYou simply don't understand how this works. Go and try it.If the content of the Iframe was loaded from the same fully qualified name, sure it will be able to access each other. If the iframe is from a different machine on the same domain BOTH pages can change their domain to a higher level and then be able to access - but note: BOTH sides have to cooperate to do this. (as an examplewww.example.comcontent and static.example.com content could interact if both set their domain to example.com.However, if the content of the document (the HTML page that contains the script tag loading the ad server code) you load into an iframe (using a src tag for instance) is on a different domain, it will not be able to access your page.Popups are similar.Quote from: IanMacdonald on October 14, 2017, 08:51:06 amOk.. um... so, you have a large number of professionals telling you something is the way it is. You have banks and large internet companies and live and die by the trust of their users telling you the same thing.. You have the very very cleaver people that spend their lives architecting these standards telling your the same thing,but you know better, its clearly a conspiracy...We've all had a go at explaining this to you, hoping that you might see where you have gone wrong, but no. Apparently we must all be wrong because we are repeating a "truth" that you don't agree with so you compare us to free energy crackpots.. Really?I think the answer is denied to you because of your ego. I'll not waste any more time on this... believe what you will, the rest of us will continue with reality.Ash.

author: IanMacdonald
author_rank: Frequent Contributor
timestamp: October 14, 2017, 01:08:44 pm
content: No, Ash. What you have demonstrated is that by resorting toad hominem remarks, that you havealready lost the argument.In any case, all of this hinges on site operators taking precautions other than, or in addition to, those embodied in HTTPS. For the moment let's forget the question of whether or not they are effective. There is no way of telling if any given site hasimplementedthese protections, so the bottom line is that you can nevertrusta site with ads on it to be secure against a MITM attack. That is the nature of the problem.

author: rich
author_rank: Frequent Contributor
timestamp: October 14, 2017, 05:42:21 pm
content: Quote from: IanMacdonald on October 14, 2017, 01:08:44 pmIan, whatever you've taken umbrage to doesn't actually devalue all the thoughtful, high quality and detailed technical replies you've received in this thread.I'm struggling to grasp what in this discussion is left in doubt. You want an indicator in the browser for non-tech savvy users to confirm that any site they visit is 'safe'. An admirable goal, but far fetched. Even if the browser could report on the specific scenario of your ad demo, that doesn't tell you if thesiteis 'safe'.Calling your demo code a MITM attack and beating on https for not solving it is wilfully broadening the scope of https to suit your argument that https is snake oil.Quote from: IanMacdonald on October 14, 2017, 01:08:44 pmThe nature of the problem is much worse than that. The bottom line is that, as a user,you can never trustanysite to be 100% secure. But as a user, you trust the origin site to have implemented measures appropriate to the risk and to use trusted 3rd party sources appropriately. I covered this in my earlier post.

author: tggzzz
author_rank: Super Contributor
timestamp: October 14, 2017, 06:00:23 pm
content: There's an old cryptographer's aphorism that is obliquely relevant to this thread, in that it adds an important missing viewpoint..."If you think cryptography will solve your problem, then you don't understand cryptographyandyou don't understand your problem".To some this extent this entire thread is a "how many angels can dance on a pinhead" discussion.

author: xani
author_rank: Frequent Contributor
timestamp: October 14, 2017, 06:33:31 pm
content: Quote from: IanMacdonald on October 14, 2017, 01:08:44 pmSometimes people got tired of ignorance and fundamental misunderstanding of topic of other person that they just do that out of annoyance. Doesn't mean their arguments are not valid.QuoteThat's not a MITM attack tho, that's just ad sites servingbasically a malware.Trojans from ad sites are still served securely over https with no option to MITM that.HTTPS have NOTHING to do with attacks you describe, it's not even in same place of the stackQuoteFirst part of it is true but it is not just "ads". Site can have any number of exploitable errors in it. Second part is bullshit, this has nothing to do with MITM. The "bad" or vulnerable code is running directly onyour machine

author: apelly
author_rank: Unknown
timestamp: October 14, 2017, 07:10:56 pm
content: Quote from: IanMacdonald on October 14, 2017, 01:08:44 pmYes. You absolutely can. That's the whole point.You might not be able to trust the site, but TLS protects your data from observation and manipulation while in transit. That's it.The OP is confused about the problem that HTTPS solves. But you pointed that out earlier.

author: grumpydoc
author_rank: Super Contributor
timestamp: October 14, 2017, 09:34:43 pm
content: Quote from: Ash on October 14, 2017, 11:33:27 amErrr, well, if you implement it that way, however the problem of replayable hashes was solved a long time ago.A more secure approach is that the server sends the client a challenge containing a random key. This is combined with the password and sent back by the client. The server also computes the hash and compares what the client sent, if they match then you are in.Only if you know the password (which never goes over the wire)andthe challenge are you able to compute the hash and the hasing functions are, obviously, one way so that the password (or password as mangled by the random number) cannot be recovered from them.

author: timb
author_rank: Super Contributor
timestamp: October 15, 2017, 12:16:35 am
content: Quote from: IanMacdonald on October 14, 2017, 01:08:44 pmThis shows that you’re the one who’s missing something fundamental.First off, if your site content is being served over HTTPS, but the third party JS is being served from a non-secure source, the user will absolutely see a warning pop up in their browser telling them as much.However, if the malicious ad is also being served up over HTTPs, the data being sent back and forth is still, absolutely, 100% without a doubt protected from a MITM attack.I think the problem lies in your use of the term “MITM”. Your use of it greatly differs from everyone else’s.You’re trying to call malicious JS served up (securely) by a third party advertiser a man in the middle attack, when it’s not. It’s simply malicious code being served up by a third party!Because it’s being served over HTTPS, the user is still protected from the data being viewed or modified in transit by any third party.Again, that’s the key that I don’t think you quite get. HTTPS is designed to protect data while it goes from a server to your browser. It was never designed to verify the content being transferred isn’t malicious. That’s a job for software like AdAware and such.

author: Mr. Scram
author_rank: Super Contributor
timestamp: October 15, 2017, 02:05:24 am
content: Quote from: Red Squirrel on October 10, 2017, 02:15:57 pmHow do yo figure they store every packet? Purely from a logistical point of view, that's pretty much an impossible task, even with the massive data centers these agencies have.I know there is some evidence that it's done in the case of a handful of specific countries, but not every country.

author: Ash
author_rank: Regular Contributor
timestamp: October 16, 2017, 01:20:56 am
content: Quote from: grumpydoc on October 14, 2017, 09:34:43 pmOk, fair call if you have a shared secret you are trying to prove one or both sides share (this is a common enough technique, but doesn't belong on a web page in Javascript), but you are still left with the question of what is stored on the server in the case of a password/credential?Both ends need to be able to create a hash of the data in question. Lets assume this is a password. Now to do this properly, we should be using something like a HMAC structure (there are known weaknesses in doing just HASH(KEY + SECRET) or HASH(SECRET + KEY), but that is not relevant here).What does the server store in its DB to be able to perform this hash? We need to be able to do the same computation as the client. There are 3 options that I can see.Store the password plain text so the server can compute HMAC(Key, password)Store the password encrypted so the server can compute HMAC(Key, DECRYPT(decryptKey, password))Store HASH(password), the client then computes HMAC(Key, HASH(password)) which can again be verified on the serverThe first two are bad simply because we now have unlimited access to the password on the server and I believe that we all agree that storing passwords is regarded as poor form. So, we are left with the third option, storing a hash of the original password.But for all three options, we are now stuck - how do we securely send this password/hash to the server without exposing it when I create my account or change my password? If I can see you send that original password hash I can now correctly respond to any challenge key you issue from the server. This HASH(password) is now again effectively the password.At some point you will need to securely send the password (or hash) to the server so all the additional code you have used to hash the password on the client still relies on a secure connection.So just do it the way it was designed in the beginning - https because this provides additional levels of security (authenticating the server to the client). How does your Javascript hashing code determine that it is talking to the correct server? I can setup a proxy between you and the server and truly Man-in-the-middle this process.Now the next problem is how are you maintaining the "session" state? normally this is handled in a cookie (or a query string argument) but access to this cookie is now access to your identity. Again without HTTPS encryption, there is no way to protect this data and again, If I can get this cookie, I can access your account on the server.This is the problem that the HTTPS-all-the-thingz campaigns are about fixing.I see these issues all the time. There is a lack of understanding by a lot of web developers, even of quite popular systems used by many sites.For instance, even the forum code used here on EEVBlog will let you happily sign in using HTTP, then proceed to issue you a session token. Again all over HTTP, using some javascript client side "hashing". It simply isn't secure. Make sure you use HTTPS. If you do sign in using HTTPS, the forum software doesn't seem to set the HTTPS only flag on the session cookie, so if you then hit the server with HTTP it is still sent and your account exposed.The thing is, you as the user have to make a call about what information is important enough to protect. Personally, there is very little information about me here and I use large random passwords which are all different for all my accounts so I'm not too worried about leaking my password. I make the judgement that I get more utility using the site than what I risk.Ash.

author: IanMacdonald
author_rank: Frequent Contributor
timestamp: October 16, 2017, 10:40:04 am
content: The basic principle behind devising a (reasonably) secure system is that:All passwords are hashed using a salt, at the client. This can be a 'site secret' which applies to all accounts, or it can be based on a particular (unchanging) feature of the account such as the user's name or DOB.At signup this hash has to be sent to the server, which does represent a slight security risk. Encryption could be used to mitigate this. However, signup occurs only very infrequently so the chances of interception are small.At login, the client requests a nonce from the server.The client hashes the user's input with the both the site secret and the nonce, and sends it.The server gets the database password field for this user and hashes it with the nonce. If this equals the submitted value, the correct password was typed. The nonce is then discarded.Consequences:The hash seen on the wire at signup will not log the user in.An intercepted login hash will only work once, and only if the MITM can beat the user to logging-in, because the nonce will not be accepted a second time.A weakness is that the values stored in the database would work for login if the hacker understands the nonce mechanism, and can request one. This can be prevented by reversibly encrypting the stored values. Of course the hacker might be able to pinch the encryption key too, but hey, if he owns the server to that extent, why does he need a login anyway?No process is ever 100% secure, the best you can do is make it hard enough for a hacker that they will decide to try somewhere easier.

author: bd139
author_rank: Super Contributor
timestamp: October 16, 2017, 11:08:54 am
content: You shouldn't salt with a known value. See Common Weakness Enumeration CWE-760:https://cwe.mitre.org/data/definitions/760.html... Salt should be per credential from a hardware RNG. If someone finds out how to generate the salt from the data then the salt serves no purpose.HMAC signs the body and performs authentication so you know it's not been tampered with and that the originator is authentic. It doesn't however protect the body in transit. That's what TLS is forOur API stuff uses HMAC signed requests/responses over TLSv1.2.

author: Mr. Scram
author_rank: Super Contributor
timestamp: October 17, 2017, 09:09:47 am
content: Quote from: bd139 on October 16, 2017, 11:08:54 amA salt is historically intended to prevent the use of rainbow tables. That way, generic rainbow tables aren't enough to compromise your environment. Deducing your salt and creating targeted rainbow tables is considerably more work, which should provide a level of protection.Obviously, creating unique salts provides even more protection, but you could argue the same way that if someone finds out your salt list, they serve no purpose. Both already mean a significant breach of security.

author: bd139
author_rank: Super Contributor
timestamp: October 17, 2017, 09:27:49 am
content: That's not where your responsibility ends. You need to plan to lose your identity, the hash and the salt. If your salt is predictable than it makes it considerably easier to attack the passwords. After that you can then leverage the credentials elsewhere where users have reused credentials. The whole point of the salt is to make that computationally more difficult.If you're deriving the salt source from anything that isn't random, then there's no point in having a salt.There's no such thing as secure or more secure. It's boolean, and only while the mathematics are intact.

author: Mr. Scram
author_rank: Super Contributor
timestamp: October 17, 2017, 03:34:09 pm
content: Quote from: bd139 on October 17, 2017, 09:27:49 amPatato, patato, I guess. You could be saying that means my 4 character password isn't anywhere near as secure as your 4859 character password. You could also be saying that both are equally secure until the actual breach. Both could arguably be true.

author: Bud
author_rank: Super Contributor
timestamp: October 17, 2017, 04:10:54 pm
content: Quote from: timb on October 15, 2017, 12:16:35 amWhat timb said.HTTPS provides point-to-point protection of traffic. Once your traffic hits the remote server it pops out in clear text and the remote web server sees it as clear text and therefore can insert whatever it wants in its response back. As far as HTTP is concerned it is outside of its scope. Think of a robust hard pipe that connects two buckets. You can pump gold through the pipe or you can pump shit, the pipe does not care. However the pipe will make sure nothing leaksbetweenthe end points.

author: Marco
author_rank: Super Contributor
timestamp: October 17, 2017, 05:16:23 pm
content: Quote from: madires on October 14, 2017, 11:41:58 amGoogle, Mozilla and Microsoft "independently" closed feature requests for DANE with WONTFIX. Security wise EV certificates with pinning/transparency are almost as good, but cloud services have pretty much killed those. The hardware nature of EV certificates clash with the cloud concept and no one has bothered fixing it.This is all of course a complete coincidence. HTTPS is definitely not security theatre as the WPA2 hack shows, but the CA clusterfuck does seem custom designed to allow security agencies to abuse it ... as long as they are willing to throw a CA under the bus to do so.

author: madires
author_rank: Super Contributor
timestamp: October 18, 2017, 02:32:25 pm
content: That clearly shows how much they care about security and privacy.It's a tradeoff between supporting the middleboxes (plus law enforcement) and providing secure communication to users.

author: cdev
author_rank: Super Contributor
timestamp: October 18, 2017, 02:48:32 pm
content: The entire process should be transparent in a way it has never been.All OS defaults now seem likely to be badly chosen and likely to be insecure. Complexity adds likelihood of insecurity and should be minimized, not maximized.