How do clients extract the public key from a self-signed certificate? I have gone over:

Why do we need to sign the self-signed certificate?
Basic self-signed certificate questions
Is a self-signed SSL certificate much better than nothing?

I am new to cryptography and security. I am trying to understand the point of self-signed certificates.
I understand the point of certificates as a means to distribute the public keys where a trusted CA (issuer) "signs" (=>encrypts) the public key of the subject and the client that trusts this CA uses the public key of the issuer (CA) to decrypt (un-sign) the certificate and get the public key of the subject (server). I have understood this from Data Communication and Networking by Behrouz.A.Forouzan 4th.edition Page 987.

Now, if my understanding is right, how do I extend this logic to a self-signed certificate! If the subject matches the issuer and the overall purpose for the client is to obtain the public key of the subject (server), how does the client decrypt (un-sign/open) the certificate using the public key of the issuer to get the public key of the subject as the public key of the issuer is the public key of the subject? How does the client get the public key of the subject (server) that it ultimately wants to establish an SSL session with?
The fact that this question has not been asked anywhere tells me that there must be something really basic that I have not understood. Please help me in understanding this.

A few major misunderstandings here.
First of all, that book is just wrong. Signing is not a form of encryption, nor verifying a form of decryption. In the specific case of RSA, signing is mathematically the equivalent operation to "encrypt with the private key", but this is more a coincidence than an inherent property of digital signatures. Some digital signature algorithms don't support actual encryption at all! (DSA and ECDSA being the obvious ones.) Conceptually, the two operations (sign vs. encrypt) are used very differently, and practically there are many considerations for an implementer (padding, length, etc.) that apply to one operation but not the other, even for RSA.
Second, signing is never done in a way conceals the original message. This may be another way that book is misleading you: all signing operations, conceptually, take two inputs (a message and a private key) and produce one output (a short-ish string / huge number that is meaningless without the original message). The inputs - including the message - are not modified. Usually the actual operation of signing can only be done on a short piece of data, so the message is cryptographically hashed first (this is usually done automatically by the crypto toolkit when you say "sign M with K"), but some algorithms instead work on the whole message and internally are doing something akin to hashing while also producing the signature. From the user's perspective, this is still "sign M with K", just for a key K used with a different digital signature algorithm.
In any case, the practical output of a signing operation is that now you have two pieces of data - the message M and its signature S - in addition to your private and public key pair. Anybody with M, S, and the public key can verify that S was created from M using the corresponding private key if neither of M or S are modified, so of course you have to distribute M and S unchanged. Note that S is usually much smaller than M, and it is conceptually impossible to extract M from S.
Now, in practice, signing is often (not always!) used with encryption. When encrypting, you do modify M - or at least, you produce C, the ciphertext of M, and keep M secret - and then either the recipient must decrypt C (to get M) before they can verify the signature S, or you sign the encrypted message C rather than signing the plain text M and thus your S is verified against C not against M. Nothing in this paragraph is relevant to certificates, because certificates do not have any encryption!

With those cleared up, hopefully you understand why the question is trivial. A certificate consists of (arguably) four parts: data about the cert (when it was issued, by whom, for how long, for what purposes), data about the subject (who the cert is issued to), the subject's public key, and the issuer's signature of the previous three parts. The first three parts are plain text, and the last - the signature - is meaningless without that plain text.
So, you extract the public key from a self-signed certificate - or any other part, from any certificate - by parsing the certificate structure and reading the relevant part of the certificate (the one that contains the public key). That's it. The act of signing does not, and must not, modify any part of the certificate (except the part where the signature itself is stored).


Is there a documented security standard that forbids or discourages rolling your own crypto? Is there any security standard published by NIST or another reputed body in information security that explicitly forbids or discourages rolling your own crypto? If yes, would you please post the standard name/title, a link to it, and quote the section that says something to this effect?
Such a document will help me to define information security policies for our organization.

Perhaps you are looking for FIPS 140-2.
The Federal Information Processing Standard Publication 140-2, (FIPS PUB 140-2), is a U.S. government computer security standard used to approve cryptographic modules. The title is Security Requirements for Cryptographic Modules. Initial publication was on May 25, 2001 and was last updated December 3, 2002.
FIPS 140-2 establishes the Cryptographic Module Validation Program (CMVP) as a joint effort by the NIST and the Communications Security Establishment (CSE) for the Government of Canada.
The key here is that it defines Validation of acceptable crypto. It does not explicitly forbid roll your own or custom encryption. You are welcome to make your own and submit it for testing. If it achieves FIPS validation then you can use it with justification. Note that FIPS validation is a years long multi-million dollar process, but you certainly can pursue it if you like.
To back track a bit. The phrase "Roll Your Own" may mean the algorithm (see FIPS 140-2), or it may mean implementation of an approved algorithm. Problematic implementation of an approved crypto algorithm is surprisingly easy to do. One trivial example is using the wrong pseudo-random number resource and severely crippling an otherwise strong crypto. Always try to use vetted libraries instead of coding your own. Even then it's not guaranteed.


What are the basic implementations that make a network secure from the inside\outside? Would you consider the following to be "basic hardening items" for network equipment and servers?

VLANs
Access Control (such as NTFS permissions)
Patch Management System
Host Firewall
Anti-Virus

What is the minimal protection needed to segment my internal network from the DMZ and the outside? Is having a WAF, assuming you're dealing with web applications, and default denies, with tightly confined rules, on the firewalls sufficient?

The most basic principles for making a network secure are 'defence in depth' - that is, you want to put as many things between your 'crown jewels' and potential attackers as you can - and 'minimal surface area' - that is, you expose as little as possible to attack.

To get there; your list of basics is pretty good at addressing 'defence in depth', though you may want the ability to blacklist/whitelist applications.

You should have a look at minimal surface areas - turn off admin privileges, unused services, blacklist any application running outside of c:\windows, c:\program files, etc., block executables on USB, turn off web access on machines that don't need it, etc.

Finally, the Australian DSD guide at http://www.dsd.gov.au/publications/Top_35_Mitigations_2012.pdf is a very good 'checklist' for this sort of thing - start at the top and work down.


How to check whether the cookie is set securely I found solution for setting asp.net_sessionid cookie as secure.
The solution is to make changes in web.config file as given below.

<\system.web>

I want to know if this change is setting the cookie as secure for sure.
How can i check that ?

Easiest route is to use a traffic interceptor, such as burp proxy, which lets you read cookie parameters. There are various other proxies which will do this as well.

This OWASP page has further information:

https://www.owasp.org/index.php/Testing_for_cookies_attributes_(OWASP-SM-002)


Trying to get a grasp of approximate risk level of CMS admin logins over http Forgive me for a potentially obvious question--and I know the safe answer is "Always use https!"--but I'm trying to get a grasp on how necessary it is to use https for anyone accessing the backend on CMSes like Drupal, WP, etc. I was wondering if anyone can clarify a few things.

My understanding of man-in-the-middle attacks is that they are relatively uncommon over networks with basic security measures (such as home networks, password-locked Wi-fi networks, office networks). Basically, there are no easy access points between, for instance, my home and my ISP, where someone can setup a system to intercept traffic.
Of course, open wi-fi networks provide an easy entry point to intercept traffic, but even large password-protected networks such as hotel internet connections, etc., provide another easy access point to intercept the traffic of anyone else accessing the internet through the same network.

Assuming these two things are correct, I would make the following deductions:

It would seem to me that logging into a CMS via http under any sort of administrative account is generally safe (though perhaps not a best practice) from home and from the office, as long as there is no open wi-fi and, of course, no malicious users using systems within said home or office.
This would also mean that if you are logging in to a CMS in such a capacity over a hotel connection, in-flight connection, or anything of the sort, you should always use https. I'm pretty sure this is obvious, but just asking to clarify, as security is not my specialty.

Can anyone tell me if I am basically on the right track here, or if any of my understandings or assumptions are flawed in any way? Basically, what I'm trying to determine is whether it is pretty necessary to always have CMS-based websites setup using https, or whether plain old http is, generally speaking, safe enough for sites on which administrators log in only from networks where all users are trusted.

It's always good to have encryption over a CMS-based website. I would recommend to always use HTTPS because you never know who is intercepting traffic and what their intentions could be. For instance, if someone was intercepting your traffic and performing MITM attacks; they could respectively grab your login if they decided to and could pretty much try this login with other accounts of your own etc etc.

Although all users can become trusted as you are implying in your question, I would still use encryption. Anything can happen. I don't know if you have heard of an HTTPS exploit called 'BREACH'. This is a security exploit upon HTTPS that uses HTTP compression. More about it here: https://en.wikipedia.org/wiki/BREACH_(security_exploit)

You're definitely on the right track, I don't see flaw within your knowledge and you're definitely asking the right questions to build your base.


Is it possible to put a certificate as trusted CA into Windows, given only physical access to system drive I have a physical access to HDD with "target" Widows 10 Pro installation. I can't login into this system as admin. I need to make the system trust my own CA. Is it possible to somehow put the certificate as trusted into windows while it's off, and then, when booted, have it trust that certificate?

I know Windows stores it's certificated in Registry. By looking into it, it seems like it is in REG_BINARY format, a blob of a certificate. I could load a system hive from the target disk via different windows installation that I am an admin off, and theoretically, I could put entries into it. The question is - do the blob for the same cert in registry differ from one Windows installation to another. Any hint on the matter would be much appreciated.

Additional info: The disk is not encrypted, I can modify files on it, I can see the Windows folder, and write to it.

You can do this. First, look at the certificate properties and locate the thumbprint value. For an example, let's look at an existing certificate which you can do this using the MMC Certificates snap-in:

Select Local Computer and then browse to the Trusted Certification Authorities section to view the certificates there:

Looking at the properties, locate the Thumbprint value:

Now, open Regedit.exe and navigate to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SystemCertificates\AuthRoot\Certificates and find that thumbprint in the list:

Right-click that value and export it to a file. Next, mount the other hard drive and load the HKLM registry hive and import that value. The certificate should now be trusted.


Should the average user care about DNS security? Denise is an average user who goes with the default settings of her OS and home router. She knows that her browser and email client use DNS and she's worried because I told her that the DNS protocol has no built-in security.

Are there sensible security measures that Denise should take to increase her privacy and security when using DNS? And what do these measures protect her against?

Examples: Should she switch from the default DNS servers provided by her ISP (e.g. to OpenDNS)? Should she, as an end user, care about DNSSEC or DNSCurve? Should she adopt DNSCrypt? If she consistently browses via HTTPS, does she have to worry about DNS attacks at all? Anything else to consider?

The average user, in my opinion, shouldn't spend any time whatsoever thinking about DNS attacks.

First, it just doesn't happen on a scale that warrants such attention. When phishing attacks, ransomware, and password guessing are rampant, users are infinitely better off spending what little security motivation they have on preventing those.

Second, an attack involving a malicious DNS server realistically means you're connecting to a rogue access point. If you're using TLS for as many things as possible, you greatly mitigate the amount of damage that can be done in that scenario anyway. For anything that doesn't go over TLS, the problem is better solved by the developer of the application getting their act together, and unlikely to be solved by encouraging users to switch to something else.

So in practice I'd really say it boils down to "use TLS for everything", which insulates users from so much attack surface it's frankly absurd that there are still holdouts on this front. While it's not perfect, it at least greatly increases the difficulty of most attack scenarios and seriously hampers most non-targeted attacks.


Turn exisiting USB drive into hardware encrypted drive? I'm currently using a completely unprotected SANDISK Ultra 256GB and want to secure my files on there. I made a quick google search and found a device called Skydigital Ezsafe Lockdock which simply encrypts your harddrive.

So is there a USB-Stick version? Is there an adapter which encrypts your existing USB-Stick and adds a fingerprint module between USB-Port and your stick and your stick is encrypted without this module?

Why not use BitLocker? BitLocker To Go is exactly what you need. Allows for quick and dirty encryption on removable drives. Instructions included below.

http://infosec.nmsu.edu/instructions-guides/how-to-enable-bitlocker-to-go-for-external-hard-drives-and-usb-flash-drives/


Is email less secure than voicemail, snail mail, or internet portals for transferring sensitive information? Where I am in the US, health care providers refuse to send anything through email (e.g. even something as trivial as an appointment reminder). Often they claim that email is not secure.

Instead they will send me information (e.g. appointment reminders, messages from my doctor) through voicemail, snail mail, or through some clunky and badly-designed internet portal you have to sign up for.

Is there any evidence that email is any less secure than these other modes of communication? It would seem to me that the exact opposite is true - e.g. snail mail could get lost, or get stolen by anyone willing to stroll over to my mailbox (at least in most of suburban America where there is no lock and key for mailboxes).

Yes, email is insecure, and should not be used to deliver sensitive data.  That doesn't mean that they should need to refuse to send anything via email, but given HIPAA and the potential penalties for non-compliance, I'm not surprised that some providers would choose to err on the side of extreme caution and shun email entirely.

Here is why email is insecure:  The provider does not control the entire channel.  Once an email had left their system, they have no way of guaranteeing that it will be encrypted and protected from prying eyes as it wends its way through the Internet and to your inbox.  In fact, with the state of email today, its quite likely that it will not be encrypted along the entire transit path.   Hence, if they were to send PHI via email and it was stolen or misused, it would not be unreasonable to argue that they failed to show due care and should have known better, thus, be held liable for the leakage.  When you're looking their web site in a browser, they can confidently assert that the information is being delivered securely to your screen by using HTTPS.

This risk can be (virtually) eliminated through the use of encryption such as PGP.  Do you know what the uptake of PGP is?  Percentage-wise of all email-users, in the low single digits at best.  And even those users tend to hate it and view it as a necessary evil.  That may change in the future, but that's the reality today.  Do you know what the uptake is for HTTPS use over the web?  100%.  Every browser supports it, and it's completely painless for the end user.  Use what works for your users.

Is snail mail more insecure?   No.  Why not?  Because the piece of mail has a single physical location, and can't be stolen without the physical presence of, and personal risk to the thief.  Stealing things off the Internet is easy.  That network captures trap, and mail servers log unencrypted email is not so much a risk as it is a given.  Snail mail is very unlikely to be intercepted.  The USPS has an entire law enforcement division to protect it.  They have a stellar (not perfect by any stretch of the imagination, but very very good) record of delivering mail to the intended recipients.  It is reasonable for a company to entrust the USPS to deliver confidential information to you unmolested.  It is not reasonable to expect the same from SMTP email.

So, yes, the fact that they won't provide any information to you over email is excessively conservative.  The fact that they won't provider PHI that way, is certainly not.


How to protect against injection into a 3rd party REST API? I have a 3rd party API that I need to connect to and they allow queries that can be formed by changing url parameters.

E.g. /data?startDate=2019-01-01 or /data?name='John Doe'

It is easy to form these queries by concatenating strings, but input cannot be trusted (our API is exposed to end users and accept parameters that have to be sent to the 3rd party API). That means that anyone can try to inject something in our URL and then it will be sent to the 3rd party.

If our API was talking to an SQL server, we could have used prepared queries to prevent injection, but how to do that with a URL?

It's possible to check that e.g. date is just date, but what if a custom string like name is allowed?

Example:

A query to a 3d party API: /data?$filter=(userId eq 1) and (startswith(someField, {userInput})) eq true

User input to my API: userInput="someText) or (userId eq 2"

In general the answer is: you can't.

The best way to protect against injection attack is to separate data and commands.  When talking with an SQL database, this means using prepared queries.  When updating the DOM, this means using javascript methods that tell the browser "this is data to display and should not be interpreted as HTML".

Unfortunately this is one particular "backend" for which that simply is not possible, since you only have one communication channel.  You definitely can and should use proper URL encoding so that an attacker can't use your system to feed extra parameters to the third party but, for instance, there is no great way to stop an SQLi payload from moving through your system.

Certainly you could do what some do and implement a WAF.  I.e. if you see SELECT, 1=1, or other common SQLi payloads then you can refuse to pass along the request.  In general though blacklisting is not an effective security measure (there are plenty of questions on this site about bypassing WAFs) and it also tends to get in the way for regular users through false positives.  As a result I don't really recommend it.  Indeed, you don't even necessarily know what technology they use on the backend.  You might assume an SQL variant and try to detect SQLi payloads, when they are really operating a NoSQL database with a completely different syntax and vulnerabilities.  Therefore you may end up protecting against the completely wrong "kinds" of things.

All that to say: you really have no choice except to trust that the third party is practicing proper security themselves.  Which is really the only answer anyway, since as a general rule of thumb you have no control over the security of someone else.


authby=secret serve as alternative to AH in ipsec.conf I have employed strongSwan U5.5.1 for installing an IPsec tunnel between two Debian hosts with a firewall in between. My ipsec.conf specifies authby=secret but not ah=. The firewall currently permits ESP (protocol 50) and UDP/500 (ISAKMP), UDP/4500 (NAT-T), but not yet AH (protocol 51).

My question is this: if I want to prevent men-in-the-middle attacks, does authby=secret suffice or do I have to activate AH is well? In other words: do authby= and ah= address the same or different kinds of authentication in the IPsec protocol stack?

These options serve a different purpose.

authby specifies an authentication method that is used during IKE. secret sets it to pre-shared keys (PSK), which need to be strong to be secure (they are otherwise vulnerable to active attackers). IKE security associations (SAs) are used to negotiate ESP or AH SAs between the two authenticated peers.

ah causes the negotiation of an Authentication Header (AH) SA, which can be used to integrity protect and authenticate actual IP traffic. However, the AH protocol does not provide confidentiality, i.e. traffic won't be encrypted, and it won't work over NATs because the addresses in the IP header are included in the Integrity Check Value (ICV).

For both confidentiality and integrity protection/authentication use the Encapsulating Security Payload (ESP), which is the default in strongSwan (specific algorithms can be configured using the esp option). It doesn't protect the outer IP header, like AH does, but it works over NATs.

Note that strongSwan does not support the combination of the two protocols, that is, the ah and esp options are mutually exclusive.


What is Non-Repudiation of content? How can we combat it? Can anyone explain precisely the term of non-repudiation of content (eg. of email)? What sort of security requirement is used to combat this? Perhaps a digital signature?

This is how I understand the concept:

If the nurse emailed a critical information about a patient to the hospital, the hospital admin staff need to be able to prove about the content of the email in any time. This would be useful if the nurse happens to make a mistake about the condition of the patient, which results in a death of the patient.

Does this sound correct?

There are 4 basic principles used in cryptography:

confidentiality
integrity
authenticity
non-repudiation.

I will not go through the mathematical aspects involved, instead I will try to offer a more general description.
Confidentiality is assured by the encrypting of data, integrity and authenticity are assured by a signed digest and non-repudiation is assured by the linking of the above operations to a digital certificate.

The owner of the certificate is responsible for every action performed using that certificate.

The best example I can think of right now for non-repudiation is that of the law of digital signature. This law  states that a digital signature is in every way equivalent to the signature made by hand by a person. Usually, a trusted time source is also involved, so that the time of the signing is also trusted.

All these ensure the fact that if you digitally sign the document, you cannot deny the signing, because of the fact you are the owner of the certificate and because there is a trusted time reference.
It is the same as signing a contract for a loan at the bank. You cannot say you were not the one that agreed to the contract, because they have the proof (the handwritten signature).


Does every hash value have an inverse value? There are many different hash functions, md5, sha, and others. They take a value V and produce a H via transformation Function(V) = H, where Function is md5, sha, etc.

My question is: Does every hash value H have a value V?

For example, given md5 hash value f2c057ed1807c5e4227737e32cdb8669 (totally random), can we find what it was from?

In other words, if we list all hashes:

00000000000000000000000000000000
00000000000000000000000000000001
...
fffffffffffffffffffffffffffffffe
ffffffffffffffffffffffffffffffff

Can we find a value for each one of them?

Edit (from OP's comment): I want to know if there exists an input for each possible output. I'm not interested in finding inverse

My question is: Does every hash value H have a value V?

  For example, given md5 hash value f2c057ed1807c5e4227737e32cdb8669 (totally random), can we find what it was from?

These are actually two very different questions: whether there is an input for each output, and whether we can find an input for each output.

For your first question: we do not know. For a given hash function, the number of possible inputs is a lot bigger than the number of possible outputs, so we would find it very surprising if the function was not surjective (i.e. if there was an output without any matching input). For instance, with MD5, there are 2128 possible outputs, and 218446744073709551616-1 possible inputs, so we expect that each output has, on average, about 218446744073709551488 corresponding inputs. It is rather implausible that there is an output with no corresponding input.

However, we do not know how to prove it. To a large extent, we expect that property to be very hard to prove for any concrete hash function (such a proof of surjectivity would not, per se, be a weakness of the function, but, handwavingly, the security of a hash function relies on the intractability of its structure with regards to that kind of analysis).

For your second question: we strongly hope that it is infeasible. This is what is called preimage resistance: for a given output y, it should not be computationally feasible to find a x such that h(x) = y. Even if it was mathematically proven that such an input must exist (it is not proven, but it is strongly suspected, as explained above), it should still be atrociously expensive to actually find it.

If the hash function is "ideal" then the best possible attack is luck: you try out possible inputs until a match is found. If the output has size n bits, then "luck" should work with an average effort of 2n; with n large enough (and n = 128 is large enough), this is way out of what can be done with existing computers. A hash function is said to be resistant to preimages if "luck" is still the best known attack.

It is known that MD5 is not ideally resistant to preimages, because an attack with effort 2123.4 has been found -- about 10 times faster than 2128, but still a long way into the realm of infeasibility, so that attack is theoretical only.

I would like to point out two things, though:

If I give you h(x) for an unknown x, but you have some idea about the x that I used (e.g. you know that x is a "password" that a human user could choose and remember), then you can try possible x values that match that idea. The actual average effort for finding a preimage to h(x) is the smaller of 2n and M/2, where M is the size of the space of possible inputs x. For a "raw" preimage attack, all sequences of bits are possible inputs, so M is huge and the cost is 2n. However, if, in a specific context, x is known to be taken out of a relatively small space, then finding the "right" x can be substantially faster.
As pointed out above, preimages are not supposed to be unique: for a given y, a large number of values x should exist, that hash to y. Thus, for a given output, you do not find "the" corresponding input, but "a" corresponding input, which may or may not be the "right" one.

And the third question ? Indeed, trying to make a complete map of inputs-to-outputs is yet something different.

As @Xander pointed out, the number of possible MD5 outputs (2128) is too large to be stored anywhere on Earth; it exceeds the cumulated size of all hard disks that have ever been built. However, if you could solve that storage issue, then it is possible to think about the cost of making a complete map.

If you use the "luck" attack on all 128-bit outputs independently, you may expect an overall cost of 2256 (2128 times a cost of 2128). However, you can do much better by handling all outputs simultaneously, i.e. trying out random (or sequential) inputs and simply populating your big table as you obtain the outputs. With effort 2128, you should get about 63% of your complete map, while the "independent luck" method would need an effort of a bit more than 2255 to achieve the same result.

Edit: as was pointed out by @Owen and @kasperd, the arguments on the number of inputs are not actually sufficient to induce surjectivity; the internal function structure matters. MD5 and SHA-1 are Merkle–Damgård functions, meaning that they are built as follows:

There is an inner pseudorandom permutation P: for an input block b of a given size (512 bits in the case of MD5 and SHA-1), Pb is a permutation of the space of sequences of exactly n bits (n = 128 for MD5, n = 160 for SHA-1).
A compression function is defined, as:

    f(b, x) = Pb(x) + x

That is, for block b, we apply the permutation corresponding to b on the second input x, and then we "add" x to the output of that permutation. (In the case of MD5 and SHA-1, that addition is done on a 32-bit word basis, but the details do not matter here.)
To process a complete input message m, the message is first padded with extra bits so that the total size becomes a multiple of the block size, and also encodes the original message length. The padded input is then split into successive blocks b0, b1, and so on. A register r is initialized to a conventional value of size n bits (the "IV", specified in the MD5 and SHA-1 standards). Blocks are then processed one by one: to inject block bi, we compute f(bi, r), and the output is the new value of r. When all blocks have been processed, r contains the complete hash function output.

The addition step in the compression function turns the pseudorandom permutation Pb into a pseudorandom function. A relevant consequence is that, for a given value b, f(b, x) is very unlikely to be surjective. In fact, we expect values f(b, x), for all 2n inputs x, to cover only about 63% of all possible 2n sequences of n bits.

This processing has interesting consequences. First, consider all inputs of size exactly 1 GB (the "traditional" gigabyte of exactly 1073741824 bytes): there are 28589934592 such sequences, i.e. a lot more than 2128. However, when applying MD5 on all these messages, they will all be padded with exactly one extra block (8589934592 = 16777216×512, so an extra block of size 512 will be appended), and, furthermore, this final block will be the same for all 1 GB inputs (it encodes the input length but is otherwise deterministic, with no randomness and no dependence on the values of the input bits). Let's call bz that final block. MD5 on a 1 GB input message m thus begins with a lot of processing on the first 16777216 blocks, resulting in a 128-bit value x, and the hash output MD5(m) is equal to f(bz, x).

Therefore, all 1 GB messages ultimately reduce to a single of the same compression function f(bz, x). Thus we expect the hash outputs to cover only about 63% of all 128-bit sequences. This example demonstrates that the argument on the number of inputs to the hash function is incomplete (though it gives the right idea).

Conversely, if we consider all messages of exactly 300 bits in length, then they will all be hashed as f(b, IV), with 2300 distinct blocks b. We thus have 2300 pseudorandom permutations Pb, all applied on the same 128-bit input (the IV), yielding 2300 128-bit results which are all, then, supposed to be uniformly distributed over the space of 128-bit values. Adding IV to all of them does not change that uniformity. In that case, the counting argument works and thus sujectivity becomes highly probable.

Edit 2: About the "63%". When you generate a random value, uniformly, in a space of size N, then the probability of hitting a given value x is 1/N; thus, the probability of not hitting a given value x is (N-1)/N.

Now try it N times: you generate N values randomly, uniformly and independently (in particular, you may generate several times the same value). For a given x, the probability of not being part of these N values is the probability of having been missed N times, i.e.:

    P = ((N-1)/N)N.

This can be approximated as follows:

    P = eN ln (1-1/N) = eN(-1/N + o(1)) = e-1+o(1)

Thus, with big values of N, the probability of any given value being missed approaches 1/e. Therefore, the expected coverage of the space of size N, with N random values, would be close to 1-(1/e). This is approximately 63.21%.


Are there ways in determining if devices such as router have been tampered with? In working with a number of non-profit organizations, devices such as routers that may have been used by other businesses are often acquired. Equally, devices are often inherited through other channels e.g. donations

What are the methods if any in determining if devices such as routers have been tampered with both from a hardware and software perspective?

The threat model includes the following scenarios;

Compromised operating system
Modifications to firmware e.g. modding
Malware infections e.g. VPNFilter

Does resetting the device overcome 1 and 3?

Can monitoring traffic on the device help identify malicious traffic that may be an indication of a tampered device?

Monitoring may or may not uncover malicious traffic.

What you will want to do is to reflash the firmware with a known good version, either from the manufacturer or an after-market one (e.g. DD-WRT, or Tomato). This takes care of the OS, any modifications, and likely the infections.

The after-market option might be something to consider so that all your devices are running the same OS and the same configurations.


How can you find the PHP version a website is using using LFI? If a website is vulnerable to Local File Inclusion (LFI), how can you use it to find out the PHP version? Is there any file which says the version of PHP being used. I'm trying to do a PHP sessions LFI to RCE attack, but I don't know where the session files are stored. I think finding the PHP version will help.
It has to be a file that is already there because there is no way of putting a file on the target (apart from the session file).
Of course, I am not attacking a machine without permission, this is part of a challenge.

There's a couple ways to get PHP version:

If you can find the error.log from the webserver, you can request http://example.com/<?php phpinfo() ?> first, then use the LFI to include it on a PHP file, and get it to parse the log.

You can include /usr/bin/php or /usr/local/bin/php or another path, and get the executable itself.

You can use the logfile trick to send a webshell, and use the webshell to get PHP version.

You can use the logfile trick to execute functions that aren't available on certain versions, and by the error message you can deduce the version in use.


Why does switching to "n-only" help preventing attacks on WPS? I just cracked the access-point (AP) of a friend of mine (Linksys E4200) and we were looking for a way to rid him of the vulnerability. Unfortunately, the E4200 does not disable WPS when you tell it to (via web gui).

While we are waiting for a firmware update (due beginning of March) Cisco recommends to disable wireless. A forum post recommends switching the AP to N-only (where possible) to make reavers life harder.

Does anyone know why/how N-only helps?

Switching to “n-only” help preventing attacks on WPS because some hackers don't have n adapter :). It doesn't matter n-only or mix mode enable on your AP. You should turn off WPS.. but some devices don't have this feature :(. I have cracked a few AP (n-only mode) and haven't had any problem with cracking.


Suddenly scans for 'formvars.php' Suddenly, starting from about yesterday-evening (gmt+2), I get lots of -what I suspect are- 'hits' from bots that are looking variations of this file:
/somerealpath/includes/formvars.php

I'm curious to know what this is, and why so suddenly these scans?
They all seem to scan the same couple of paths. They could be getting these as the first xx google hits, but they are exactly the same each time.

The root,
one special subdir
a pdf in that subdir.

The scans come from various ip's, but they all have useragent:
 "Mozilla/5.0 (iPad; U; CPU OS 3_2_1 like Mac OS X; en-us) AppleWebKit/531.21.10 (KHTML, like Gecko) Mobile/7B405"

Which, for some sort of security-problem-scan seems odd: why would they all scan with the same tool? Why are they all using the same urls to check for the file?
I doubt think it is a targeted attack as the volume isn't big enough to cause major trouble, and the file doesn't exist anywhere so trying again and again isn't usefull...
It could be some sort of zero-day for something that came with a tool that included user-agent spoofing and some sort of "find some valid urls trough google" code, but that would also be strange? And why not guess some sort of /pathwiththelibraryname guess in between?
The only thing I can find with that filename is some sort of webcalendar but the file itself seems clean enough (only some function), so I doubt that's it.
I'm just curious about what this might be (and annoyed at the errors it creates in my logfiles, but that's another thing :) )

The file /include/formvars.php is used by WebCalendar.  About 2 days ago,  (April 23rd 2012) a remote code execution exploit for WebCalendar was released.

Although /include/formvars.php isn't the actual vulnerable file,  its a good way of verifying that WebCalendar is installed.  Its looks like the attacker needs install/index.php and includes/settings.php however these files are very common so by checking if these files are present there is no guarantee that its actually a WebCalendar instance.  If the file /include/formvars.php is present,  it will likely follow up with a POST request to install/index.php to inject PHP code and then a GET request to includes/settings.php to execute the injected payload.


Why is sub resource integrity (SRI) only limited to certain files? I'm quite enthusiastic about the Sub resource integrity (SRI) features. But, why is it only limited to JS and CSS files? I tried to pin a LESS (CSS variant) file, of which the integrity tag was ignored by Firefox and Chrome.

Additionally, I understand that SRI is mainly designed to protect against unwanted changes of files, hosted on external sources like content delivery networks (CDN's). But since we all love layered security, why not add it on all browser included files (even self-hosted)?

Even if you'd be able to change a JS file, it's likely (but not sure) you'd be able to change the integrity tag as well.

Why is it limited to JS and CSS resources?

  But, why is it only limited to JS and CSS files?

The W3C SRI specification states:

  The scheme specified here also applies to link and future versions of this specification are likely to expand this coverage.

It also states:

  A future revision of this specification is likely to include integrity support for all possible subresources, i.e., a, audio, embed, iframe, img, link, object, script, source, track, and video elements.

It seems that this is just the first version of the specification. The github repository also includes a milestone referencing issues the second version of the specification will address. For example, there is a github issue regarding adding the ability to add the integrity for downloads.

  I tried to pin a LESS (CSS variant) file, of which the integrity tag was ignored by Firefox and Chrome.

It seems that they follow the specification to the letter. The first version is about adding the integrity for the link and script HTML elements, but not for every attribute. The specification only mentions handling the integrity attribute for the link element when rel="stylesheet":

  Whenever a user agent attempts to obtain a resource pointed to by a link element that has a rel attribute with the keyword of stylesheet

By looking at the Less website, we see that they rely on link but with rel="stylesheet/less", thus not part of the specification.

Maybe you could file an issue on their github repository for the second version?

Why is it limited to external resources?

  why not add it on all browser included files (even self-hosted)?

By looking at the specification it does not seem it was part of their threat model. Here is their first goal:

  Compromise of a third-party service should not automatically mean compromise of every site which includes its scripts.

One could assume that if an attacker is able to modify a self-hosted JavaScript file, it can probably do more. Sure, it does not mean the attacker can modify the HTML part, but the attacker in their model cannot modify internal resources.


Is there a secure way to share a PGP key for use with security.txt? One of the options for security.txt is to include a reference to a PGP key for use when reporting vulnerabilities in a private, secure way. It makes perfect sense to me for a researcher to use PGP to encrypt a message to report a vulnerability.

But what about the business on the other end? How should they safely implement a PGP key for such reports?

One option would be to simply use one person's personal (or, more likely, a one-off created exclusively for this purpose) key, but then there is a risk that the person will die (unavailability) or turn rogue (DOS/MitM reports).

Another option would be to generate a key pair for this purpose and share it among a group of trusted individuals. This solves the "death" problem and the DOS problem, but it expands the risk of one person receiving a report and doing something nefarious with it.

A shared key also means that any private-keyholder can issue a message "as" that identity. This can probably be solved by making it clear that the key is for "receipt-only" when minting it.

Is there a standard and/or safe way to do things like this?

One way I would suggest is sharing the Encryption subkey among the members of the security team, but not the Certify ("master") nor Sign subkeys. This way every member of the team can decrypt the messages sent to the email address listed in security.txt, but not send signed email back. The Certify and Sign subkeys can be kept on a vault system, which can be used for the purposes of the following:

signing official communication
revoking the shared Encryption subkey and creating a new one in case a member of the group leaves

The vault system can have those subkeys on some kind of HSM that can't be physically removed; similarly, either the system itself or the room where it's kept must require multiple people present in order to operate.


How to properly secure a resource server, but allow public clients to have access in a machine-machine style without user interaction My scenario is this, I have my web API that I want to protect from external people, but I want to allow access for my own applications that could be running in windows forms/android/ios/etc... [considered public clients].

I'm using identity server 3 (Github link) as my identity provider implementation.

My initial thought was generate a client id + secret and put that (somehow) in all clients, but this is considered dangerous because public clients should not have secrets on it, because with more or less effort you can always have access to it.

So what is the best way to protect my web API?
I have control of web API, clients and Identity server, because they are all my products.

Rate-Limit the API calls. Make sure every installation gets a unique ID (see UUID). Limit the calls to your API per UUID and IP to a strict limit.

But be careful: sometimes, multiple legit users use the same IP address. So you should set your limit to about 2-3 times of what a single client will consume.


Received a set of SMS/MMS containing 2 photos, a voice message, and a text "I need help" with Google Maps link from a known contact. Is it spam? My girlfriend (let's call her Jane) just got a set of SMS or MMS messages coming from a friend of her (let's call her Hellen). These messages contain:

Two photos of Hellen
A voice message
A text that says "I need help" followed by a Google Maps link.

And according to the messaging application, they where sent to 5 people, including Jane.

With all this being extremely suspicious, I told Jane not to touch anything on that messages and asked Hellen a bunch of questions:

The phone is a Samsung Galaxy J5
Android version is 7.1.1
Hellen did not take those pictures of herself, and the pictures do look accidental, like if the phone took the photos while she was just holding it doing another thing.
Hellen doesn't remember any recent apps installed or unusual web activity (I don't fully trust her on this one but whatever).

With the case exposed, I ask you:

I am right assuming this is only Hellen's problem and as long as Jane ignored and deleted her messages she is safe?
How should we proceed about Hellen's phone? A simple factory reset would do it?
How can we prevent this situation in the future?

This does not seem to be a virus. It is a panic function in some android phones, that allows to send these messages in case you are kidnapped or otherwise in danger by pressing the power button 3 times. She must have activated it accidentally.

More info here and here.


Using an example to comprehend why “safely” erasing a drive yields better results than filling it up with meaningless data A hypothetical 1GB USB stick is full of sensitive documents/images/etc. and it is not encrypted.
The owner wishes to discard it and is aware of having to safely erase it first.
There are several tools and utilities to do this. Some can be configured to do it “faster yet less safely”, others do it “slower but more safely”.
As opposed to have it erased using all the different ways known to do this, the owner chooses to simply drag all the current items to the recycle bin and then paste one 1GB (~2-hour) black screen movie file to the USB stick.
Again, no fancy erase utilities are used. The USB stick is then discarded.
If it falls into the wrong hands, can any of the sensitive files (that filled the stick before the movie file was pasted) be retrieved?
(1) If no, why do complex hard drive erase utilities exist? Some of them feature “safe” erase procedures that take houuurs, when simply filling a soon to be discarded HD with meaningless files, can do the job?
(2) If yes, how can 2GB (movie file + sensitive files) co-exist in a 1GB stick? Seems to me like the only logical explanation is (a) the movie file was in fact less than 1GB, (b) the USB stick was secretly larger than 1GB as stated, or (c) the movie file was copy-pasted only partially and the owner did not notice.

There was a worry that hard-discs (of an appropriate vintage now) would be vulnerable to having overwritten bit read. Rebutted here: https://www.nber.org/sys-admin/overwritten-data-guttman.html
Therefore schemes were devised to do multiple passes overwriting the data enough times and randomly so that no future development could extract data.
For a USB stick the main risk is probably 'bad blocks' - where the original data is preserved read-only, because the re-write failed. In this case it doesn't matter how many times or how cleverly you over-write it; someone who can dismantle the drive and read the flash directly can extract (some of) your original data. Drives handle bad-blocks by over-provisioning - the stick has more space than is exposed to the operating system, and remaps bad-blocks to the reserved space when they fail. Flash tends to fail read-only (i.e. fail on write), so the old data is present, if an attacker reads the flash directly.


How to proactively prevent data-theft if a stolen android phone is immediately taken offline? Assume that it is an android 10+ and that it is encrypted.
Assume that it is protected by a 6-digit pseudo-random pin code.
Assume that the data can be erased remotely using google's "Find My Device" service.

If the thief shuts the phone down, they'll have to contend with the disk encryption (it will wipe itself after a certain number of incorrect tries)
If the thief doesn't shut down the phone, they'll have to bypass the lock screen before I am able to erase the data remotely.
If however, the thief simply places the phone in airplane mode (which doesn't require a pin to access), then they'll have unlimited time to bypass the lock screen.

How likely are they to bypass the lock screen assuming little to no resources, fairly skilled, but hellbent on data extraction?
Is there anything I can do proactively to not only make it harder for them to bypass the lock screen but also to prevent data-theft once/if they do.

The type of data I would be concerned about:

Email inbox.
Chat conversations (Skype; Whatsapp*; Slack).

I do have the option of removing said applications and data, but that would also make the phone largely useless to me.
Whatsapp Is especially tricky given that they'll also have access to the sim card, and I am unlikely to be able to have it blocked/replaced somehow. Can we please stop with this phone number based identity madness?

If the android device is on latest security patch, there's no way to bypass screen lock protection. Even if it's not fully updated, the thief has to search what known vulnerabilities can be exploited from locked screen for privilege escalation. He will also have to come up with an exploit. Many high severity and critical severity vulnerabilities that are fixed each month don't have publicly available exploits.
Screen unlock attempts timeout increases exponentially after every few incorrect attempts. If your unlock code is not in most predictable android lock patterns, it cannot be brute-forced. If the thief goes to recovery mode to factory reset the device, on first boot, device will ask for Google account credentials that were used to set up the device by the owner. This is called Factory Reset Protection.
By default, sensitive messages like OTP are displayed on lock screen. It can be disabled from settings. First, the thief will use your SIM to learn your phone number. Then he will reset Google account password using OTP showing up on your lock screen. After 72 hours, the new password can be used to unlock factory reset protection.
Your data is always safe but your accounts can be compromised with possession of SIM card alone. Use TOTP to protect your accounts and immediately request your network operator to block SIM. That will leave your device un-reusable for the thief.


Find out Browser's AES generated key on SSL/TLS When it comes to TLS handshakes. first of all I'm wondering how the AES keys which are responsible for encrypting data is generated? is it generated randomly on the browser and it's securely sent to HTTPS server or server is responsible for generating those keys? Secondly is there any way to actually find out those AES keys which browser(for example Chrome) uses for encrypting data (after SSL/TLS authentication successfully handled)?

This depends on the Key exchange algorithm being used.

When RSA key exchange is used, the key is randomly generated by the client and then encrypted using the server's certificate. When using RSA key exchange, the server has very little say on the session key.

Modern TLS configuration usually recommends Diffie Hellman, which is more appropriately called key agreement protocol because the Session key is calculated using random inputs from both the client and server.

Both Firefox and Chrome have a feature to dump the Session key to a file, by setting the SSLKEYLOGFILE environment variable.


What security risks are involved with sending data over the internal network on Rackspace Cloud servers? What are the risks involved with sending unencrypted data over the internal network with Rackspace Cloud Servers?

Assume the interfaces use iptables to filter IPs and ports so they only accept data from other trusted servers within the internal network. Let's also assume that we trust our provider, Rackspace, but not all the other servers in the internal network.

Rackspace says they block IP spoofing and network sniffing.

Then the risk of getting sniffed is low. You could do some test to see if what they are saying is true. If you really trust rackspace you can send data unencrypted. If your data is really critical or sensitive ... Encrypt it anyway just in case ;)


How can a server certificate be valid when there's a hostname/issued to mistmatch? i was wondering - i was surprised to check out a specific site's certificate and see that the hostname is not equal to the issued to parameter - the certificate is valid.

The example is here: https://www.cyberark.com

How is it possible? ssl762713.cloudflaressl.com is definitly not Cyberark.com

cyberark.com is in SAN (subject alternative name) of that certificate


Website using http and https Is there any method to know whether a website is https only or uses http webpages as well?Can it be done without using browsers cache?

I'm going to assume that you are talking about HSTS.

HSTS is TOFU (Trust on first use). So from the second use onwards, you are protected.

But what about that first use?

I don't think there's a unified strategy. But here's some ideas:

What works today

HSTS Preloading: Use a browser that makes use of the HSTS preload lists. (Chrome, Firefox) And hope that the developers of the website you want to visit have had their website listed for HSTS preloading..
HTTPS Everywhere: There the EFF's HTTPS Everywhere project that uses a browser side plugin (Chrome, Firefox) and a large list of redirect rules, to redirect plain HTTP requests to encrypted HTTPS sites. This plugin only works if a rule has in fact been created for the website you want to visit. But -- other than HSTS pre-loading -- this does not depend on the webmaster's cooperation. Rules can be submitted by anyone.

What might work in the future

DANE/DNSSEC: DNS is a general purpose vehicle for storing info about domain names. Wikipedia's article about HSTS hints that this might be used in the future to indicate HTTPS support. -- "DANE" is a mechanism that uses DNSSEC (the signed form of DNS) to communicate crypto info about a domain name. For example to say "only this CA may sign my HTTPS certificates". This is achieved by placing something like this record in your DNS server:

_443._tcp.www.example.com. IN TLSA (
0 0 1 d2abde240d7cd3ee6b4b28c54df034b9
7983a1d16e8a410e4561cb106618e971 )

Now this tells anyone (who cares to ask) that a specific crypto key is associated in some way with a service that's running on port 443 on www.example.com.

Now a human can easily guess: "Okay, 443 will probably be HTTPS", and then connect with HTTPS instead of plain HTTP. But what you can't deduce from an existing DANE record is that HTTPS is fully supported for every bit of the web site. (For example HTTPS might only be used for login to that site and not work for the rest of the site.)

But AFAIK there is no mechanism yet to communicate "Use HTTPS!" to a browser.


Website online assistant asked for my password Sorry if this isn't the right place for this..

I recently joined an online gambling website and clicked on live assist.  To verify my identity they asked for my username and password.  I did not give it to them.  I am not really worried about it being a phishing attempt, but my main concern is that he can see my password in plain text.  Is this normal procedure/okay/safe?  I use a variation of the same password for every website that would be pretty easy to figure out if one tried, because i thought no human would ever see them and everything is supposed to be store encrypted.

I did some research before joining and the website has a good reputation but this just gets my (admittedly ignorant) spidey-sense flashing.  This website now has a grand of my $ and is linked to my bank account and has me a bit worried.

Thanks for any wisdom you can share.

You have two big issues here.  Let's get the easy one out of the way:
No, they should not ask for your username and password
Since you said you contacted them through their on-website live support, one of two things was happening:

Their website got hacked and you narrowly avoided giving your login credentials (and therefore money) to a hacker
They have poorly developed support infrastructure and poorly trained support employees.  If you gave away your credentials something bad may have happened, although you might have been perfectly fine.

Personally I'd probably go with #2 - if a hacker had taken over their website they would have had much more effective ways of collecting your credentials than waiting for you to open the chat and ask for your username/password.
Unfortunately many businesses don't understand the dangers of asking their users for their credentials.  There are two main risks in doing this:

Employees are a major source of trouble themselves, and so there is no guarantee that an employee won't steal from a customer if they are given username/password while doing their job, and see an account full of money on the other end.
Regardless, it trains their customers that giving away username/password is okay, making them more vulnerable to future phishing attempts.

So: This is not safe.  This is not reasonable.  It is however somewhat normal (although shouldn't be).  I would at least complain to their support and try to explain why this is such a bad idea, although don't expect them to take you seriously.  I would probably try to find a new place to take my money.  Onto issue #2
Using an easy-to-guess rule for password generation is not a great idea
You said:

I use a variation of the same password for every website that would be
pretty easy to figure out if one tried, because i thought no human
would ever see them and everything is supposed to be store encrypted.

This is a very bad assumption.  Unfortunately passwords get leaked from websites all the time, and oftentimes in forms that allow hackers to figure out what the password is (presuming it isn't just in plain text, which still happens).  As a result you should assume that your password will be seen by people, and therefore pick a password that is completely unique to each account.  The best way to do that is with password managers.  Obviously many people don't want to put forth the effort to use one, but at the very least if you have an important account (which this would be if it is protecting >$1,000 of your money), it should really have its own strong, unique password.


Uploading files through an HTTPS web application We have a client facing web site. A certificate is required to land on the login page, where upon you then need a username and a password to further login. System has Apache servers acting as load balancers up front, and then Tomcat servers serving the application. All OK so far, but we now want to provide file upload ability for users to bulk upload configuration changes to settings within the application.

Any thoughts on best approach? Is it possible to establish a true SFTP connection over the already established and authenticated HTTPS link somehow? If not, how? Thoughts?

Is it possible to establish a true SFTP connection over the already established and authenticated https link somehow?

Yes, if you rewrite the HTTPS implementation in the server and client, and write a custom SFTP client, and solve the problem of sharing a socket across different processes....

Why don't you just use HTTP (over SSL) to upload the file?


Is it safe to install Anaconda/Spyder/Python on laptop with netbanking? I am new to learning Python and I am using a laptop for netbanking etc.

Is it safe to use the same laptop for netbanking and installing packages like Anaconda/Spyder/Python on the same laptop?

It is not "safe" in the meaning of "no risk at all" to install any kind of software on the same device you use for online banking since you don't know what the software is actually doing. This does not matter if this is open source or closed source software.

But, it is also not "safe" in the same "no risk" meaning to use online banking at all or to use a laptop, because you cannot fully trust the online banking, the browser you use, the operating system or the hardware and firmware of the laptop. Additionally you cannot fully trust your internet provider, the vendor of the router etc.

This means, that there is no such thing as "safe" in the meaning of "no risk". There is always some risk and the question is if this risk can be accepted. In the case of installing a widely used software from a well known source and keeping the software up-to-date the risk is probably acceptable. But, if you got the software on an USB stick from a suspicious person I would probably not trust it even if the person claims that this is the well known and widely used software you can also download. And, if the download was not done from the original site I would be suspicious too.

But, even if the download was done from the official site it still might be malicious in case the download site was infected. Thus, again, the risk is not zero. But, this is true not only for open source software but also for closed source.

In summary: the risk is never zero and the more software you install your on your system the larger the attack surface gets. You have to decide yourself if the risk is acceptable or not based on various factors. Closed source vs open source software might be such a factor but there are way more relevant ones, like download source or reputation of the vendor or author of the software.


How can I find out whether my browser traffic can be intercepted? In theory, visiting a website with HTTPS and seeing the neat, little lock icon next to the address (e.g.: ) should give us a warm, cozy feeling of safety. However, how safe is it really?
As it stands, HTTPS (if implemented correctly), can prevent man-in-the-middle attacks, so that no one knows what is exchanged between my client and the server. But that is not a given.
Q1: Are there tools to verify whether or not the servers are set up correctly? In 2021, is (for example) Chrome Dev Tools Security tab sufficiently reliable to determine the security of the contents transferred between my client and the server? Are there scenarios, where that Security tab is all green, but (given I do not use a VPN, and my computer and (if applies) proxies are not compromised), a third party can still see my traffic?
Aside from web content, another important piece of data to protect is the domains that I visit. In theory, a third party can easily know the IP address of the server, but can they know the domain name? E.g. TLS's plaintext SNI field can be used to find out the domains that certain clients visit. TLS 1.3 has introduced Encrypted SNI (ESNI) to circumvent censorship and tracking. Because of that, China, for example, has started banning TLS 1.3+. Cloudflare made a big point about upgrading to ESNI right away, and Firefox apparently was the first to support ESNI in 2018. Thus the question:
Q2: How can I verify whether someone can sniff the domains of my visited servers? Are there good tools for that?
A first answer to this question is: Cloudflare Browser Security Check. But it is not enough, since I want to check individual connections, not just my browser.

TLS secured web connections are the basis of the web. Within ordinary usage, this is as good as it gets.
True, the system is not perfect and any organization capable of obtaining an appropriate cert can perform a MiTM, but this is a pretty high bar. In some ways it's like asking about counterfeiting cash, good counterfeits are difficult but not entirely impossible.
As far as what you can personally do:

Check the TLS Certificate and CA
Encrypted SNI is still not common, although I believe Cloudflare supports it.

If you feel you need something more:

Escalate to using a VPN.
The last practical step is to use Tor, specifically the TBB Browser.


Do I need a WAF in front of an API exposed by Google Cloud Endpoint I have a backend API exposed to the internet by GCP's Google Cloud Endpoint (Extensible Service Proxy). Cloud Endpoint allows us to control which other backend services can access the API and block unauthorized requests.

Is it worth adding a WAF in front of Cloud Endpoint to further protect the API?

ESP is more for authorization and monitoring but does not provide the same functionality as a WAF. A WAF will protect you from attacks like SQL injection , well known Joomla ,  Wordpress , Apache vulnerabilities , etc.

It's seems like adding a WAF would be a good idea and improve your security posture.


Is it secure to send sensitive data through gmail smtp with nodemailer? I have an application that would send sensitive data through email to specific users. I have setup a google business account which provides email addresses on my business domain. I will be sending email through Nodejs nodemailer over smtp protocol with TSL. The way I understand it, the data being transferred between my server and gmail server and the data then delivered to the user's email address will be sent through secure encrypted channel. I am mainly worried about the security of data being sent from my server to the gmail server. Will this be secure or should I set up my own email server?

If you are worried about the confidentiality of the data you're sending:

Use end to end encryption and make it mandatory.

The two (more or less popular) alternatives that will both serve the purpose are

S/MIME
PGP/GPG

If you are only worried about the transfer to the google servers, as long as you are making sure using SMTP with TLS, the transport way is secure.

Note:

While the end-to-end encryption only encrypts the message body, the message headers  (as well as the message, if you are not using end-to-end encryption) are transmitted over the secure connection between the SMTP server and your server, thus only accessable for those parties, as far as your question goes.

Keep in mind that this is only true as long as the user does check his/her mails over a secure connection, which you have no control over.

At a minimum, if you soley rely on SMTP with TLS, all hops of your email must be trusted by you to exclusively use TLS and to respect the privacy of the user.

Those are, at least:

Your server,
Google,
The user's email provider,

Potentially many of those, depending on the forwarding the user has configured,

The user.

Bottom line being, as you cannot establish that trust:

Make either S/MIME or PGP/GPG mandatory to protect the sensitive user data on transit.


Difference between Form Post Response and Authorization Code I am trying to find the best OAuth2 mechanism for our VueJS Single Page Application (SPA) hosted on our own backend. From my research, Authorization Code should be sufficient, as long as the redirect URI happens on the backend. If it happens on the front end, I would need to use PKCE.

The confusion I have is that I saw this spec about Form Post Response at https://openid.net/specs/oauth-v2-form-post-response-mode-1_0.html
Auth0 also had an article about it https://auth0.com/docs/login/spa/authenticate-with-cookies
Since my SPA is hosted on the same domain as our backend, this looks to be a valid option. I do not see much info about this spec online.

I do not understand the difference between Authorization Code grant type and using Form Post Response. Wouldn't both the mechanisms send the access token to my backend? It would be great if someone can shed some light on this.

I think I found some explanation here: https://auth0.com/docs/protocols/oauth2#how-response-mode-works

form_post: This response mode is defined by the OAuth 2.0 Form Post Response Mode specification. A successful response is 200 OK and the response parameters are embedded in an HTML form as hidden params. The action of the form is the redirect_uri and the onload attribute is configured to submit the form. Hence, after the HTML is loaded by the browser, a redirection to the redirect_uri is done.

For example, a response might be:

HTTP/1.1 200 OK
<html>
 <head><title>Submit</title></head>
 <body onload="javascript:document.forms[0].submit()">
  <form method="post" action="https://my-redirect-uri.com/callback">
    <input type="hidden" name="state" value="klsdfY78FVN3sl6DWSjsdhfsd8r67832nb"/>
    <input type="hidden" name="id_token" value="eyJ...plD"/>
  </form>
 </body>
</html>


Safenet iKey 4000 RSA key pair generation in FIPS 140-2 compliant mode I need to generate RSA key pair and show the public and private key on screen (in a secured room). The algorithm must be FIPS approved.

So I would like to use iKey 4000 which is FIPS 140-2 level 3 compliant (I am using PKCS#11 library from Safenet to work with the token). If I generate RSA key pair on token (I mean using CKA_TOKEN attribute set to True) I am not able to show private key exponent. If I use CKA_TOKEN attribute set to False, I can see the all information about key pair.

But the question is if method of generation of key pair outside token is still FIPS compliant?
Also it is faster than generating key pair on token so I assume it is not generated by token itself but by some software part of Safenet library.

Would it be still FIPS compliant?

The difference between generating a KeyPair with CKA_TOKEN set as true and false is that when it is set to true, the KeyPair that is generated by the hardware token is persisted (it exists on the token even after you logout of the session). If it is set to false, it is still generated on the hardware and not outside the token, it is just not persisted (it exists only during the active session, when you logout, it is gone).


Know LastPass fishing apps? Ways to protect? I am a LastPass user, which I use with the browser extensions.

Lately it occurred to me that it should not be too hard to create a fishing browser extension that

would replace LastPass upon install,
look exactly like last pass,
Behave like LastPass up to the submission of your password, and
Simply send your login information to some email address upon submission.

My questions are:

Is that a known hack?
If not, why wouldn't it work/be easy to implement (beyond making the user install the extension, which can be done using traditional fishing techniques)?
If it is, is there any ways to protect oneself against that kind of threats? Does LastPass recommend something in this matter?

My question is about LastPass in particular, but I guess the same issue occurs for any keychain software, and I'd be happy to hear about those (for example, 4. if there is a threat at all, does any keychain software offers a better protection than LastPass?)

First installing extension is not a harmless operation. Common usages say:

  Once you have installed an unwanted software on your computer, it is no longer your computer

But the attack should be slightly more complex than what you describe (after all Lastpass is a well known security solution). The master password that you type into the password extension is only the password of the keyfile used to decrypt locally the content of the vault stored on Lastpass server.

So the fake extension should have to:

either to install itself between the browser and the real extension - not that easy
either to fully reproduce all the decoding operations of the true extension - indeed possible but still a fair amount of word - meaning

locate and unlock the keyfile with the master password
download the vault
decode the vault

Such an extension would be a major attack against Lastpass, and as such with a great risk of being identified and de-activated soon.

Following in only my own opinion:

This is still a serious threat that should be considered, mainly in case of a targetted attack, meaning if the attacker can manage to activate his software only on a small number of machines to limit to risk of being identified soon. That assumes that the attacker is a strong organization and you as the target represent a high value because in any other case the ratio gain/cost is low and the whole operation does not worth it.

Anyway, the rule here is that Lastpass can allow you to securely use passwords from a local machine, as long as the local machine is secure. If the local machine is compromised - and having an unwanted extension in your browser is a serious compromission - anything can happen (keyloggers, theft or suppression of files, etc.)


Advantages of client certificates for client authentication? I'm no security expert, so please just ask in a comment if I haven't made my question clear enough for an answer.

The Scenario

We have a server running WCF services, and a number of clients connecting.  These clients are actually Linux PCs which we build.   We need to establish secure communications between our server, and our clients (again we build them, and deploy them to customer sites).

Client trusting the server

We will implement this by allowing the client to establish a trusted connection with the server via implementing SSL communications.

Server trusting the client

We now have the task of authenticating the client.   Obviously this is done by keeping some sort of credentials on the client.  Once the client is connected, it can send the credentials to the server and the server can validate them.

One option for these credentials is to store some sort of Guid or other id/password which is generated by the WCF based application.   Upon receiving the credentials, the WCF service does a lookup in the database and verifies they are correct.

Another option is to use Certificate Services to create client certificates which are copied to the client pc before it is sent out.  After establishing the secure connection, the client sends the certificate to the server which authenticates the certificate with Certificate Services.

The Questions

What advantages does using a certificate to authenticate the client have over a username/guid?   What disadvantages does it have?

Please consider:

Security
Complexity of implementation
Complexity of programming Integration with the application.   This includes the workflow of creating the authentication token, associating appropriate (authorization / association) metadata, managing authentication such as disabling access etc.

Deploying client certs could fit here.  The advantages of using a cert over username is somewhat simple.  Anyone can type in a username from any client device.  If you're using a combination of username with guid, then the "security" or assurance that the client is connecting from a known/authorized client device is dependent on the strength and uniqueness of the guid.  If there's a way to clone or spoof the guid (mac addresses can be spoofed fairly easily), then the assurance level would decrease.

Client certificates can be deployed to clients, with or without validity checking (aside from validity date, cn, ski/aki, fingerprint, etc).  On-demand validity checking mechanisms such as ocsp would require the server application check with an ocsp server each time client connects/attempts to auth.  But from the description, I didn't read that validity checking is as important as being able to tie cert to client device.

One important detail with clients certs (certs in general) is that is can be exported and most implementations do not lock down portability of the cert.  Regardless of whether or how the client certs are going to be stored, without proper measures, the cert can be easily copied from device to device.  Some implementations store the cert on the filesystem (files that end with .cer, .der, .key, .crt usually are indications that certs are stored in the filesystem).  Stronger implementations (application dependent) may store the certs and keys in a keystore (i.e. java key store).  The key store can add additional protection like ensuring the private key is not exportable.  However, the assurance that the key hasn't been exported is only as strong as the key store itself.  Hardware key stores (i.e. smart cards, usb hsm, ironkey, etc) offer a much stronger assurance that private key is not exportable than software key stores.

BTW, the above point also affects server keys.  Most implementations store the private key in a software key store and is usually marked exportable.  Further, the private key is usually not password protected so anyone with access to the server can walk away with the private key.  If a cert can be copied, then it doesn't offer non-repudation.

To answer your question, if there's a good way of leveraging a hardware id of sorts (guid, serial number, cert stored in HSM, etc), that'll likely provide more assurance than using a software-based id (client certs included).  Using client certs with password protection enabled for private key access provides a bit stronger validation because not only does a client need to have access to the private key but also the password to use it.

If you do decide to use client certs, then you'll have to build or use an existing PKI infrastructure.  Vendors like Codomo, Entrust, Symantec (formerly vrsn, thawte, and geotrust), Godaddy, and a bunch of others offer both public and private infrastructure for use.  However the cost of implementing a software-based client cert will likely be higher than using a software based hardware id or perhaps even a hardware-based unique id.

If anything, determine the level of assurance you want to have and decide whether software, software + password, or hardware is sufficient.


XSS prevention when inserting HTML without user controlled data into the DOM? I have some JavaScript that fetches some HTML from a PHP script, and inserts it into the DOM:

$.get("/includes/blocks.php",{
  id:encodeURIComponent(id)
}).done(function(Data){
  $('
'+Data.html+'
').appendTo('.main');
});

What is the best way to prevent XSS if I want insert HTML code from blocks.php into the DOM? It contains a lot of HTML elements with tons of attributes. No user controlled data is returned, but could the returning data be modified somehow?

No user controlled data is returned, but could the returning data be modified somehow?

No, you're safe. You're fetching a document from a fixed URL (/includes/blocks.php) on the same origin which reflects no user-controlled data. For it to be vulnerable to XSS, you'd need an entry point where the attacker can inject code, but at no point does the your code even handle user input.


Is there any flaw in my API-to-API authentication concept I missed? I've been thinking lately about how all of the authentication methods used in API to API communications (RESTful API's) are mostly methods that have been designed to be human oriented first (tokens\passwords\etc) & how in API this often means you need a secret store to have the passwords\tokens stored in it.

My idea is that the requester API will contact the receiver API with a hashed token & a callback webhook address, the callback URL effectively acting as the user identity which the receiver API then contacts (in a new connection) which he then get the unhashed token from, the receiver token then compares the hashed and unhashed version of the token and should they match he knows that the requester of the original request is in fact the API of the webhook.

A workflow of the auth process is described in full in the following diagram:

I've also created a docker-compose POC of the concept at my github which works as i thought it will (please note that while I will gladly receive notes about the POC this question is only about the theory of the concept as a whole being secure).

The question

Is this really secure? Is there any vulnerability I missed to this authentication method which will allow an attacker to trick the receiver API by impersonating another API?

If there isn't is there any way to prove without a doubt (mathematically or otherwise) that this is in fact secure?

Assume the following are given:

HTTPS is used (to protect against MITM) on all requests
The backend DB is secure
Tokens are generated randomly & are not reused
The hashing function is modern enough to provide good protection
The DNS register of both API's is secure
This is an authentication only method, not authorization nor encryption.
The main risk that this tries to protect is from attackers impersonating the legitimate requester API and sending a request to the receiver API pretending to be him in a network facing API (IE not localhost only, possibly a local subnet but also possibly through the internet).

The main flaw of this concept is that it is totally pointless.

Essentially, what it does is:

Requester: Hi, I am going to tell you my random name I just created just for you. Here its hash. [step 2]
Receiver: Haha OK, here is your hash back [step 3]
Requester: Let me check [step 4]. Cool, the hash you returned indeed belongs to my random name for you. I let you see it now and I will forget it immediately [step 4,5]
Receiver: Hey, your name indeed produces the hash you provided earlier. Cool! [step 6] I now finally know your name! [step 7]. Let's get down to business. How can I help?

You claim that

  This is an authentication only method

whilst the flow inherently only attempts to make sure that the Requester remains the same throughout the communication — something that HTTPS itself already takes care of.

The point of authentication is to verify that whoever claims they are are indeed that party. This implies that the Receiver either already knows the party (e.g. has its password hash or public key in its database to check against) or is able to verify against trusted third party (e.g. using OAuth). In your scenario, the Receiver does not know the Requester at all, and the fact that it merely tells its name does not authenticate it in any way.


What is the point of hosting database externally? Following the breach of avast:forum.avast.com
they decided to host the database externally@id.avast.com.
My question is, what is the point of hosting the database externally since when you get remote code execution on forum.avast.com (which has access to the database hosted on id.avast.com) you can then steal the database anyway?
Is it so no data is lost/destroyed?

They're using id.avast.com as an authentication provider, not just an external database.  So RCE doesn't get you access to it (assuming it's a different host machine), SQLi certainly doesn't, and since it's on a different domain (origin), neither does XSS.  They've now significantly isolated their password hashes from the forum software.

Using an authentication provider is very different than just moving the database to another host.  (And also consider there are many threats other than RCE.)


Encryption key generation performance A question for cryptography experts. Imagine we have a conceptual Notes.app:

There are notes (title|content) stored as AES-256 encrypted strings
Application has to present a list of all notes (titles) in a list on its main window
Every title|content is encrypted with a key, generated from a password and a salt
Let's imagine the key generation algorithm takes ~80ms on a mobile device to generate a key

With the following conditions, it would take almost 1 second to decrypt 10 note titles. But what if there are lots of notes?

My 2 pennies on the problem: Encrypt all notes with different initialization vectors, but also with identical salt. That would allow me to generate a decryption key only once and decrypt lots of notes fast.

The question: doing so we would end up with lots of different notes, encrypted with an identical key. Does that somehow compromise the security of AES encryption? Is it possible that knowing there's a bunch of files with not just identical password, but also identical salt somehow makes it possible to crack the encryption?

Thanks for your thoughts

The password hashing business is about turning a password (which is weak, by definition) into a sequence of bits suitable for storage (password verification token) or use as an encryption key (in which case we are doing password-based key derivation). In order to tolerate the use of passwords, we have to follow some strict rules:

Use slow hashing with a lot of iterations (that's easier said than done, so don't get creative and use a well-studied standard like bcrypt or PBKDF2).
Never ever reuse a salt value for another password instance. Indeed, if two passwords are hashed with the same salt, then attacking one will allow attacking the other "for free".

However, once you have turned the password into a key, you can use it as any key. If your encryption system is not laughably frail, there should be no problem with using the same key to encrypt many files. In the case of something AES-based, this implies, indeed, a new IV for each file. The properties of that IV depend on the actual encryption mechanism; AES-CBC requires a random, uniform, unpredictable IV, while some newer modes like GCM only need a non-repeating IV and thus work well with a simple counter.

Note that many password-based encryption scheme use a salt-and-iterations system to hash the password into a sequence of bits containing both the key and the IV. Such systems cannot be used for more than one file; otherwise, you will reuse the IV, and that's a deadly sin (usually worse than reusing a salt, which is already quite bad). If you want to reuse the key resulting from the password hashing, then you MUST generate a new IV for each encryption run.

Also, note that if two files are encrypted with the same password-derived key, chances are that their headers will include the salt (you have to put it somewhere, so that it is available for decryption), so you will leak the information that both files are protected with the same password, and thus have the same source. Depending on your context, this may or may not be a problem for you.

Other people have found it convenient to use the simple expedient of storing their ten files into a single Zip archive and encrypting that as one file. Depending on your context, this may be applicable (or not), and is much simpler to implement without botching the crypto layer. In general, designing your own encryption format and assembly of algorithms is a path fraught with perils, and you cannot know whether you did it right or not (security is not functionality: it cannot be tested).


Is sending password to user email secure? How secure is sending passwords through email to a user, since email isn't secured by HTTPS.

What is the best way to secure it? Should i use encryption?

You should never send passwords in the clear, nor should you store them in the clear. You should hash them using a slow one-way cryptographic hash such as bcrypt or PBKDF2. If a user forgets their password, you offer them a "reset password" function, which sends a one-time reset link to their account.

A scheme such as the following is reasonable:

Hash all passwords using a salt plus bcrypt / PBKDF2. See my reasoning here. (EDIT, March 2019: use Argon2)
Validate the hashes upon login.
If a user forgets their password, send them a secure one-time reset link, using a randomly generated reset token stored in the database. The token must be unique and secret, so hash the token in the database and compare it when the link is used.
Enforce that a token can only be used to reset the password of the user who requested it.
Once the token is used, it must be deleted from the database and must not be allowed to be used again.
Have all password-equivilent tokens, including reset tokens, expire after a short time, e.g. 48 hours. This prevents an attacker exploiting unused tokens at a later date.
Immediately display a form to allow the user to set a new password. Do not use temporary random generated passwords!
Do all of this over SSL.

I highly suggest reading through The Definitive Guide to Forms-Based Website Authentication for a full set of guidelines on how to build secure login systems.


Accessing document using a 6 letter token We are building a web app where users can insert 6 letters/digits (A_Z, 0-9) into a form to access a document. Every document has a randomly assigned access code like this: 1ABH5F.

When the user inserts a valid code, the document is shown. There is no user login (no authentication) - it is open to the public.

The front end will access the document via a stateless API - the code will be sent to the API, which will return the document.

How should we implement information security? Nobody here is a security expert, but we were thinking like this:

Using a captcha on the front end
Limiting calling the API from a single IP more than 3 times/hour

What other things do we have to implement to prevent access to the documents?

I guess it is very important to specify the use case for this system:

It is a system for anybody holding this document to see the original (digital) document. It will be used in an environment where users can print the documents (for example: car dealer) and bring it to other companies (for example: car registration office).

The problem here is, that users can (and DO!) falsify the printed documents, then bring them to other companies (car registration office). These other companies have to have a way to check if the original is the same as the printed version.

Since we do not know which the other companies are (any of 10000+ car registration offices), any person holding/viewing the 6 letter token can access to the original document.

Brute forcing

So you have an alphabet of size 36 and 6 characters. That gives you about two billion different tokens. Lets say you have a thousand different documents. That gives you a chanse of one in two million of guessing a token associated with a document. Trying from a thousand different IP:s every hour for a year would give you almost ten million guesses - that should give you a couple of documents.

Sure, the CAPTCHA makes this harder. But they are not perfect, and they can always be cracked by humans.

The problem here is that since you only enter a token and no document ID you can only rate limit on IP and not on document. That makes it very hard to protect against brute forcing unless you have a very large space to pick tokens from.

Sharing

A password is personal and you are encouraged not to share it. That means it can be easily changed if it is compromised, and you have some control over who gets their hands on it.

A document token like this is supposed to be shared by design. You have very little control over who gets it. It will end up on mailservers and backups and post its on peoples desktops all over the world.

You have no idea who has access to the token, and if you need to change it you will need to redistribute it to all the persons who are supposed to have it. That is neither secure not practical.

Conclusion: There must be a better way

This will not give you very good security. If the resource you are protecting is not very important it might be enough, but I would not use it for anything of value.

I do not know your exact use case, but whatever it is there must be a better way to solve this problem than rolling your own API. Using an existing solution would also save you the problem of having to write your own code.

Use an existing cloud storage service, a VPN connection into the company intranet, or something else. Just don't fire up your IDE and start coding away.

Update: Your use case

This is one of the cases where an access token is probably a good idea. But to get around the problems mentioned above I would do this:

Keep both the CAPTCHA and the rate limit by IP. (You might want to reconsider how the rate limiting is done in order to prevent accidental or deliberate DOS.)
To deal with the brute forcing, I would increase the size of the token. Google Drive uses 49 characters with both upper case letters, lower case letters and numbers. That should be enough for you as well.
To get around the sharing issue, print the URL with the token in a QR-code on the document itself. This brings the hole problem into the domains of physical papers that peoplpe are used to dealing with. The people who see the paper will have access to the digitial original. That is easy to grasp.
Consider setting a limit on how many times the document can be accessed, or at least a maximum time for how long the token can be used. If the car should be registered within one week, there is no reason for the token to work after two.
Do not store the tokens in plain text in your database. Hash them. (Something fast like SHA256 should be enough here - no need to roll out bcrypt when you have large random tokens.)
Use a CSPRNG to generate the tokens, otherwise they could be guessed by an attacker having access to a few tokens.


Why are handheld tape recorders used? Pardon me for doing the following as I know it is cringed upon (the tv reference) but:

Is there any good reason for the usage of 'tape' recorders? Ex. taping a conversation between criminals, this is done digitally nowadays and the signal can be relayed between multiple points I believe. Is there any good reason to continue using tape recorders rather than digital ones?

At least in some cases, they're just props. A law professor once gave a talk about not talking to cops and devoted the second half of his time to a cop. In the video, the cop told a story about when he was interrogating a suspect. He told the suspect that he'd turn off the tape recorder (which he did) and things would be "off the record". He also said that it didn't matter -- the actual recording that would be used was hard-wired into the room.

Much like with police cars and surveillance systems where everything is basically digital, I strongly presume that in most places recordings are being kept digitally. Tape recorders provide visual cues for Hollywood that are better than modern digital technology, and they do the same for officers. For actual recordings or covert surveillance that you mentioned, I don't think tape is really out there anymore for audio recording and storage.


Does a self-signed certificate offer more protection than a public key? I'm having a hard time finding the practical benefits of transmitting a self-signed certificate over simply a public key. I understand that a self-signed certificate proves the integrity of the public key and user ID (i.e. proof the signer is able to decrypt messages encrypted by the public key), but what is the practical security benefit of this knowledge?

From what I understand, both sending a self-signed certificate and sending simply the public key are susceptible to man-in-the-middle attacks:

Alice and Bob want to communicate privately.
Mallory is between Alice and Bob.
Mallory intercepts the public keys or self-signed certificates.
Mallory creates a new keypair, and self-signs in the case of self-signed certificates.
Mallory relays the new public key or self-signed certificates to Alice and Bob respectively.
Mallory is now transparently between Alice and Bob.
Alice and Bob mistakenly believe their communicate is private.

Since Mallory can just generate new keypairs and self-sign, a man-in-the-middle attack can be executed quite as straight forward as if solely public keys were exchanged.

I've also considered a denial-of-service attack, where Mallory can corrupt the user ID or public key so that Alice/Bob can not decrypt messages and thus not communicate. A self-signed certificate would allow Alice/Bob to tell that there is a corruption as soon as the certificate arrives, whereas a public key by itself would only be discovered as corrupted once the response is received.

However, Mallory gains no benefit in performing this convoluted denial-of-service: Alice and Bob are not revealing secrets as they are not able to communicate, and simply blocking network packets is a far simpler way of denying service to Alice and Bob.

So in what practical ways is transmitting a self-signed certificate more secure than transmitting a public key by itself?

See also

  Public keys without the certificates

None. There's no security benefit.

Tools like openssl are designed to work with certificates. This means it's more practical to give openssl a self-signed certificate rather than no cert at all.