Extract pre-master keys from an OpenSSL application

Note: as of OpenSSL 1.1.1 (unreleased), it will be possible to set a callback function that receives the key log lines. See the SSL_CTX_set_keylog_callback(3) manual for details. This can be injected as usual using a debugger or a LD_PRELOAD hook. Read on if you are stuck with an older OpenSSL version.

For a walkthrough of the LD_PRELOAD approach for Apache on Debian Stretch, see my post to Extracting openssl pre-master secret from apache2. Technical details follow below.

If you have just gdb access to the live process or a core dump, you could read data from data structures. It is also possible to use an interposing library.

In the following text, the basic idea of key extraction using GDB is described, then an automated script is given to perform the capture.

Using GDB (basic idea)

Based on this Stackoverflow post, I was able to construct a function that could print a line suitable for Wireshark's key logfile. This is especially useful while analyzing core dumps. In GDB, execute:

python
def read_as_hex(name, size):
    addr = gdb.parse_and_eval(name).address
    data = gdb.selected_inferior().read_memory(addr, size)
    return ''.join('%02X' % ord(x) for x in data)

def pm(ssl='s'):
    mk = read_as_hex('%s->session->master_key' % ssl, 48)
    cr = read_as_hex('%s->s3->client_random' % ssl, 32)
    print('CLIENT_RANDOM %s %s' % (cr, mk))
end

Then later on, after you step upwards in the stack until you get a SSL structure, invoke the python pm() command. Example:

(gdb) bt
#0  0x00007fba7d3623bd in read () at ../sysdeps/unix/syscall-template.S:81
#1  0x00007fba7b40572b in read (__nbytes=5, __buf=0x7fba5006cbc3, __fd=<optimized out>) at /usr/include/x86_64-linux-gnu/bits/unistd.h:44
#2  sock_read (b=0x7fba60191600, out=0x7fba5006cbc3 "\027\003\001\001\220T", outl=5) at bss_sock.c:142
#3  0x00007fba7b40374b in BIO_read (b=0x7fba60191600, out=0x7fba5006cbc3, outl=5) at bio_lib.c:212
#4  0x00007fba7b721a34 in ssl3_read_n (s=0x7fba60010a60, n=5, max=5, extend=<optimized out>) at s3_pkt.c:240
#5  0x00007fba7b722bf5 in ssl3_get_record (s=0x7fba60010a60) at s3_pkt.c:507
#6  ssl3_read_bytes (s=0x7fba60010a60, type=23, buf=0x7fba5c024e00 "Z", len=16384, peek=0) at s3_pkt.c:1011
#7  0x00007fba7b720054 in ssl3_read_internal (s=0x7fba60010a60, buf=0x7fba5c024e00, len=16384, peek=0) at s3_lib.c:4247
...
(gdb) frame
#4  0x00007fba7b721a34 in ssl3_read_n (s=0x7fba60010a60, n=5, max=5, extend=<optimized out>) at s3_pkt.c:240
240     in s3_pkt.c
(gdb) python pm()
CLIENT_RANDOM 9E7EFAC51DBFFF84FCB9...81796EBEA5B15E75FF71EBE 6ED2EA80181...

Note: do not forget to install OpenSSL with debugging symbols! On Debian derivatives it would be named something like libssl1.0.0-dbg, Fedora/RHEL call it openssl-debuginfo, etc.

Using GDB (improved, automated approach)

The basic idea which is described above works for small, manual tests. For bulk extraction of keys (from a SSL server for example), it would be nicer to automate extraction of these keys.

This is done by this Python script for GDB: https://git.lekensteyn.nl/peter/wireshark-notes/tree/src/sslkeylog.py (see its headers for installation and usage instructions). It basically works like this:

Install breakpoints on several functions where new pre-master keys can arise.
Wait for the function to finish and write these keys (if not known before) to a file (which follows the SSLKEYLOGFILE format from NSS).

Paired with Wireshark, you perform a live capture from a remote server by running these commands:

# Start logging SSL keys to file premaster.txt. Be careful *not* to
# press Ctrl-C in gdb, these are passed to the application. Use
# kill -TERM $PID_OF_GDB (or -9 instead of -TERM if that did not work).
(server) SSLKEYLOGFILE=premaster.txt gdb -batch -ex skl-batch -p `pidof nginx`
# Read SSL keys from the remote server, flushing after each written line
(local) ssh user@host stdbuf -oL tailf premaster.txt > premaster.txt
# Capture from the remote side and immediately pass the pcap to Wireshark
(local) ssh user@host 'tcpdump -w - -U "tcp port 443"' |
    wireshark -k -i - -o ssl.keylog_file:premaster.txt

Using LD_PRELOAD

SSL/TLS can only negotiate keys at the SSL handshake steps. By interposing the library interfaces of OpenSSL (libssl.so) that performs said actions you will be able to read the pre-master key.

For clients, you need to interpose SSL_connect. For servers you need to interpose SSL_do_handshake or SSL_accept (depending on the application). To support renegotiation, you will also have to intercept SSL_read and SSL_write.

Once these functions are intercepted using a LD_PRELOAD library, you can use dlsym(RTLD_NEXT, "SSL_...") to lookup the "real" symbol from the SSL library. Call this function, extract the keys and pass the return value.

An implementation of this functionality is available at https://git.lekensteyn.nl/peter/wireshark-notes/tree/src/sslkeylog.c.

Note that different OpenSSL versions (1.0.2, 1.1.0, 1.1.1) are all incompatible with each other. If you have multiple OpenSSL versions installed and need to build an older version, you might have to override the header and library paths:

make -B CFLAGS='-I/usr/include/openssl-1.0 -DOPENSSL_SONAME=\"libssl.so.1.0.0\"'


How can I store and manage my GPG key pair securely?

Are both of the following commands returning the ssb key
(17832C40CF826BA9)?
gpg --export-secret-keys 17832C40CF826BA9
gpg --export-secret-subkeys 9AB628FC04C23871

Yes, in addition to other things: The first command exports the full key - the primary key pair (public+secret) and its subkey pair (even though you specified the subkey ID, gpg exports the entire key because of the command used). The second command exports almost the same thing, the only difference being the primary secret key it exports is unusable.

Is it fine to remove the key 9AB628FC04C23871 from my system, after I backed it up on the drive, created above?

Yes. But take care that the output of the first command export-secret-keys is the one you have saved.

Should I save a revocation certificate with it?

If your use case doesn't involve sharing the public key with others so they can encrypt documents to you and/or verify documents you sign, there's no web of trust and hence no reason to be concerned with the revocation certificate.
A revocation certificate is used by importing it to your keyring (you can also revoke any key pair in your keyring using the --edit-key menu) then uploading the revoked key to a keyserver and/or sending it (and the revocation certificate) to any of your contacts with whom you had shared the public key. The certificate itself contains instructions on how to apply it.

I've read that the process of changing the expiring value creates new certificates. Can you confirm this?

Changing expiration dates has no effect on revocation certificates. The only times a revocation certificate is created is when you first generate the key and if you manually generate a revocation with --gen-revoke

I want to have two certificate stores like this on different locations. I'd renew the key on a yearly base.

Given that your use case seems restricted to securing personal data in cloud storage there is little reason to set an expiration date at all. If you ever want to change to a new key, you'll still have to retain the old one (or else unencrypt everything made with the old key then re-encrypt it with a new one).

Should I use paperkey or the same digital method above?

Be advised that if you use Paperkey it saves only the secret key data without any accompanying public key material. The public key will be required when you want to import the secret key to gnupg. The best thing you can do is use a strong password for your private key.
The only thing you might consider is a larger RSA public key. Your current key is 2048-bit  the default key size in GnuPG is now 3072-bit. While it is probably far outside the realm of possibility for any entity to factor a 2048-bit RSA modulus, that probably won't remain the case forever.


How secure is Slack for sensitive information compared to other alternatives like Mattermost?

There are some additional points that are worth mentioning:

Cons

You have to trust Slack team because they have access to all your messages and conversations
You need to check that former employees or third people don't have access to your Slack chats (you need an additional robot)
Slack servers are available from any device including those security settings of which you can't control (for example, devices out of MDM). You can't use additional protection in this place (corporate VPN etc.)
In case of Slack you can't enforce password strength and rotation policy, can't link Slack with, for example, your corporate Active Directory authentication system. In case your data is compromised, it will be hard to investigate the incident as you don't have any logs etc.
There is no Slack Enterprise version available at the moment

Pros

Slack can be cheaper to set up and maintain
There is a guarantee that new versions will appear and developers won't drop support for the project (sometimes it happens in open source world)
You'll have technical support from Slack team
There is a chance that Enterprise version will appear and will have some enterprise security features

In my opinion Slack is better for small teams and startups, but in case of big companies you'd better choose something you have control on. Anyway, it's up to you - just evaluate the risks and find out what you can and can't accept in your situation.

Update: As it's noticed in comments, point 2 of Pros is quite arguable - just recall the story around Google Reader.


How to synchronize a nonce between two participants?

The simplest solution is to just send the nonce along with each message.  There is no requirement that the nonce be secret, and in fact, in the protocol you're describing an attacker would likely be able to guess the nonces anyway.  If you just transmit the nonce then there's no need for synchronization—all the server needs to do is guarantee that it won't reuse a nonce.

Note that don't need to literally send the full nonce along—any value that you can send that can be used to reconstruct the full nonce will do.  For example, if you're using a counter for the nonce, it's ok to use a compact, variable length encoding of the counter.

There are other tricks you might want to consider to reduce the need for synchronization between the two participants.  For example, if the communication is full-duplex (i.e., both participants alternate roles as sender and recipient), you might want to either:

Use separate keys for each direction of the communication—Alice uses key #1 to encrypt her messages to Bob, and Bob uses key #2 to encrypt his messages to Alice.  That way if either mistakenly uses the same nonce as the other does, there's no problem because of the use of different keys.
Make the parties use distinct subsets of the nonce space.  For example, the server might use only even-numbered counter values when it encrypts messages, and the client odd-valued ones only.

These additional suggestions share a theme with your question: reliance on synchronization is risky.  Ideally you want make sure that the participants in the protocol can act safely even if they don't know their counterparts' state.  Or even worse, you want them to act securely even if their counterpart is in fact an adversary.


Sending encrypted data through android app

There are a number of problems with your scheme. Let's start with the most basic, though:
What you're trying to do cannot be done.
As I understand it, you're trying to hide a symmetric token (authkey), that is present in every copy of your app, such that somebody cannot write another client for your web app. This is impossible.

Your app has to contain the token. Anybody who can obtain the app - which, if it's on the Play Store, is anybody at all - can reverse-engineer the app to get the token out. Encrypting the token doesn't help; the attacker only needs to know the token in the same form that the app knows it.
Your app has to use the token. That use is implemented in code. No matter how you implement it, that same code will be available for an attacker to simply copy into their own app (even as a binary blob, if needed).

Ok, on to specific flaws in your scheme.
Replay attacks
Your scheme has no nonce, and thus the attacker can simply send the same auth_token + time_token pair that were captured from the app's traffic. The time_token will still decrypt to the same rand_number used to encrypt the authkey. This is the most trivial way to break the scheme; the attacker doesn't even need to know what the tokens mean.
Key size
You don't mention the length of the authkey, or how it was generated. Given that you explicitly mentioned this for the rand_number, but instead gave a very short example token for the authkey, I'm worried it might be too easy to brute-force. The attacker doesn't need to know what the decryption key is if the entire range of values to search is small enough.
Performance
Your scheme requires two asymmetric cryptographic operations per message, one client-side and one server-side. Asymmetric crypto is computationally quite expensive. Schemes that use it almost always either merely perform a single key exchange using it and then re-use the exchanged key for all actual communication, or are for asynchronous operation (like email or verifying download integrity) where performance is not important.
Re-implementing the wheel
If you're only worried about communication security (which is usually not actually sufficient when there's a static token involved, but it is the only threat vector you explicitly mentioned...) then you should just be using TLS (HTTPS, since the message format is HTTP). If you want to be extra secure, pin the server's certificate (or at least the subject public key info from the cert) in your app, so that an attacker can't do things like install a new trusted root certificate and use its private key to spoof your server's cert.


Allowing access to a programmatically created user without compromising security?

Sending passwords in a mail is not good under any circumstances. As I see, the best thing you can do is send a one time link in the email so then user navigates to your externally exposed interface(web?) and create a password. To secure the email you can use PGP.This is a good guide.

This requires some burden of configurations in the user's end. If you are sending mails to known users, ( eg: in a corporate environment ) this is feasible. But if you are sending mails to general public, I do not think PGP is a viable option.


How to Perform a Security Review of Apps from Mac App Store

The most basic review is stuff already provided by Apple, such as "what permissions does the app request" and "who published it". You can also instruct the OS to not give the app some of the permissions it may want. All apps published through the store are digitally signed, so you can be sure that the version you install is the one that was uploaded to Apple. However, Apple's review is not guaranteed to catch all the ways somebody might do something malicious with the app.

For completely black-box testing an app, the best option is network review; set it up behind an intercepting proxy such as Burp Suite, set the proxy on the device, and start capturing traffic. Verify that the app is using HTTPS or other encrypted protocols. For HTTPS, verify that it is validating TLS certificates both by signer and by host. If the app uses certificate or key pinning, you won't be able to decrypt the data (though you can still tell what hosts it tries to connect to), but if not you can decrypt the data after you install the proxy's root CA certificate on the device. This will let you see if the app is sending any data it shouldn't or to anywhere that it shouldn't.

If you want to actually inspect the app's on-device data and files, you'll need a jailbroken device. Using a privileged file browser, you can retrieve the install files and check them for any libraries from untrusted authors or known-vulnerable versions. If you really want to, you can throw the files into a disassembler / decompiler and figure out exactly what they do, though this is a skilled and somewhat laborious process even if the files aren't obfuscated at all. You can also retrieve any data files (including database files) the app stores in its sandboxed part of the file system. You can check if any sensitive data is encrypted (although its of limited use to do so unless the app itself demands a password at startup; there's nowhere to store the key that an attacker with root access couldn't access with enough effort), or if it's storing data you don't expect it to. You can also check the app's keychain, and see what's stored there.


Data masking is a common request and various vendors have their own solution or rely on 3rd party solutions. You could implement your own (for example using hashing) but as you pointed out, it would be difficult to maintain data integrity, constraints and formats. For example, you might have a national identity column which is validated using modulo X (or some other verification mechanism). How would you mask such columns but preserving the validation?

Using tools already available will solve this for you. Just as an example of features such tools offer (and I'm not implying you should use this specific tool), take a look at Oracle own solution for data masking and subsetting. Quote from Data Sheet describing some of the data masking formats supported:

  Encryption encrypts the sensitive data using a key while preserving the format of the data. This transformation is useful when
  masked data sent to a third party has to be merged back along with
  further updates.
  Format Preserving Randomization (or auto mask format) randomizes the data, preserving the input length, position, the case
  of the character (upper or lower), and special characters in the
  input.
  Conditional Masking masks columns according to different conditions. For example, identifiers that belong to the United States
  can be masked using Social Security Number format and those that
  belong to the United Kingdom can be masked using National Insurance
  Number format.
  Compound Masking groups and masks related columns together. For example, if you want to shuffle address fields like city, state, and
  country, then grouping city and the state will keep these columns
  together during the shuffling process.
  Deterministic Masking generates consistent masked output for a given input across application schemas and databases. This makes it
  possible to mask names consistently or deterministically across different modules across your organization.

I believe these formats fit into your requirements.

Regardless of your RDBMS architecture, the point is the request is common, there is a market for it and on this market there are various vendors who provide the solution. You should choose one of them rather then reinventing your own.


Is there a form to know the PostgreSQL version using nmap?

The pgsql-brute script performs password guessing (brute-forcing) against PostgreSQL databases. It does not retrieve any version information or display anything but successfully-guessed usernames and passwords.

A more useful command for getting information from a PostgreSQL database would be:

nmap -p 5432 -sV myhost

This performs service and application version detection on the database, and results in output like:

PORT     STATE SERVICE    VERSION
5432/tcp open  postgresql PostgreSQL DB 9.1.12 - 9.1.14

This is based on a line number in an error response to a probe to that port, and can usually narrow down the version to one of a handful of minor versions.


Key management for builds for embedded systems

The normal approach is to store your private key in a manner in which it can't be compromised by a human being. Meaning, a disgruntled employee can't just steal it and then create malicious updates which have been signed with your key.

You will want to look at Hardware Security Modules which are capable og being fed your update or install image and and outputting the signature. That hardware device is then maintained under strict physical security ("locked in a safe").


Why ASP.Net Identity sends sensitive information to clients?

DISCLAIMER: I am not a Microsoft employee and I do not know the exact causes of presenting ASP.NET Identity. My opinion base on documentation and some practical experience.

The main purpose of presenting ASP.NET Identity by Microsoft was to make the authentication mechanism more generic, reusable and testable. There also show up some new ways of user authentication using social media which have happen to be more and more popular and would be nice to support.
To achieve those goals some standard behaviors have been implemented in library like Cookie-based authentication (using OWIN middleware), where some user data are put into the encrypted token and provided as authentication cookie.

I think that you are correct in your exact use case. But thinking of different scenarios that ASP.NET Identity can be used, you can use the same code and same mechanism (after some easy reconfiguration), which means that it is quite easy to change the behavior of an application.

If you would like to change authentication mechanism to your idea, you still can do this without breaking the rules of ASP.NET Identity. The only you have to do, you need to implement some interfaces presented with ASP.NET Identity and add this to your app stack.


Correct way of establishing if the switch is sending syslogs?

I'm sorry to say that @Don Simon is wrong too.

Syslog defaults to udp which will not show up on a nmap -sV scan. syslog over tcp is a rather new implementation and most network gear will not use it.

If you would like to detect the syslog packets, you could sniff your network traffic between the switch and your remote syslog server. You will see UDP packets with a destination port of 514.


Cryptography behind chip based credit cards (smart cards)?

Exact cryptography depends on the bank. The communication standard (ISO 7816) is flexible and does not mandate specific cryptographic algorithms. In practice, you would find the two following models:

The card does symmetric cryptography only (symmetric encryption, MAC). The card has a static identifier (which contains, roughly speaking, the card number and similar information) which has been signed by the card issuer (a copy of that signature is stored by the card, who sends it to the payment terminal). The card chip contains a secret value which is also known to the bank; that secret is used as a key for a MAC computed over the transaction details.
The card can compute digital signatures. It contains a private key which is known to no other entity (in particular, the issuing bank does not know the private key either). The corresponding public key is certified by the bank, i.e. the bank has signed a package containing that public key and the card ID. That signature is stored on the card, and sent to the payment terminal.

In both cases, the PIN code is sent to the chip, to convince it that its rightful owner is present, and therefore the chip should MAC/sign the transaction details. Main differences between the two models are:

In the second model, the payment terminal can make sure, in an offline way, that it talks to a genuine card, by verifying the bank signature over the card ID and public key, and then the card-computed signature itself, using the card public key and the known transaction details. With the first model, the payment terminal cannot make sure that it talks to a genuine card; through the bank signature over the card ID, the terminal can make sure that a valid card with that specific ID exists, but the card actually inserted in the terminal could be a clone, who spews out random junk instead of the expected MAC.
In the first model, since the bank has a copy of the card secret key, it could theoretically frame the card owner, by computing fake transaction. With the second model, the bank can claim that since it does not own a copy of the card private key, transactions are necessarily genuine. This may matter in a legalistic point of view, in case of litigation between the bank and its customer. This is not an absolute -- we enter here a lawyer-infested battlefield, where mathematics are just an element among others -- but the second model may reduce costs for the bank, through their own insurance system against payment defaults.
Cards which can do asymmetric cryptography are more expensive than cards which can do only symmetric cryptography. The price difference has much reduced; an order of magnitude would be 0.1$ vs 2$ (building price). However, the card market has a huge latency because changes in the protocol can percolate only after cards have expired and payment terminals have been updated (the latter being the slowest of the two).

Ten years ago, all payment cards in France ("Carte Bleue") followed the first model. This meant that a payment terminal, typically offline, could be fooled with a card clone, containing a copy of a valid card ID, but producing only junk instead of the MAC. So in practice, terminals would allow offline mode only for small amounts (say, less than 60$ or so) and even then would require online mode (the terminal asks to be stowed on its charging base, which contains a modem or ethernet plug) on a random basis. In later years, WiFi and 3G allowed terminals to go online much more frequently, in a smooth way (the restaurant waiter can stay with the customer, at his table, for the duration of the operation).

There was also, around 2000, another issue, which was that the signature from the bank (over the card ID) used 320-bit RSA. Factoring a 320-bit modulus is computationally easy. A self-taught engineer called Serge Humpich noticed that, factored the modulus, created a fake card (i.e. not a clone at all, but still accepted by offline payment terminals), and thought himself a genius. He then tried to "sell his expertise" to the group which manages the banking smart card standards in France (anecdote has it that he contacted them through a lawyer, and he selected a blind lawyer so that the man could not describe his facial features, should legal hijinks ensue). The said group thought it was blackmail, and called the police. Humpich was arrested during a demonstration involving a couple subway tickets with a fake card (the dozen innocent bystanders suddenly turned out to be a dozen police officers in plain clothes). The whole story was a conflation of blunders: a 320-bit RSA modulus, in 2000, was certainly a mark of severe incompetence (even when it was chosen, in the late 1980s, it was already too small with regards to known academic cryptanalytic results); and the ludicrously wacky details of Humpich's actions showed that he was quite severely deluded on the novelty of his "findings".


What Certification(s) to look for in Hard Drive destruction?

You could look for a company that uses DoD 5220-22.M standards for data destruction : http://www.dss.mil/documents/odaa/nispom2006-5220.pdf

I use a company that says they meet that standard.


Investigating a possible rogue DNS Server (maybe DNSChanger)

Does anyone know if this server is known to be rogue? If not, what
  should be done to test whether it is?

Using tools like the one you used is one way to see if your DNS has been hijack, but it is not foolproof. In this case you may have a false positive as the address in question is owned by serverius. They offer a wide range of network services which could legitimately explain why that IP was set as the routers primary DNS server. I would investigate if they are a vendor to your organisation (or network provider). There is a chance that the popups and DNS settings are unconnected. (Although they may indeed be connected, obviously I do not have the information to make that judgement.)

  Finally, if it is, what next?? Do we submit to CERT?

If the IP was in fact being used for malicious purposes you could submit it to various groups to investigate and have it placed into their blacklists or reputational databases. Examples of these groups are amongst others, the various national CERT groups. Gaining access to the CERT groups is different depending on what country you are in. The UK for example allows users who belong to various organisations across industry and government to apply to join online.

Many vendors such as Symantec and McAfee also allow you to submit IP's you believe to be rouge for investigation, normally via their customer portals.

The information on these lists is only of use if you have both access to it and make use of it in some way. Even then black lists and reputational databases are not a magic bullet as malicious IP's are an extremely dynamic entity and one that is very easy to subvert.

For quick investigations I usually use the Internet Storm Centre IP checker and IPVoid which is a meta site for many open source intelligence based blacklists and reputation lists. This is a good way to quickly find out if a particular IP is known for being connected to malicious activity.


What are the common features to identify XSS attack from Apache log file?

There are a couple of things I want to establish first before I give you general advice for spotting common characteristics of cross-site scripting (XSS) probing attempts in your logs.

I am assuming you will be manually inspecting your logs;
Without loss of generality, let's also assume you are able to keep up with the number of entries in your logs. So there are not thousands of requests hitting your server at any given moment.

These points should make it easier for me to describe the process by which you can spot XSS payloads in your logs.

The first issue I would like to tackle is trying to reduce the common characteristics down to a small handful of payloads. XSS is such an immensely large group of attack vectors and scenarios that even renowned security researchers often struggle to agree what term is appropriate to describe a particular type of XSS vulnerability. This can actually be seen in your question, you are trying to reduce XSS down to three specific payloads (

 (" → )
 (" →  < → )
 (</script → <\/script)
 (</script → <\/script)
 (</script → <\/script)
 (</script → <\/script)
 (</script → <\/script)
 (</script → <\/script " → \")

My resulting payload which covers all the contexts above was:

javascript:"/*\"/*`/*' /*</template></textarea></noembed></title></style></script>-->&lt;svg onload=/*

This payload is known as a polyglot; i.e. the payload covers multiple contexts at once. To break out of the first context (

), I had to use double quotes. "><svg onload=alert(1)> alone would have worked in this context.

Next, let's pick the second-to-last case: . To keep things simple, we will ignore the filter that was implemented in the challenge. This is a JavaScript-context and */alert(1)/* would break out of the multi-line comment.

For the last case that we will look at, I would like to include the filter.  (" →  < → ) replaces double quotes and the < character. To bypass this filter, one can simply HTML encode the < character: &lt;img/src=x onerror=alert(1)>. This results in ">. This case did not require breaking out of the context.

Notice how with just three contexts alone I was able to show three very different payloads. So before attempting to use your logs to find XSS payloads, make sure to familiarise yourself with a few common contexts.

+-----------------------------------------------------+----------------------------------+
| Context                                             | Example payload                  |
+-----------------------------------------------------+----------------------------------+
|

                     | "><svg onload=alert(1)>          |
+-----------------------------------------------------+----------------------------------+
|                     | */alert(1)/*                     |
+-----------------------------------------------------+----------------------------------+
|  (" →  < → )` | &lt;img/src=x onerror=alert(1)> |
+-----------------------------------------------------+----------------------------------+

Probing from an attacker's perspective

This section covers how I as an adversary (more precisely, my personal experience as a bug bounty hunter) might go about probing for XSS vulnerabilities in your application and what this would look like in your logs.

Most notable bug bounty hunters that I can think of use a very basic probing vector for manually determining if user-input is reflected anywhere. So we might use something along the lines of '">foobar or '">foobar to quickly gather various endpoints that reflect these payloads. Note that the foobar bit is actually quite important. We want to be able to quickly search for our payload in the source code, so hunters like to use unique strings in their payload. From your perspective, this means we leave a trail of fingerprints that you can follow to see where we are testing for XSS vulnerabilities.

This brings me to the next probing characteristic, you will very rarely see someone testing a single endpoint once and then giving up. If an adversary is scanning or manually testing for XSS vulnerabilities, they will usually be very persistent and you should see a long series of consequent XSS payloads popping up in your logs.

In addition to all of the probing characteristics listed above, there is one last and very important one that must be mentioned, the use of JavaScript methods such as alert, prompt, and confirm. I am sure you have come across these before while reading up about XSS. When testing for XSS vulnerabilities, one might want to get an immediate indication of a vulnerability and the easiest way of doing this is to get a big prompt fire right in front of your face. It becomes immediately obvious that you have an XSS vulnerability when the modal shows up. Also, the rush you get from the payload firing in that way never gets old. :)

You could grep your logs for keywords such as alert, prompt, and confirm. That being said, this is definitely not foolproof since, depending on the context, it may be possible to adjust the payload and mask this keyword — this is commonly seen when attempting to evade web-application firewalls and filters.

The problems with spotting self-XSS and DOM-based payloads

The issue can be quite simply summarised as: The payloads do not always show up in your logs. Take this example of a DOM-based XSS vulnerability:

<!DOCTYPE html>

  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width">
  <title>DOM-based XSS example</title>

When navigating to http://example.com/test.html#javascript:alert(1) the payload should display an alert box, but when I check my logs all I see is the path.

GET /test.html HTTP/1.1

Hopefully, this example better illustrates some of the issues you might face when trying to spot certain XSS payloads in your logs.

The best advice I can give you for tackling this problem is to implement a Content Security Policy (CSP) with the report-to directive. Now whenever the CSP detects a policy violation, it will notify you at the report-to endpoint (see https://report-uri.com/ for a service that logs these errors via the report-to directive). You can also just implement the Content-Security-Policy-Report-Only header if you are only interested in logging errors.

A small idea for getting a feel for manually spotting XSS payloads in your logs

Here is a fun idea that might help you see all of what I have described above in action. Build a small XSS Capture the Flag competition (CTF) and get a group of friends that are security-oriented to try to find the XSS vulnerability in your application. During this time, take a look at your logs. Then do the same thing but with a scanner such as Burp Suite. After a while, you should get a feel for what basic XSS probing (for a certain number of XSS vectors) looks like in your logs.


Securing communication over LAN in VPS provider

Using an encryption layer between your infrastructure nodes is always a good practice and it's not overkill.

Absolutely always exists a risk of data leakage due to human or other factors. For example, imagine if one of your hosting company routers gets compromised or even worse - there is an ungrateful employee who decides to obtain sensitive information of his company clients. Both situations will permit attackers to view all the netflow traffic and steal data that is unencrypted including yours, allowing to access your further infrastructure.

Organising a VLAN using an encrypted tunnel is a traditional practice within information security area. There are lots of software that permit to do so, such as OpenVPN. It is not taking much time to setup a TLS tunnel and concerning about your sensitive data definitely worth that time.


Tool for public key cryptography where password is the private key?

Generating the private key deterministically from a password is known and works; however, it has the following drawback: anybody can then try to brute-force your password, since the public key is public. It suffices to re-run the key generation algorithm from any potential password and see if the result matches the public key.

We rarely want passwords to be on the front line, because we are humans with limited brains, and the passwords we are ready to remember are rather weak. The more common methods of using the password to encrypt the private key file allows to add some level of extra security by managing the locations where a copies of the private key file are kept (you want several copies, because the key must not be lost, but having a copy in safe somewhere is not the same as making it public).

Also, a private key file encrypted with a password can become two private key files , each encrypted with a specific password: this way, two people can potentially open the backups. After all, you worry about losing your data, but if this is in an enterprise setup, your colleagues might worry about losing the data as well -- and, from their point of view, data loss can take the form of you being run over by a bus. In the password-is-private-key configuration, protection against the loss of the password must be sharing the password, which is not very flexible.

If you do want to follow the password-is-private-key road, you will probably have to write your own tool. You would then find it much easier to use an ElGamal key, which is just an integer modulo a given value q (the group order, which is public). The OpenPGP format supports ElGamal keys, so this should be relatively simple to develop with an OpenPGP library, such as GnuPG.


File Carving Tools: There are several paid and free forensics tools available. For instance, Autopsy is an open source digital forensic tool and it does pretty good job in file carving. It has different type of viewer builtin like text view, string view, hex view and meta view. You can also perform keyword searches. (check their official website for detailed list of features/options available)

There are two other choices available for image carving. The first is the open source photorec. The second is the commercial tool Adroit Photo Forensics. Adroit will recover files that are fragmented and does a better job eliminating false positives, but it is pricey. In all likelihood you'll be fine with photorec.

Carving with Scalpel: This type of carving requires in-depth understanding of file types and their behavior for example a pdf file starts with
%PDF and ends with %EOF while a jpeg image file begins with 0xFFD8 and ends with 0xFFD9 Now with this information, you can configure scalpel to extract JPG/PDF files.


When GeoTrust CA issues certificate for the domain Google, does it also provide private key to Google by which the certificate is digitally signed?

In theory a CA does not even see the private key of the certificate it issues. To get a certificate issued one first creates a key pair and based on this key pair a certificate signing request (CSR). This CSR only includes the public key, not the private key. This CSR is then used by the CA to create a signed certificate which includes the public key. There is no need for a CA to have access to the private key of the new certificate at any time.

In practice some CA allow the user to create the key pair online and could thus in theory have access to the private key because they control the code how the key is generated. This is only to make it easier for technical less capable users. But probably no CA requires the user to proceed this way, i.e. they offer a way to create the key in private and only upload the CSR.


Is SHA-3 used in a Password creator like this safe to use?

From an information security perspective, you should consider your scheme public, as per Kerchkoff's principle, even more so since you disclosed your source code. Thus the security of your generated password are reduced to the security of their input. The algorithm chosen is irrelevant as an attacker would choose the input and use your algorithm to generate the password.

So let's evaluate the security of your inputs:

Master password: If an attacker find it, all your passwords are compromised. Do not reuse your passwords.
Site name: This is public knowledge, it doesn't increase security.
Password length: Contains very few entropy. It's almost useless.
Available characters: I guess you always use the same ones or use all the characters allowed by the website, as doing otherwise would be very inconvenient. Thus it is equivalent as a second small master password.

So, basically your scheme hash public information with a master password. An attacker only needs to successfully brute-force one of your generated passwords to recover your master password and then be able to easily find the other generated passwords.

A better solution would be to randomly generate your passwords. This way, they cannot leak information about the others. You could then save them in an encrypted file or database. You would only need to remember the master password of encrypted database, without having to remember the password length or how the website was called before they changed their name. Password managers allow you to do just that.


Log Attacks in Different Files using Snort

The Bad News

Not directly, not in the way you want. You can specify multiple alert outputs, as described in the Section 2.6 of the manual. However, this will simply send the same alerts to multiple locations. You'll still have alerts from signatures imported from both ddos.rules and log.rules logged together.

The Good News

Fear not, we can make it work. What you'll have to do is create multiple config files and have each log separately. Depending on exactly how you want to split up the traffic there are a couple of ways you can go.

MultiConfig

Snort has a mechanism built-in to allow for processing different packet streams against different config files. This can be useful if you segment your applications based on network or VLAN. That is, all mail apps are on one VLAN, web servers on another, etc. The full details are provided in Section 2.10 of the manual, but the most relevant bits to understand are that it's dependent on VLAN or network, and there can be no duplicates. Meaning this is a valid configuration,

config binding: /etc/snort/snort.conf-ddos net 192.168.100.0/24
config binding: /etc/snort/snort.conf-log net 192.168.200.0/24

but this is not.

config binding: /etc/snort/snort.conf-ddos net 192.168.100.0/24
config binding: /etc/snort/snort.conf-log net 192.168.100.0/24

To use this feature in your situation you would have to run ddos.rules against one set of hosts and log.rules against another. I doubt you want to take this approach.

Multiple Instances

To date snort is single threaded so when processing is intense enough to utilize one complete processor it will begin dropping packets. Thanks to this you will often find snort sensors with several versions of snort running. Generally this works by using something called PF_RING, which is like a packet load balancer, or by chopping up your network space and running one version of snort per block. So one instance of snortd monitoring 192.168.1.0/25 and another watching 192.168.1.128/25. In most cases each process will use the same config file but just only watch certain addresses.

We can take that approach and turn it on it's head a bit to work in your situation. Let's create two config files and set the relevant portions thusly.

/etc/snort/snort.conf-ddos

output alert_syslog: LOG_LOCAL0 LOG_ALERT
include $RULE_PATH/ddos.rules

/etc/snort/snort.conf-log

output alert_syslog: LOG_LOCAL1 LOG_ALERT
include $RULE_PATH/log.rules

Then we set up your syslog engine, restarting it after making the change of course.

local0.alert       /var/log/snort/alert-ddos.log
local1.alert       /var/log/snort/alert-log.log

Now when you run snort you do it twice, each using the -c option. The first time we'll use -c /etc/snort/snort.conf-ddos and the second with -c /etc/snort/snort.conf-log.

Now you should have two instances of snortd running, one only with the ddos.rules signatures and logging to LOCAL0 and one with the log.rules signatures and logging to LOCAL1. As long as syslog is behaving you should have your alerts logging separately with no co-mingling.


Can you still be tracked when using a VPN?

Using a VPN in and of itself isn't going to stop people who want to trace specific activities on-line.  A VPN encrypts the traffic from your machine to the exit point of the VPN network.

So what it protects you from is someone trying to look at your network traffic if they sit between you and your VPN provider (for example a correctly set-up VPN should prevent someone on the same wireless network as you from reading your traffic)

A VPN isn't therefore likely to protect you from an adversary like "Anonymous" unless they happen to be on the same local LAN as you.


Constant hack attempts for the past 2 days

Best way is to drop this on the firewall itself, since the IP address 46.183.120.20 seems to be in Albania, the best way would be to drop it for a while. (Unless you have admins in Albania).

A simple:

iptables -A INPUT -s 46.183.120.20 -j DROP

Would probably discourage their brute force attempts, and you can add them then again later on.

Alternatively, let your fail2ban script drop them on the ifrewall, not adding it to .htaccess; you don't need your Apache webserver to handle this request that you don't wanna deal with anyway.


If my phone for two factor authentication is stolen, can the attacker access my account via an email password reset?

As @Limit pointed out in the comment under your question, the attacker would first need to unlock your phone. Which is a problem. But not impossible. But for the sake of argument, let's assume the attacker stole your phone in an unlocked position (you were sitting at a coffee shop, checking the internet, put it down, turned to talk to someone, and someone else swiped it for example).

To answer in a word, yes. This compromise is completely possible, and part of the reason I personally have a slight disgust to the ways some companies are rolling out 2FA. Especially google, who knows for a fact that the user that uses their authenticator on their phone likely also has their email on the very same device.

The security trinity is supposed to be this:

Something you are (username, fingerprint, retina)
Something you know (password, key-file, other data blob)
Something you have (OTP generator, Keyfob, yubikey)

The problem is that the user desires convenience, and this always presents a threat to this model. The key factor in the way the trinity works is that these things aren't supposed to overlap.

don't use your username as your password
don't use the same key-file for your "password" as your yubikey's public key (or worse private)
etc...

However this line is becoming increasingly blurred, because many users desire all of their tech related tasks to be aggregated on one device. It creates the situation where a person turns a password (something you know) into essentially a device that autocompletes the password when required. Effectively turning it into something you have since you are no longer required to know it, simply posses the device.

This situation is made worse by allowing that same device to also house the software authentication mechanism that is supposed to turn the device into a replacement for keyfobs.

What this ultimately creates is the false sense of security where the user thinks they have the trinity of username-password-authenticator but really only has a username and two authenticators, and both of them are one device.

This is made even worse if you think about how much of our lives are linked to said e-mail account. Not just your email itself, but any service that also registered with that email likely also has a reset policy that utilizes it. Thus allowing the attacker to also transfer ownership of any services to themselves through your one device. Furthermore the email also provides an easy to use interface that will search for the services you have signed up for. All they must do is search for "please verify your email by clicking here:" and out comes a list of domains where you have attempted to register.

This is something I try to advise people on whenever I see it. Each security step must be segregated as much as possible within reasonable usability constraints. If you have the google authenticator on your mobile, don't allow your email to remember the password to your gmail. If you do allow it, use the authenticator on another device, such as an ipad, another phone or your computer. There are some great alternatives to traditional authenticators that work with phones and don't require any operations to be done on the phone itself. Like the Yubikey NEO which has NFC enabled. Even if your phone is stolen, the yubikey likely wont be (Please don't attach it to the key-chain that hangs from your phones...).

These methods keep each part of the trinity separated, and thus, mitigate the problem somewhat. It's always important to clearly define the line between each part of your security structure. Otherwise they start to mesh together until, as in this case, they effectively become the same part of that trinity, leaving a void where one of them once stood.


Failed to connect to netcat reverse shell

Put your cmd parameter before the null byte injection in the exploit URL. Null will cause your query string to terminate early.


You should be asking this question directly to your QSA as they will be determining what they accept as evidence for your compliance. The PCI Council does not necessarily bless or damn any particular application, if you find a tool that will help you easily manage or address a requirement, and its cost effective and accepted by the QSA, you'd have a win.


If you are not the web site author/admin, and you can (as a site user) add a stylesheet to a page, then this could be an attack vector--though arguably not a full-blown vulnerability.

For example, one could link a style to an image on a web server of their choosing and log the requests.  This would reveal data from all page visitors (IP addresses and user agents) which can then be used for further attacks.


How big is the risk of hash fixed points/cycles?

Suppose that you have a pseudorandom function with an output of n bits. A good hash function with a given salt ought to behave like a PRF.

The general, average structure of a PRF, with regards to repeated application, is to have one big "cycle": if you hash and rehash repeatedly, you will, at some point, enter a cycle and obtain a value which you already encountered. The average length of the cycle will be about 2n/2. The number of steps you will have to perform before reaching the cycle will be also about 2n/2. Almost all initial values will bring you, ultimately, to this inner cycle. There can be a few initial values which will not lead you to this cycle; there could even be some fixed points (there is a probability of 63.2% that there exists at least one fixed point); but the number of such potentially
troublesome points will be no more, on average, than 2n/2.

To sum up, if n is large enough that a probability of 2-n/2 can be neglected, then things are fine. In particular, if n = 256 (you are using SHA-256), then you can stop worrying. Even with MD5 (n = 128), short cycles are sufficiently rare that they will not imply any actual security issue.

A full-space cycle would mean that your hash function is not a PRF, but a permutation -- and a permutation with only one cycle. Building a one-way permutation is possible, but quite harder than a one-way function for which it seems that you can just throw together some bitwise operations and rotations; for a permutation, you need mathematics (e.g. with exponentiation modulo a non-prime integer) and this will imply some extra trouble (the permutation will be on average one-way, but there may be values for which it will not be).


Is there a way to exploit unused GET POST parameters in web application?

Under normal circumstances, your application should solely only handle the parameters you expect.  If you use a framework though, there might be additional security problems such as "Mass Assignment" (good -read on Ruby on Rails' -dated- security problem).

But, as in your example, if the application is only accepting "number" as a POST argument, it should not handle anything else.


Keepass2 vs KeepassXC - KeePass-Http connector - Mono in Linux

The mix of questions makes harder to reply, it would be preferable to ask different things as separate questions.

I will try to provide a general answer to the things you mentioned:

  Some would say Mono is a security risk. See https://sites.google.com/site/easylinuxtipsproject/security Is this correct? –

This page makes the point that by installing wine or mono, as that allows you to run more programs (like windows ones), they make easier to run malware, too. (As you could be infected by eg. a ransomware that was developed for a windows target).

YMMV, as you may find that the benefit of running program X outweights the added risk of higher exposure.

  I would have thought that I would be unlikely to ‘accidentally’ execute malware as root under Linux?

How unlikely you are to accidentally a malware of coke depends a lot on the person. Some people are very conscious and won't run as root anything not completely innocuous (or even they restrict themselves to their package manager), while others will do all kinds of stupid-to-harmful actions… as root (and should have no root privileges at all).

However, running malware != running malware as root. Even if it isn't able to run as root, a malware can do a lot of harm, for example a ransomware. Your system files (which could be recovered by reinstalling) will be safe, but your photos and personal data won't. This was expressed by Randall Munroe on his xkcd 1200: Authorization:

(image under a CC-BY-NC-2.5 license)

  Is KeePassXC-Browser better than KeePassHTTP ?

I haven't studied the extensions themselves, but from a brief look at the linked bugs, I think it is. If any, because KeePassHTTP has some shortcomings that KeePassXC-Browser was designed to fix.

Note however that you could be using either of them without any browser plugin, in which case this piece may not be important.

I personally prefer KeepassXC to Keepass2 under Linux as I find it to integrate better with the system overall. But in a security aspect, I would say that you can use both of them in a safe way. KeepassXC would probably be safer? Maybe. Although the difference may be marginal.


Why do we keep our keys secret, rather than our algorithms?

We don't just want secrecy. We want quantifiable secrecy. The point is not only to ensure confidentiality of data, but also to be able to know that we indeed ensured confidentiality of data.

When we have a public algorithm and a secret key, secrecy can be quantified. For instance, if we use AES with a 128-bit key, then we know that an attacker will have to try an average of 2127 possible keys (i.e. way too many to actually do it) to find the right one.

If instead we tried to use a secret algorithm, then we would face three big challenges:

As @Xander explains, making a secure encryption algorithm is awfully hard. The only known method which offers some decent reliability is public design: publish the algorithm, let loose hundreds of cryptographers on it, and see if they can find something wrong in it. If, after a couple of years, none of them found anything bad in the algorithm, then it is probably not too weak. With a secret algorithm, you have to do all that cross-review work yourself, which is not feasible in any decent amount of time.
We do not know how many "decent encryption algorithms" can exist. An attacker may try to enumerate the possible algorithms, were "possible" means "that which the designer may have come up with". Part of the issue is in the realm of psychology, so quantification will be hard.
A secret algorithm still exists as source code on some computer, compiled binaries on some others, and in the head of at least one designer. Unless all the involved development computers and the designer's corpse were dissolved in a big acid cauldron, it is very hard to prevent that "secret algorithm" from leaking everywhere. On the other hand, a secret key is a small element that can be more efficiently managed and kept secret (stored in RAM only, possibly rebuilt from a key exchange algorithm like Diffie-Hellman...).

Since the algorithm will probably leak, you may as well consider it public and rely only on the secrecy of the key. By actually making the algorithm public, you may then benefit from extended review by other people, which is an almost unavoidable precondition for achieving security.


Is this a secure way to hash passwords?

Probably, but a lot of your additional complexity isn't really adding much security value.

The idea of a salt is to ensure there is no method more efficient to recover your passwords than to brute force every single one of them from scratch. Therefore your goal is to ensure no two accounts have the same salt (ie. it is globally unique). If they did then an attacker could basically get two for the price of one by constructing a lookup table while brute forcing the first.

To achieve this you need enough entropy in your salt to ensure the probability of two accounts having the same salt is negligible.

  Randomly generated salt (using a function designed for generating password salts). This is stored in the MySQL database.

This is the primary source of entropy in your scheme. You could probably discard the rest without significantly compromising security.

However for this salt to be effective you must ensure:

The random value you're using is generated using a RNG suitable for cryptographic applications.
There are enough bits in your random value to minimise the probability of a salt collision. OWASP suggests at least 32 bits.

  Static salt - hard-coded into the source-code.

This is adding minimal cryptographic value. From a theoretical perspective it's not adding any additional entropy assuming the attacker knows the value, and from a practical perspective DB servers are usually compromised via credentials obtained from a compromised application.

  I'm using sha512, by the way.

While this is a cryptographically secure hashing algorithm, it's also a "fast" one which is a problem given that users typically use weak passwords. You should consider using either a slow hashing algorithm (eg. bcrypt) or iterate your hashing process 10k or so times. If it takes a second or so to calculate every hash then even a dictionary attack will take a very long time.

  When checking the hash, I generate 100 different hashes with the static salt, the salt stored in the database, and one of the possible random numbers, if one of them matches the password is correct.

The value of this is minimal, you're basically just adding one or two extra characters to the password then brute forcing them every time. This will slow your hashing scheme down which is good, but you'd be better off using the technique above to achieve this.


While Android is the "official" OS on the device, the underlying OS on all Android phones is Linux. It's also the most popular Linux OS currently in use on smartphones. So if you see a phone that gets identified as running Linux, then it's probably Android.

The catch is that Android phones typically have NO network listening ports open at all, unless there's an app running that does. So a portscan will reveal nothing.

Instead, you'd have to intercept any device-initiated traffic and watch for any clues, such as the User-Agent on an unencrypted web request.


Can certificates be extended and/or does the serial number change in that case?

No. This is not possible as you would be reissuing the certificate. Your serial number will change.


Why does OWASP recommend security questions?

Specifically with regards to security questions you are asking users to share potentially sensitive facts about themselves that are likely obtainable by social engineering of people near them.  That makes security questions at best inadvisable and at worst downright dangerous from a holisic security perspective.

OWASP, while good overall for explaining the problems, makes many inadvisable recommendations including security questions, input whitelisting, and XXE mitigations that prevent loading most XML documents.  Take the problem descriptions as they are, but really scrutinize their solutions before considering implementing them.


What steps should be taken to validate user uploaded images within an application?

Even it sounds simple to be done, image check could be tricky. At least following actions should be done:

check size of image - width, height and size of itself;
check extension and name of image;
check content type;
limit amount of uploaded images;
limit access to uploaded images - better if user don't know where it is stored;
limit types of image extensions;
preferably change name of image;

Also, I would recommend change type of image explicitly after those checks, e.g. from PNG to JPG. Simple, but works well.


How does blockchain client trust other nodes?

There are a couple ways to bootstrap a blockchain client node. The most used one is the DNS Seed. On most coin wallets, there are a list of hard coded DNS records that point to peers.

It's possible to download peers.dat from the blockchain official site, and have the client node connect to peers from that file. This is not the most used way, but works.

The least used way is to connect to the IRC channel for the blockchain, ask the users there and connect to the peer by hand. Not used today for the absolute majority of coins, but can be used for a little know, small ones.

After discovering and connecting to the first peer, the node will listen to transactions and build its own node list.

On the bitcoin software (and almost every other coin), there are a special block named Genesis Block that are hardcoded on the application. This way you can connect to any peer, trusted or not, and ask for the genesis block. Having the genesis block and a list of peers, you can ask different peers for different blocks and calculate their hashes following the chain, and see if their hashes match. With enough known nodes, it's practically impossible (mathematically possible but very, very unlikely) to nodes to fool your client. It's like having dozens of complete strangers that never communicated before telling you the same fake numbers on the lottery draw.


How to verify the signature value in a SSL certificate?

I'm sorry you don't need the private key to "decrypt" something encrypted using asymmetric encryption, you need the public key, which is in the certificate, or the certificate of the trusted authority.
Using the public key also provides proof that the private key was indeed used.

See this prior entry  Digital Signature and Verification?
and this:
how digital signature verification process works


After running a string through mysqli_real_escape_string, is it safe to remove the backslash before echoing it into a page?

You should never use anything "to reduce the likelihood of SQL injections" in the first place. It just doesn't work this way. You are not adding some code just in case, sort of a rabbit foot. SQL injections simply must be prevented, using certain tools. It is so easy to achieve that it just should go without saying. Just never add any variable to your SQL queries - that's all. Here is how. It doesn't only prevent injections but also relieves you from that escaping mess.

That said, no extra slashes should appear in the output unless there is an error in your code. Escaping is used for the query only, it is not stored in the database. So instead of removing extra slashes you must not add them in the first place.


What is the physical security (Evil Maid) threat model of a modern hardened laptop?

What you're describing is an Evil Maid attack. There are a bunch of ways you might go about gaining access in this scenario, but the main one is DMA.

M.2 would give you direct and complete access to the system memory over DMA, assuming that the IOMMU is not configured to prevent this, which it almost certainly will not be by default for a direct PCI-e link. The same goes if you have an ExpressCard slot. One toolset for this is PCILeech, which is capable of dumping the first 4GB of memory from a system without any OS interaction or drivers installed, and all memory if a driver is first installed.

It is also potentially possible if your laptop has Thunderbolt or USB-C, because both of those interfaces support DMA. Generally speaking those interfaces now tend to have hardening features in the firmware and drivers to prevent arbitrary DMA using the IOMMU, but this protection isn't perfect or universal, and there have been some issues (e.g. Thunderclap) that allow an attacker to bypass the IOMMU in some hardware.

What you might wish to do is enable Virtualisation Based Security (VBS) and Windows Credential Guard (WCG), which puts your entire OS in a Hyper-V hypervisor and shifts most of the LSASS service (which caches credentials) into an isolated virtual machine. There are few, if any, toolkits out there at the moment that allow an attacker to recover the cached BitLocker master encryption key from the WCG enclave using a non-interactive memory dump. This also allows you to enable Device Guard and KMCI/HVCI, which should make it exceedingly difficult for an attacker to gain persistence on the system from a one-time DMA attack.


Smart card + GnuPG: what is stored in my keyring/how to adopt smart card?

You need to run:

gpg --card-status

and gpg will do it for you:

/tmp$ mkdir gpgtmp
/tmp$ chmod go-rwx gpgtmp

/tmp$ GNUPGHOME=/tmp/gpgtmp gpg --keyserver pgp.mit.edu --recv F8713BDF
gpg: sleutelring ‘/tmp/gpgtmp/secring.gpg’ is aangemaakt
gpg: sleutelring ‘/tmp/gpgtmp/pubring.gpg’ is aangemaakt
gpg: opvragen sleutel F8713BDF van hkp sleutelserver pgp.mit.edu
WARNING: gnome-keyring:: couldn't connect to: /run/user/robin/keyring-55e9Rg/pkcs11: No such file or directory
gpg: /tmp/gpgtmp/trustdb.gpg: trustdb created
gpg: sleutel F8713BDF: publieke sleutel “Robin Sheat <robin@kallisti.net.nz>” geimporteerd
gpg: no ultimately trusted keys found
gpg: Totaal aantal verwerkt: 1
gpg:               Geimporteerd: 1  (RSA: 1)

/tmp$ GNUPGHOME=/tmp/gpgtmp gpg --list-secret
*nothing here*

/tmp$ GNUPGHOME=/tmp/gpgtmp gpg --card-status
*snip boring card info*
General key info..: pub  2048R/F8713BDF 2013-09-24 Robin Sheat <robin@kallisti.net.nz>
sec>  2048R/F8713BDF  aangemaakt: 2013-09-24  vervaldatum: nooit
*etc*

/tmp$ GNUPGHOME=/tmp/gpgtmp gpg --list-secret
/tmp/gpgtmp/secring.gpg
-----------------------
sec>  2048R/F8713BDF 2013-09-24
      Serienummer van de kaart = ....
uid                  Robin Sheat <robin@kallisti.net.nz>
*etc*


How can I prevent Facebook from gathering data when using it on Linux?

Use a separate browser with a different fingerprint from your main browser. For example, if you're using Linux with Chrome for your daily browsing, use Windows and Internet Explorer for Facebook (in a VM for example). In the VM set a slightly lower screen resolution so they can't track you on that either.

Finally in your daily browser use an extension such as uBlock Origin with an aggressive filter list (tracking lists and "anti third-party social") to prevent your daily browser from loading Facebook's tracking scripts such as the like buttons.


Sending credit-card numbers through SFTP?

Regardless the file transfer protocol, I would recommend encrypting files before sending them out. You may use for example pgp encryption, it can be automated.


Specify Amount of type of character Crunch

There's no way to do this with Crunch that I'm aware of.

The only way I know of to do this is the generate all masks that conform to the composition rules using the policygen tool from the PACK toolkit, as described in my answer here.

In your case, it would be something like:

$ policygen --minlength=8 --maxlength=12 \
  --mindigit=2 --minlower=1 --minupper=3 --minspecial=1 -o test1.masks

You'll have to adjust the minimum and maximum password length accordingly. The number of masks can go up rapidly with the maximum length, so adjust carefully.


Are there any exploits that would allow a user to "spoof" a screen capture system?

If you're asking if a user could display an arbitrary screen, then yes.  Any software that can display a full-screen image (ie: display an image not in a window, but that displays over the full real-estate of the screen) is capable of spoofing your system.  Most photo editing software can do this, as can pretty much any video playback software, or even web browsers.  I could photoshop up a credibly accurate image of a normal desktop with whatever else I wanted to display. If you are asking if the user could cause the capture system to capture one image while the same user was actually doing something else, then this depends on your image capture software. Without administrator privileges, probably not directly on the workstation.  I would probably have to attack the system by hooking up one of my own computers to the network and intercepting the network requests of the server to the client.  It probably wouldn't be too hard for me, if I had access to your network, to hook up a laptop and sniff your network to find out the protocol your central server is using to request screenshots, and then build some software that responded as if it were one of the workstations and give back an image of my choosing.  Your system needs to do bidirectional authentication with some sort of protocol that is not subject to man-in-the-middle attacks, like SSL.


How does Malware use Unregistered Domains

As the previous two mentioned the malware wasn't connecting to the unregistered domain name. The unregistered domain was a technique that was supposed to be used to prevent analysis of the malware.

Often times when malware researchers are picking apart malware they do so in a virtual machine. The way certain virtual machines resolve domain names result in unregistered domain names being considered "resolved" to applications running on the VM. Advanced malware authors know this and so before they infect the machine they will often attempt to connect to random unresolved domain names. If these domain names get resolved then it signals to the malware that it is functioning on a virtual machine. If it detects this then it will abort and refuse to infect the machine. This makes analyzing the malware more difficult.

In the case of the WannaCry malware the author attempted to create a similar countermeasure, but did so incorrectly. He hard coded the unregistered domain names. So once the researcher pointed the hard coded domain name to a DNS sink all the infected computers thought they were infecting a VM. So instead of starting the encryption process the malware just exits to prevent further analysis.

Had the author randomized the domain name check, registering the domain name would have done nothing.


How to prevent ISP impersonating me to send email?

If an entity other than you (e.g. your ISP, or your email service provider) has control of the domain of your email address (e.g. comcast.net, or gmail.com), then it's possible that this entity could forge messages from you (in that these messages would appear to have been sent from your email address), and do so in such a way that these messages would pass all SPF, DKIM, and DMARC checks.

However, if it is your practice to digitally sign messages that you send (e.g. using S/MIME or GPG or PGP), and your recipients are in the practice of verifying the digital signatures on your messages, then this would prevent these entities from forging messages from you, assuming that these entities do not have the private key that you use to sign your messages.


Why do client certificates not protected against improper server certificate validation

The certificate-based client authentication in SSL takes the following format: the client shows its own certificate chains, and computes a signature over all the handshake messages previously exchanged. These handshake messages include the message where the server sends its certificate chain to the client.

In that way, the server can make sure that the client (the one who owns the shown client certificate) really saw the correct server certificate, not a fake one. In that sense, requiring certificate-based client authentication can prevent a true Man-in-the-Middle (if your "testing" shows otherwise then either you broke SSL, or there is something subtly wrong in your test situation). However, the Devil is in the details...

A MitM attack is a simultaneous double impersonation: the attacker runs both a fake server (when talking to the client) and a fake client (when talking to the server). A true MitM is prevented as long as either the client or the server correctly verifies the certificate from the peer, which entails validating the certificate (verifying that it has been issued by a trustworthy CA, and so on) and also using the name in the certificate as identity of the peer (a client connects to some IP address but the certificate is about a guarantee for a server name).

However, there are other kinds of attack which, while not being MitM in a strict sense, can be troublesome. For instance, if the client does not perform a correct validation of the server's certificate, then the attacker can run a fake server and talk to the client. The genuine server is not involved at all ! That kind of thing is not a genuine MitM, but can be sufficient to recover some client secrets (the client believes that it connected to the correct server, and then sends sensitive data through the tunnel).

There is nothing that the server can do to ensure that a client always validates the server's certificate properly -- in particular when the server is not contacted at all, because the client is connecting to a fake, attacker-operated server. At best, the server can make sure, through certificate-based client authentication, that the client for this time uses the correct server public key for its cryptography.


Why is the password hashes in my database the exact same for all my users?

The short answer is, you obviously have a (critical) bug in your hashing code. As it stands, anybody can probably sign in as any user at all, because for some reason your password hash function always produces the same output.

There's a few different ways this bug could come about, none of which are inherently obvious. One thing to definitely consider, though: are you actually including the password in the data you feed to the hash function, or are you terminating the hash without adding the password?

Another risk is that the bug is actually in your database code, and when you're creating / updating a user's password hash, you're accidentally setting it for all users, rather than just the specific one being created/updated. This is highly exploitable: I create my own account, and then use my password to log into an arbitrary user's account, because their hash got changed to mine.

OK, enough hypothesizing. The actually important stuff here is that, frankly, you should not be writing your own authentication system. Even leaving aside a critical bug that really, really should never happen - you should have found this during development or unit tests, if nothing else - your system is just not secure.

You shouldn't be using MD5 for anything, it's got known weaknesses and is considered cryptographically broken. You should be using the SHA2 family of hash functions for anything that needs a fast hash; even SHA1 is considered deprecated for new code.
You should never use a fast hash for storing passwords. Passwords should be hashed using a deliberately slow and expensive hash function, so that if somebody dumps your password database, it's hard to brute-force the password hashes it contains because each round of the brute-forcing takes a while.

Using a single-round fast hash, commodity hardware (like high-end graphics cards) can compute billions of hashes per second, enabling absurdly fast password-cracking. Instead, use functions that are intended for key generation or password storage.
PBKDF2 is the most common, but is also the oldest and least-amount better than a single fast hash. Bcrypt is better, but against modern hardware it's not really very good anymore. Scrypt, with the right parameters, is the generally-accepted modern option. There also exists Argon2, a new algorithm designed specifically for password hashing, and the winner of a three-year competition to determine the best such algorithm, if you want to be cutting-edge.

You don't mention per-user salts; are you using them? Salting passwords is absolutely essential, otherwise an attacker can just generate a rainbow table for whatever hashing scheme you use. Per-user salts break this attack, and also mean that (under normal situations) even if two users choose the same password, you can't tell that they did just by looking at the database; their hashes are different. All the good password hashing schemes take a salt. Salts should be randomly generated and are usually at least 16 bytes (128 bits) long.

At this point, I really have to say: you should use an authentication system written by somebody who is familiar with such systems, and who has a well-tested version ready to use. Don't try to write such security-critical features until you understand the area better.


Good (preferably free) resource for CISSP practice questions

Though not practice exams, I found the mind maps offered @ mindcert.com to be a great set of notes to use in keeping things together. They have published maps of 7 domain areas for the CISSP and a few for the CEH. Additionally, you can always make your own.

http://www.mindcert.com/resources/

Hope this helps, and good luck to you!


Encrypt secret with numeric PIN, but allow wrong attempts

Technically, this is how unauthenticated encryption works by default - an incorrect key will produce an incorrect plaintext. Only if there's authentication or error-checking (and in ready to use libraries, there generally is) will decryption produce an error.

Note that it's trivial to brute-force the PIN due to its tiny keyspace, unless the number of attacks is very limited and they can't be distributed between many machines. The PIN can be made perishable to that end, removing the file after N attempts, provided there's a more permanent way to authenticate.

Generally, you should NOT encrypt the secret with the PIN itself, but rather with a key generated using a PBKDF (bcrypt, scrypt, argon2) from user-known secrets such as the password or the PIN and a salt. Depends on what exactly you're trying to do.


The is a variation of the Ken Tompson Hack.  If the platform is modified to lie,  then what tools can be trusted?  No form of reflection can be trusted, because anything can be modified on a compromised iOS device.

Apple is addressing this issue as best as they can by trying to make it difficult to jailbreak an iOS device.  In practice there is no solution to this problem,  because iOS devices can almost always be jailbroken.


How to determine start and end bytes of LUKS header?

LUKS Emergency Erase

To solve your problem, the best way to wipe a LUKS header is to use the luksErase command. To quote the man page:'

luksErase <device>

       Erase all keyslots and make the LUKS container permanently inaccessible.
       You do not need to provide any password for this operation.

       WARNING: This operation is irreversible.

LUKS Header Start Byte

But to answer your question, a LUKS device's header necessarily starts at byte zero.

This is distinct from a LUKS container, which could start at a non-zero offset. In any case, the LUKS container's start byte would be found by searching for the byte sequence 0x4c55 0x4b53 0xbabe, where (encoded as ASCII):

0x4c = 'L'
0x55 = 'U'
0x4b = 'K'
0x53 = 'S'

LUKS Header End Byte

The end byte is trickier, and to get it we must parse the LUKS headers.

First we parse the LUKS binary header to determine the version. Currently, there are only two versions of LUKS. LUKS1 was released in 2014 and LUKS2 was released in 2018. The encoding for both is very different, but the first 8 bytes of the LUKS header are encoded the same:

+-------+--------+-------------+
| Start | Length | Field Name  |
+-------+--------+-------------+
|     0 |      6 | magic       |
|     6 |      2 | version     |
+-------+--------+-------------+

LUKS1

For LUKS1, the easiest way to determine where the LUKS header ends is to determine where the encrypted data (aka the "payload bulk data") begins, since the encrypted data begins immediately after the LUKS header ends.

To get the start offset of the payload, we need to get the value of payload-offset. From "Figure 1: PHDR layout" of LUKS1 On-Disk Format Specification, the payload-offset field starts in the LUKS header at an offset of 104 bytes and is 4 bytes long.

For example, the following hex dump of a LUKS container shows the version is 00 01 (LUKS1) and the payload-offset is 0x1000 (hex) = 4096 (decimal).

root@disp4117:~# hexdump -Cs 6 -n 2 luksVol1
00000006  00 01                                             |..|
00000008
root@disp4117:~# hexdump -Cs 104 -n 4 luksVol1
00000068  00 00 10 00                                       |....|
0000006c
root@disp4117:~#

The payload-offset just defines the number of sectors (in 512 bytes sectors) after the LUKS header start where the encrypted data begins (note that the location where the encrypted data begins is exactly the same as the location where the LUKS header ends), so in order to get the bytes where the payload begins, we have to multiply payload-offset by 512. 4096 * 512 = 2097152

Therefore, for the example above, the LUKS1 header ends at byte 2097152.

LUKS2

For LUKS2, there is no payload-offset field in the binary header. Rather than iterating through all of the keyslot objects, the closet equivalent to the payload-offset is the offset attribute for the first segment object defined in the JSON metadata (this is new to LUKS2).

But before we can parse the JSON object, we have to find it! In LUKS2, there's six distinct header areas:

  \/ primary binary header          alignment padding \/
+----+----------+----+----------+-------------------+----+
| /\ | 1st JSON | \/ | 2nd JSON |   Keyslots area   | /\ |
+----+----------+----+----------+-------------------+----+
                  /\ secondary binary header

As you can see above, LUKS2 actually has two copies of the plaintext metadata. It's stored in the "primary binary header" and the "1st JSON" area (the second copy is called the "secondary binary header" and "2nd JSON" area, respectfully). In most cases the data in the "primary binary header" exactly matches the "secondary binary header" and the data in the "1st JSON" area exactly matches the "2nd JSON" area. The data is stored twice to aid in recovery and protect against corruption.

The "1st JSON" area always starts at byte 4096 (immediately after the "primary binary header" area), but its length is variable. The length can be determined by looking at the binary header's hdr_size. From "Figure 2: LUKS2 binary header on-disk structure" in LUKS2 On-Disk Format Specification, hdr_size is 8 bytes long and starts at byte 8 in the binary header.

For example, the following hex dump of a LUKS container shows the version is 00 02 (LUKS2) and the hdr_size is 0x4000 (hex) = 16384 (decimal).

root@disp4117:~# hexdump -Cs 6 -n 2 luksVol2
00000006  00 02                                             |..|
00000008
root@disp4117:~# hexdump -Cs 8 -n 8 luksVol2
00000008  00 00 00 00 00 00 40 00                           |......@.|
00000010
root@disp4117:~#

From the above hex dump, we clearly see the hdr_size field has a value of 0x4000 (hex) = 16384 (decimal). But the hdr_size field defines the size of the the "primary binary header" plus the "1st JSON" area. So the length of the "1st JSON" area is the hdr_size value minus the "primary binary header" length. In this case, that's 16384 - 4096 = 122881. Depending on the length of the LUKS2 container's metadata stored in the JSON object, the JSON area could be up to 4194304 - 4096 = 4190208 bytes long.

So here's a hexdump of the "1st JSON" area, starting at byte offset 4096 with length 122881:

root@disp4117:~# hexdump -Cs 4096 -n 12288 luksVol2
00001000  7b 22 6b 65 79 73 6c 6f  74 73 22 3a 7b 22 30 22  |{"keyslots":{"0"|
00001010  3a 7b 22 74 79 70 65 22  3a 22 6c 75 6b 73 32 22  |:{"type":"luks2"|
00001020  2c 22 6b 65 79 5f 73 69  7a 65 22 3a 36 34 2c 22  |,"key_size":64,"|
00001030  61 66 22 3a 7b 22 74 79  70 65 22 3a 22 6c 75 6b  |af":{"type":"luk|
00001040  73 31 22 2c 22 73 74 72  69 70 65 73 22 3a 34 30  |s1","stripes":40|
00001050  30 30 2c 22 68 61 73 68  22 3a 22 73 68 61 32 35  |00,"hash":"sha25|
00001060  36 22 7d 2c 22 61 72 65  61 22 3a 7b 22 74 79 70  |6"},"area":{"typ|
00001070  65 22 3a 22 72 61 77 22  2c 22 6f 66 66 73 65 74  |e":"raw","offset|
00001080  22 3a 22 33 32 37 36 38  22 2c 22 73 69 7a 65 22  |":"32768","size"|
00001090  3a 22 32 35 38 30 34 38  22 2c 22 65 6e 63 72 79  |:"258048","encry|
000010a0  70 74 69 6f 6e 22 3a 22  61 65 73 2d 78 74 73 2d  |ption":"aes-xts-|
000010b0  70 6c 61 69 6e 36 34 22  2c 22 6b 65 79 5f 73 69  |plain64","key_si|
000010c0  7a 65 22 3a 36 34 7d 2c  22 6b 64 66 22 3a 7b 22  |ze":64},"kdf":{"|
000010d0  74 79 70 65 22 3a 22 61  72 67 6f 6e 32 69 22 2c  |type":"argon2i",|
000010e0  22 74 69 6d 65 22 3a 34  2c 22 6d 65 6d 6f 72 79  |"time":4,"memory|
000010f0  22 3a 32 37 34 35 33 30  2c 22 63 70 75 73 22 3a  |":274530,"cpus":|
00001100  32 2c 22 73 61 6c 74 22  3a 22 71 4a 6e 79 2b 4a  |2,"salt":"qJny+J|
00001110  5c 2f 6f 35 71 77 57 77  35 78 2b 57 31 30 7a 47  |\/o5qwWw5x+W10zG|
00001120  59 54 6f 64 44 64 57 6f  39 6e 74 5c 2f 6c 67 49  |YTodDdWo9nt\/lgI|
00001130  41 61 61 6f 78 5c 2f 45  3d 22 7d 7d 7d 2c 22 74  |Aaaox\/E="}}},"t|
00001140  6f 6b 65 6e 73 22 3a 7b  7d 2c 22 73 65 67 6d 65  |okens":{},"segme|
00001150  6e 74 73 22 3a 7b 22 30  22 3a 7b 22 74 79 70 65  |nts":{"0":{"type|
00001160  22 3a 22 63 72 79 70 74  22 2c 22 6f 66 66 73 65  |":"crypt","offse|
00001170  74 22 3a 22 31 36 37 37  37 32 31 36 22 2c 22 69  |t":"16777216","i|
00001180  76 5f 74 77 65 61 6b 22  3a 22 30 22 2c 22 73 69  |v_tweak":"0","si|
00001190  7a 65 22 3a 22 64 79 6e  61 6d 69 63 22 2c 22 65  |ze":"dynamic","e|
000011a0  6e 63 72 79 70 74 69 6f  6e 22 3a 22 61 65 73 2d  |ncryption":"aes-|
000011b0  78 74 73 2d 70 6c 61 69  6e 36 34 22 2c 22 73 65  |xts-plain64","se|
000011c0  63 74 6f 72 5f 73 69 7a  65 22 3a 35 31 32 7d 7d  |ctor_size":512}}|
000011d0  2c 22 64 69 67 65 73 74  73 22 3a 7b 22 30 22 3a  |,"digests":{"0":|
000011e0  7b 22 74 79 70 65 22 3a  22 70 62 6b 64 66 32 22  |{"type":"pbkdf2"|
000011f0  2c 22 6b 65 79 73 6c 6f  74 73 22 3a 5b 22 30 22  |,"keyslots":["0"|
00001200  5d 2c 22 73 65 67 6d 65  6e 74 73 22 3a 5b 22 30  |],"segments":["0|
00001210  22 5d 2c 22 68 61 73 68  22 3a 22 73 68 61 32 35  |"],"hash":"sha25|
00001220  36 22 2c 22 69 74 65 72  61 74 69 6f 6e 73 22 3a  |6","iterations":|
00001230  36 31 39 34 33 2c 22 73  61 6c 74 22 3a 22 46 69  |61943,"salt":"Fi|
00001240  4c 67 31 35 56 5c 2f 55  56 4b 47 72 72 4e 39 4f  |Lg15V\/UVKGrrN9O|
00001250  52 2b 5c 2f 69 59 46 51  70 38 38 59 44 77 50 4c  |R+\/iYFQp88YDwPL|
00001260  6a 4f 6f 4c 70 6a 77 6d  78 58 77 3d 22 2c 22 64  |jOoLpjwmxXw=","d|
00001270  69 67 65 73 74 22 3a 22  49 70 34 31 5a 58 70 44  |igest":"Ip41ZXpD|
00001280  76 77 52 76 6d 41 73 33  30 58 69 72 6c 48 65 6d  |vwRvmAs30XirlHem|
00001290  57 72 44 67 6c 5c 2f 44  4a 31 36 79 33 31 41 71  |WrDgl\/DJ16y31Aq|
000012a0  66 42 55 6f 3d 22 7d 7d  2c 22 63 6f 6e 66 69 67  |fBUo="}},"config|
000012b0  22 3a 7b 22 6a 73 6f 6e  5f 73 69 7a 65 22 3a 22  |":{"json_size":"|
000012c0  31 32 32 38 38 22 2c 22  6b 65 79 73 6c 6f 74 73  |12288","keyslots|
000012d0  5f 73 69 7a 65 22 3a 22  31 36 37 34 34 34 34 38  |_size":"16744448|
000012e0  22 7d 7d 00 00 00 00 00  00 00 00 00 00 00 00 00  |"}}.............|
000012f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00004000
root@disp4117:~#

To get the offset of the first data segment we need to check the segments object in the above JSON. Here's the same segments section from above, but formatted for readability:

  "segments": {
    "0": {
      "type": "crypt",
      "offset": "16777216",
      "iv_tweak": "0",
      "size": "dynamic",
      "encryption": "aes-xts-plain64",
      "sector_size": 512
    }
  },

We can see above that the first data segment begins at offset = 16777216.

Therefore, for the example above, the LUKS2 header ends at byte 16777216.


What is an intermediate certificate authority?

Any CA can be an "intermediate CA". Because "being intermediate" is defined by how the verifier sees it.

When you validate a certificate, you verify the signature which has been generated over that certificate by the CA which emitted the certificate. This signature is verified against the CA public key. If you know the CA public key "inherently" (e.g. it is one of the CA public keys distributed with the operating system), then the CA is a trust anchor, also known as root CA. On the other hand, if you know the CA public key only through validation of a CA certificate (a certificate issued to that CA by another CA), then the CA is deemed "intermediate". One can see the name "intermediate" as describing where the CA is in the chain of trust: the trust anchor is at the beginning, the end-entity is at the end (duh!), and anything in-between is "intermediate".

I can issue certificates for any CA, even without their consent or even awareness, by simply taking the certificate contents and resigning them with a key of my own. This means that every single CA on this planet is, potentially, an intermediate CA.


What happens to PKI once quantum computers can break encryption?

Quantum computing is being developed to attack specific algorithms. Right now there are efforts under way to find algorithms that are resistant to quantum computing attacks. Such algorithms are called post-quantum.

The Public Key Infrastructure is built on the concept of certificates, not algorithms. Certificates are documents that are capable of storing many types of signatures using many different algorithms. If one algorithm falls, such as happened when MD5 was found to have weaknesses allowing the forging of certificates, we can stop using it and switch to another.

Researchers give us warnings as they close in on weaknesses. An example of this happened a couple of years ago when SHA-1  fell to an attack. Even though there was no immediately apparent vulnerability to certificates, people stopped trusting SHA-1 signatures on certificates, and switched to alternatives.

As quantum computing approaches the capability of cracking a specific algorithm such as RSA, the world will switch certificates to trusting only post-quantum algorithms. PKIs will continue.

Throughout history, cryptography has always been a game of cat and mouse. The ability to modify keys, algorithms, and protocols has always been critical to maintaining security. Quantum computing may be a threat, but not necessarily a game ender.


Estimate the number of host computers on the private network by port scanning

From outside the a NAT network no direct port scanning of the internal network is possible. This is because packets from outside will only be forwarded to the internal network if there is some connection established from inside which matches the packet. See also Scan a computer behind NAT.
Scanning from inside is mostly trivial though: just enumerate all possible hosts in the local network and try the reaction to common ports. If there is some kind of response (i.e. SYN+ACK, RST, ...) then there is likely computer with this IP address.
And while direct scanning from outside is not possible, an outside attacker might actually be able to scan from inside. All what is needed for this is to lure somebody inside the internal network to a site controlled the attacker and then use the browser as the vehicle to scan the internal network. This is possible because the browser on an internal system has at the same time access to the internet and also internal network. See for example Exposing Intranets with reliable Browser-based Port scanning for more details.
Apart from active discovery there are also ways to passively detect the number of (active) devices behind a NAT. See for example Detecting NAT Devices using sFlow and other research.


How can I download an exposed mercurial directory?

I have made a tool that performs some of the following steps:

First, download the changelog and manifest files as described in the repo structure documentation.
Run hg --debug manifest to get a file list. Download the index for all these files.
Fix the repo using the convert extension
Run hg update -C to restore all files.

Now you have the source code.

Edit: more information in my blog post about this.


In AWS EKS, how should Kubernetes certificates be generated?

With Kubernetes in general, the certificates aren't actually "self-signed" so much as they're signed by an internally managed certificate authority.

Typically, one of the main points of signing a certificate is to ensure that the clients of that service can ensure the certificates validity. In the Kubernetes case this is handled by ensuring that the client has the CA cert on setup. So for example a kubelet on a cluster node will be given the CA cert file as part of setup so it can validate the certificate of an API server that it connects to.

For end users this certificate authority data is generally passed out as part of the kubeconfig file used to log in to the cluster.

So really complaints about "self-signed" certificates don't apply, either in general Kubernetes or EKS.  There are other problems with how Kubernetes manages certificates (e.g. lack of revocation support) but those aren't specifically related to this issue.


In PubKey Encryption, can a secret encrypted by a public key be validated by same public key?

Integrity is what you are looking for. With integrity, the receiver can check that the message is valid. One can use Message Authentication Codes like EAX and GCM. Take care of the order of operations if you insist on this. Also, take care of proper padding if you are using RSA. For encryption PKCS#1.5 and RSA-OAEP and for signature RSA-PSS.

Normally, we don't send messages with the public key cryptography. We use hybrid cryptosystem in which the public key cryptography is used for key exchange as in Diffie-Hellman Key Exchange, DHKE, in fact, the Elliptic Curve version is better, ECDHKE.

Once a key is exchanged, the parties can send messages with AES-GCM. The GCM mode is an Authenticated Encryption (AE) mode that gives you confidentiality, integrity, and authentication. You can also use chacha20-poly1305 AE.


How to mitigate credential disclosure in man in the middle attack

The Secure Remote Password (SRP) protocol can be used to send credentials over an unencrypted network.

You could also require SSL/TLS client authentication which will prevent the proxy from being able to establish a connection to the web server.


How do I find the source of network traffic from a specific host?

@schroeder and @Brett Littrell provided the most useful suggestion. They suggested that I run a packet capture on the host machine.

I ran the capture using WireShark and learned the names that were being queried.
The names of the domains suggested that the packets were being forwarded from the DNS server running on the host.
I repeated my inspection of the Network Settings both in the Network Settings panel and the Server settings for the DNS and found that I had made a mistake and overlooked the domain 8.8.4.4 in the permitted forwarding addresses.
Removing the entry in the Server App stopped the unwanted traffic.

Knowing now that the traffic is from a legitimate program performing proper operations, I can safely change the firewall settings to allow more frequent traffic from the affected host to the DNS forwarding servers without worrying about the effect on the safety of the system.


At the start of a TLS/SSL handshake, the server will send its certificate, containing its public key, to any client who initiates a connection. You cannot keep the public key secret.

Instead, consider using TLS/SSL client authentication. Like the server, the client will also possess a certificate, public key and private key. Both the client and the server will verify each other's certificates during the handshake. You can ensure that only authorised clients possess client certificates.

Alternatively, use any application layer authentication protocol you wish.