Diffie Hellman - how large should the random number be?

Short answer. Yes, I think it is fine to choose x as a random 160-bit number.  (I would not bet my life on it, but I'm pretty sure it is fine.)

Technical details. There are (at least) two ways to attack Diffie-Hellman.

One way is to use fancy algorithms for computing the discrete logarithm, e.g., based upon the number field sieve or the like.  The running time of these algorithms is dependent upon the prime modulus p, and you need p to be large (certainly at least 1024 bits) to resist these attacks.
Another way is to use more basic algorithms, such as the baby-step giant-step algorithm or Pollard's algorithm.  The running time of these algorithms is dependent upon the smaller of the size of q or the size of the exponent x.  You need q to be large (at least 160 bits) to resist these attacks.  You need the exponent to be chosen from a large space (i.e., at least 160 bits of entropy) to resist these attacks.

Since there are two different attacks, you'll need to choose your parameters to defeat both.  That's why you need your exponent to be at least 160 bits.

Now the most conservative thing to do is to choose x uniformly at random in the interval [0,q-1] (or [2,q-2] if you prefer, it doesn't really matter).  This will surely be secure, as long as p and q are big enough.  The quote you pulled out seems to be saying that X9.42 recommends you use this conservative approach.

However in many cases you can get away with an optimization, sometimes known as the short-exponent optimization.  The idea is you choose x uniformly at random from some smaller interval, say [0, 2160 - 1].  This is usually secure, if the protocol is designed right, though there have been some exceptions.  The benefit is that if q is large, choosing a shorter x like this can significantly improve performance.

For the group you mention, q = (p-1)/2. (*)  In other words, q is around 1023 bits long.  This means the most conservative thing to do is to choose your x to be from the full range [0,q-1], but I believe you can probably get away with using the short-exponent optimization.  I tried to read the IKE documents, and I didn't immediately find an explicit statement of whether the short-exponent optimization is always guaranteed to be safe for IKE, though I did find some passing remarks which seem to indicate that the developers did anticipate it might be OK to use.

Footnote (*): You can infer this as follows. RFC 2409 says that the "properties of these groups are described in [RFC2412]". Section E.2 of RFC 2412 lists the corresponding group, and lists the largest prime factor of the group order.  The largest prime factor can be verified to be (p-1)/2.  From this, it follows that q = (p-1)/2.

Edit 3/24: I just noticed, thanks to @CodeinChaos's answer, that you mentioned your reason for using a short exponent is not performance but rather the limited entropy in your PRNG.  I find this a bit odd.  What PRNG are you using?  Are you sure you have a crypto-strength PRNG?   Most cryptographic PRNGs can be used to pull an arbitrary-length pseudorandom string.  If yours doesn't, then you can always take the output of your PRNG and expand it to a 1024-bit pseudorandom string: e.g., generate a 128-bit AES key K, then use AES(K,0), AES(K,1), AES(K,2), AES(K,3), ..., AES(K,7) to generate a 1024-bit pseudorandom string.  Once you have a 1024-bit pseudorandom string (as long as it is crypto-quality and the original RNG has enough entropy in it), you can use that as a full-length 1024-bit exponent.

Do check that your RNG/PRNG is crypto-strength.  For example, /dev/urandom and CryptGenRandom are good, but rand(), random(), drand48() are all bad.  (/dev/random is strong but unnecessary; use /dev/urandom instead, and you will be able to draw a full-length exponent with no problems.)  Search on this site for more information about crypto-quality PRNGs.


How do I verify that an ssh public/private keypair go together, is it different for openssl and ssh-key generated keys?

You need to add -pubin to your openssl rsa invocation to work with the public key:

$ openssl rsa -pubin -in ./oci_api_key_public.pem -noout -modulus


There are two main points to your situations.

1. Does the lock should know how to unlock itself ?

There are basically two kinds of "signature" systems, and only one of them is really a signature. In your situation, the server will send to the device some sort of token which the device will verify. So the question is: is it a problem if the device would have the technical power of producing a valid token ? Or should things be such that the device contains nothing which would allow it to make such new tokens ?

In the former case, you can work with a MAC, and HMAC/SHA-256 is not a bad choice for that. However, this means that the device and the server share some secret key, which is used to both produce and verify unlock tokens. If there are several devices, then each should have its own device-specific key, and the server should then know all device keys -- otherwise, ripping apart one device would allow an attacker to learn enough to unlock all other devices.

In the latter case, the server has an asymmetric key pair; unlock tokens are signed with the server's private key, and devices verify tokens with the server's public key. RSA is a popular digital signature algorithm; the verification part is inexpensive, and thus can be performed by most devices, even power-limited devices.

2. What about replay attacks ?

A "replay attack" is, in your case, when an attacker observes an unlock token, and sends it again at some later point. There are several ways to avoid such issues, but they all imply some sort of extra capability on the device:

Time-limited tokens. The token includes an issuance date and the device won't accept a token which is "too old". This method requires the device to have a reasonably accurate embedded clock.
Duplicate tracking. The device remembers previously seen unlock tokens, and refuses to act upon a token which it has already seen. This requires some sort of memory which cannot be reset.
Nonces. The device first emits a random value, and expects the unlock token to contain a copy of that specific value. This makes the protocol more complex and requires the device to be able to emit data as well as receiving tokens. This will not map well to all situations; for instance, if the device is a door lock, with a keypad, and the user contacts the server through his smartphone. Typically, the device would need a small LCD screen to display its nonce.

You may want to consider HOTP: this is a MAC-based protocol, thus with a key shared between "server" and "device". Both server and device remember only a short counter value (say, 32 bits) and this is sufficient to protect against replay attacks. The communication is one-way (the device needs not emit anything). This protocol (or any similar variant) is what is used to unlock your car: the car is the "device" and the remote control attached to the key ring is the "server".

All of the above is about the token sent from server to device. Of course, the server should not produce the token unless it has good assurance that it is token to the right user (the device owner). You thus need authentication which, in practice, will mean SSL. Let the user open a SSL connection with the server (say HTTPS) and authenticate with the server. The SSL layer will take care of integrity and encryption so you do not have to encrypt the token itself. Actually, encryption would not be needed for an unlock token at all -- you may want to use encryption only to deter eavesdroppers, and you want to thwart eavesdroppers only if your token-verification protocol is weak against replay attacks. In any case, encryption is best left to a layer where all the tricky details have been smoothed out through years of teeth cringing toil, and that's SSL.


Does having multiple email addresses increase security?

Isolating resources is definitely a valid and widely used method of controlling security threats.

I can think of many scenarios where these benefits of isolation would apply to email addresses, including those you mentioned.

  Are there any down sides to doing this?

I don't think there's any real downsides, but I can think of a number of reasons it would be difficult for most people to derive the full security benefit of isolating their email addresses.

Firstly, you need to minimise dependencies between accounts as these weaken the isolation. For example, you'd have to avoid:

Using one account as a recovery email for another
Sharing passwords between accounts
Using the same domain across accounts (especially if it's your own domain)
Sharing an actual inbox (ie. destination for all your mail)

This creates many inconveniences:

It may take a lot of effort to setup many mail accounts, especially on email services you don't control (eg. gmail)
Having to check many inboxes
Having to maintain many sets of credentials
Having to constantly make decisions about which address to use in order to minimise risk

One common technique for people who own a domain is to have a catch-all rule that goes to a single inbox. That way they can invent an arbitrary email address every time they need one (eg. stackexchange@whatever.com) and if they get spam from a particular service they can easily block the unique address they used. This will help with the spam scenario you mentioned, but not if your account credentials are compromised seeing as all addresses will share an inbox and password.


Yes, of course. Service and OS fingerprinting is one of the first steps in the methodology.

Nmap does service fingerprinting and OS detection. But that requires that the service be available to the network (web server, SMB, etc.). You can even connect to a network port with telnet and request the service banner.

If you want to know an installed program that is not network-accessible, then you need to do meta-data analysis of files that the program created, or you need to perform social engineering or open source searches to get the version info. For instance, has an employee of the company posted on a tech support forum and revealed the version they use?

I see that the original version of the question asked for a specific service. For that, you can try nmap to see if that service will return a banner, or use the non-network accessible option I laid out above.


Router Question -- Port Forwarding Rules Added Automatically

If UPnP (Universal Plug 'n' Play) is enabled on the router, these port forwarding rules may have been added in response to UPnP requests from the devices.  UPnP is designed to allow devices to make these kind of reconfiguration requests of NAT devices.  Particularly services like video games and peer-to-peer file sharing applications use this technology to allow incoming connections from other users.


Is a CSRF required if no direct state change operations are performed?

CSRF for Proxies
It sounds like your service is acting more or less as a proxy for other services.  Looking at things from that perspective I would say that the preference is to not perform CSRF protection (which is a stronger statement than suggesting that CSRF protection may not be necessary).
The reason is that if you are not doing anything yourself but transparently forwarding requests for someone else, then attempting to provide a security measure where none is expected/required will cause headaches for the person you are providing this proxy service for.  In other words, you've increased cost/usability without necessarily increasing security, which is a bad trade off.
In short, unilaterally dropping in security checks everywhere isn't often a good idea.  Unless you know for sure which requests require a CSRF check and which requests don't, you're probably better off letting the user's who operate the actual endpoints decide for themselves if CSRF is required.  Of course if you know for sure when CSRF should or should not be required, then it may not hurt to put it in as an additional safeguard, just in case someone on the other end forgets it.
Presuming you opt to skip out on CSRF protection on your end, it may be worth noting this fact in documentation somewhere. After all, it may not occur to the people using the service that they can and should implement the usual security measures when needed by their own system.


Questions to confirm a good standpoint on web app security?

I'm sorry to revive an old question, but Jacob Kaplan-Moss - co-creator of Django - recently wrote a great article about this exact subject: Our Dumb Security Questionnaire. It consists of the following ten questions:
(1) Please answer these questions:

If you use a cloud provider (GCP/AWS/Azure/etc):

describe how credentials are provisioned, managed, and stored.
If an attacker gained access to an individual developer’s cloud credentials, what actions could that attacker perform, and how would you detect and respond to the breach?

If you don’t use a cloud provider: why not?

Describe how staff authenticate to company services (e.g. servers, email, SaaS products), particularly highlighting your use of password managers, 2FA, and SSO.

What development practices do you use to protect against the OWASP Top 10?

Describe the steps a developer or operations person takes to push new code to production.

Have you had any security breaches in the last two years?

If yes: please explain the breach, and provide copies of any postmortem/root cause analysis/after-action reports.

(2) And, please provide as many of the following as possible:

A recent (last 12 months) penetration test report, with information on what follow-ups/remediation steps were taken. Reports by an external firm are preferable; internal tests are OK. A redacted summary is fine as long as it includes basic vulnerability descriptions and severity levels.

Documentation on your organization’s Secure Development Lifecycle, or similar, if you have them. If not, please summarize the steps you take to help engineers write secure code.

A copy of any security training material provided to staff, particularly software developers. A summary/description of classes and content is fine.

A link to your Vulnerability Disclosure Policy and/or bug bounty program. If you don’t have either: please explain.


How likely is an attack for a known OTA ring 0 vulnerability in mobile devices?

The answer must vary dramatically based on many factors which are specific to the individual exploit.  For instance, consider how easy it is to actually cause the exploit.  The harder it is to write a worm which leverages that exploit, the longer it will take.  As a trivial lower bound, consider if you published openly that your root password was "password" and was accessible by SSH.  How long would it take for someone to take advantage of that?  Compare that with Heartbleed, which in theory can cause unspecified damage by revealing random blocks of memory to the attacker (including revealing passwords that could arguably lead to ring 0 compromise in particular circumstances).  It was considered to be a big deal because it was a substantial bug in openSSL, by in practice it was deemed less risky than other exploits because it required more effort on the part of the attacker to hit pay dirt.


Help with creating Heartbleed vulnerable server

Unless you are using a version of 12.04 which was unpatched for years the OpenSSL on your test server is not vulnerable even if the OpenSSL version number shows a vulnerable version. Vendors backport security fixes in the OpenSSL versions they ship and thus the version number itself does not say a lot. See Why is this version of OpenSSL (1.0.1e) not vulnerable to Heartbleed?
.

Thus if you would like to setup a vulnerable system either get old and unpatched images or get the vulnerable version from openssl.org and compile it yourself and also make sure that all applications you want to test are linked with this vulnerable version.


They are defined in the standard, maintained on the OASIS website...symbols and abbreviations.

And the list there matches yours.


Is it possible to send cookies from third party server to browser using CORS

The domain attribute of for a cookie cannot be set to arbitrary domains. While foo.example.com could set a cookie to example.com it could not set a cookie to example.org since this is an unrelated domain. From MDN:Set-Cookie: Invalid Domains:

  A cookie belonging to a domain that does not include the origin server should be rejected by the user agent.

As for how to achieve what you want: it is not clear what you want to achieve in the first place. Cookies for cloudfront.com should only be given by  cloudfront.com server. Maybe you are trying to do something with cookies were cookies where never designed for.


How to share "secure" account credential with my spouse?

Share name and password as usual
Secret questions are stupid, just treat them as another password (put randomly generated text there and shere it)
rotating passwords: If you don't mind the reduction in security, just share the rules the same way you share passwords. If you do, there is not an easy way to do this.

2FA

TOTP, you should add it to both of your phones when registering. If that is not feasable, consider using a TOTP app that allows backups and share the backups. You could also share the QR code/secret in text form, but that reduces security somewhat.
U2F unless the site allows two tokens for your account, than this may be impossible to share. Maybe there is a token with a backup feature, I don't know.
SMS: This one is also a bit tricky. You may want to set up some sort of SMS cloud backup and give your spouse access to that, or make it possible for her to get a new SIM for your number.


Can I use transport mode in IPsec if I tunnel via UDP?

In light of your edit:

An IPSec system supporting RFC3948 will do what you want, yes, assuming you do not need AH and that you're using either IKE or IKEv2.

However, the whole system has been designed for NAT traversal so I'm not certain it exactly fits your requirement (you didn't specify WHY you need to limit yourself to UDP or why you want to use IPsec).


How completely wipe information of overwritten files and folder?

You have to stop thinking about this on the file level. For a storage device, all that matters is the sector. If one sector on a hard drive* is overwritten, the data in it is gone for good. There is no known way to retrieve it even with "powerful software", and there is no need to overwrite the same sector multiple times. Modern hard drives encode data in such a dense and complex format that a single overwrite will invariably make that data irretrievable (we can't even recover data from an old fashioned low-density audio cassette tape!). However, whether or not filling up a bunch of free space on a hard drive will actually overwrite the sensitive sectors is another matter. Due to features such as damaged sector relocation, and due to the behavior of the specific filesystem, it is not possible to guarantee that free space has been overwritten without overwriting the entire drive.
Filesystems are incredibly complex, and they are far more than flat databases of files. Cutting and pasting a file for example does not do anything but move it, which involves changing only a few bytes in filesystem metadata, regardless of how large the file is. So what about deleting a file and then filling the free space with dummy data until the drive is full? That might work, but it might not. Many filesystems contain redundant copies of information. The filesystem ext4 for example can keep copies of small files in its journal, which does not get overwritten when you wipe free space.
The exact way of storing new files is also more complex than what you have mentioned. When you delete a file, you are deleting a reference to the file (as you have surmised). However, creating a new file does not guarantee that you will be overwriting the sectors that made up the previous file. A new file will likely be strategically placed at an address on the hard drive that minimizes access latency, or which decreases fragmentation. The file may not even be deleted, but simply hidden, in order to make undeletion and incremental backups ("snapshots") possible. A new file will not simply be stuffed into the newly unallocated space in all but the very simplest of filesystems.
* When I say hard drive, I mean a real spinning rust. Solid state and hybrid drives work differently such that, even if you overwrite the same sector twice, the physical location that the data is saved to may be different each time.

If you actually need to remove a file such that no one can recover it, you will not have many options at your disposal that preserve any other existing data on the drive. But you aren't out of luck:

If you have encrypted the file or drive, you can simply throw away the encryption key.

You can erase the entire drive, for example by using ATA Secure Erase.

You can destroy the drive from the outside using an expensive degaussing machine.

Deleting the file and then filling up free space, or shredding the file using data erasure tools (which typically get a list of the sectors that the file occupies, and then overwrites those specific sectors) will generally destroy the majority of the file, but comes with a high risk of incomplete erasure, with both the file's metadata and potentially small portions of the file remaining elsewhere on the drive.


is it possible to scan an umounted pendrive partition with clamav on linux?

Well you can... provided you mimic the mount of the partition!

At a low level, mounting a partition is no more that reading the raw partition to discover and load its different system tables (allocation table, root folder, etc.). From that point, the system is able to give you access to the individual files.

If for any reason you do not want the kernel to mount the partition, your application must have enough knowledge on the internals of its file system to be able to:

find and read the root folder (and any other folder once you can find it)
identify all active (plain) files and folders inside a folder
load a file from a folder

As you just want to pass clamav there, you can safely ignore all special files (devices, symlinks, unix domain sockets) and only process plain files and folders. Also, you can ignore any file attributes.

Then, you recursively walk the folders, read individually all files and pass their content to clamav

But the question is now: what is the security gain/loss regarding a normal mount of the partition?

Pros:

no risk of accidentally executing anything from the partition
no risk of hitting a security flaw in the file system driver(*) (note that you can mediate that with a FUSE driver if there is one, or by running the analysis on a virtual machine)
no risk that a third party process can access any file from the partition

Cons:

you must implement securely all the low level structures and accesses of a file system
you must trust more your code that the (highly tested) kernel and driver code
you must ensure that your code will follow all possible evolutions of the file system
you must implement all file systems that you want to be able to clamax

That being said, that is exactly what the fsck utility does: it reads a raw partition and walk along its hierarchy to discover and fix low level errors. But it is a low level system tool...

TL/DR: unless you are a file system expert, I am not sure that the security gain is worth it. IMHO the following procedure should be secure enough:

fsck the partition - if it is supported by fsck
mount it to a subdirectory of a directory with permissions 700 to make sure that no non root process can access it, with the noexec, nosymfollow, noatime flags
pass clamav on it

(*) mount man page ends with

  BUGS
       It is possible for a corrupted file system to cause a crash.


What are some vulnerabilities of environment variables (on any platform)?

There are the usual risks associated with explicitly or implicitly trusting something which comes from outside and thus can be controlled by the attacker. And there is a risk of assuming that environment variables provide a restricted visibility which they don't do always. Some examples:

Implicitly trusting environment variables like PATH, LD_LIBRARY_PATH, PERL5LIB or similar can lead to unwanted execution of code because these variables decide where to look for programs and libraries. Similar variables like IFS decide how to interpret information given to the program. And sometimes the problem can be just any variable with a malicious content, like with the Shellshock vulnerability. That's why programs which use elevated privileges (like sudo) should remove or sanitize all environment variables when run by a non-privileged user. And programs which set environment variables (like webserver with CGI interface) should be very careful which possible attacker controlled information they put into these variables.
You should not use environment variables to pass sensitive information (like passwords) from a parent to the child processes. Depending on the OS other users on the system might see the content of the environment variables and thus grab these sensitive information.

Thus the rules when using environment variables are:

Never trust the content of these variables, neither explicit nor implicit. That means remove or sanitize the variables especially when the variables might be set by users with a different privilege then the program is running.
Never put sensitive information in environment variables because other users might read these.
If you set these variables with data which you did not fully generate yourself (i.e. possible attacker controlled) be very careful and sanitize everything to disarm it.


Use RSA keypair for PGP encryption and decryption

What PGP does is described in the OpenPGP standard. In OpenPGP, "key rings" are some terminology for "public and/or private keys encoded in the format described by OpenPGP".

Among the types of keys for which OpenPGP describes a key ring format are RSA keys. There is no fundamental issue which would prevent an OpenPGP implementation from using a RSA private key which is stored in a HSM. Practically speaking, though, the free OpenPGP implementation, called GnuPG, does not inherently supports that -- but the gnupg-pkcs11 project may help: it allows integration of a cryptographic device which offers a PKCS#11 driver (virtually all HSM do that) into the GnuPG world.

Importing an existing "PGP key" (i.e. a RSA private key currently stored as a file in the OpenPGP format) into a HSM is not necessarily a good idea: a HSM is an expensive device which aims at never allowing a private key to ever exist outside of the tamper-resistant hardware of the HSM. If the key is imported, then it does/did exist somewhere in the outer world, which makes the use of the HSM much less justified.


In order to use a different protocol, your program must be able to open a socket of that type.

Typically, only TCP and UDP sockets can be opened by non-system programs.

When you try to open e.g. an ICMP socket as non-root, that will simply fail. Also, even if you could, chances are the network stack will just not know what to do with your packets and drop them on incoming.

Nor will routers even route these packets to you.

So, there's no attack or exfiltration surface here, because someone able to open a non-TCP or UDP socket can typically also reconfigure the firewall running on the same computer.

Non-TCP or UDP packets are typically not forwarded by routers, so that's kind of an inherent "drop all" firewall. ICMP is an exception, but that being pretty well-covered by firewalls, too.

There's a few speciality protocols (SCTP comes to mind) that are in use but neither TCP nor UDP, but honestly, when you're configuring your computer to allow a non-superuser to open such sockets, you might as well add specific rules to your firewall to allow these packets through: you're likely operating a backbone network.


SSL and TLS provide transport security, but after DNS resolving. So if the DNS cache has been poisoned the connection will go to the wrong server.

But after the connection has been made and the TTL of the DNS is ie. a day and then the dns poisoning takes place this has no effect anymore, because the client will do no dns resolving (because he already knows where to connect to...).


C++ : memset on a struct containing std::wstring - Is it a risk?

I believe the issue lies with the fact that std::wstring is not necessarily a simple structure in the traditional C sense.

Instead it'll be a class with an internal state, structure and possible a vtable to class functions that are not necessarily visible to you.

So if you memset() the class you may end up destroying internal state and function pointers leading to undefined behaviour if you then try and use the class.  And as we all know undefined behaviour is bad.


The response of the server reply is Brotli encoded:

  Content-Encoding: br

I think that the Burp Suite doesn't decode this, so you're seeing the compressed version of your response.

For more information about Brotli you can look here.

Update:

Until the Burp Suite supports Brotli, you can override the Accept-Encoding-Header as a workaround. Go to Proxy -> Options -> Match and Replace. There is allready a template named "Require non-compressed responses" that you can simply activate:


Can a HSM be used to make password hashing stronger?

You could, instead of hashes, use a MAC. That's a kind of hash function except that it uses a secret key, which is needed to recompute it. You would store the key in the HSM. The HSM would be needed to verify the password; if an attacker steals your database, he still cannot run a dictionary attack. This kind of scheme is sometimes called peppering and a HSM is the right way to use a pepper. To defeat such a system, the attacker would have to hijack the HSM and use it to "try passwords".


Should I use one CSRF token per page or per form?

As this question mentions, a single random unguessable token per session is sufficient.

When you start generating tokens per-page or per-form, you're simply limiting the impact of a CSRF token being leaked. Singular token leakage cases are relatively rare bugs, but XSS is obviously a common vector.

In certain cases it does indeed make sense to have a per-page or per-form CSRF token, as when combined with other protection features (e.g. disabling Ajax with CSP's connect-src directive) you can limit an XSS bug's ability to compromise CSRF tokens to the page which the XSS occurred, assuming you ensure that the token can't be re-used elsewhere.

Some people promote per-page or per-form CSRF protection as a way to increase security when using a double-submit cookie method, but I don't think this is sensible: if you have a bug that lets you read a cookie, you'd just read the session ID.

The flipside of the argument is that by implementing the more granular CSRF protection, you add additional complexity to your code, and that increases the attack surface and potential for bugs.

Personally, I'd stick with a single token per session, then implement other anti-automation controls (e.g. captcha, password re-entry) on top of this when needed for additional security.


Capture WiFi AP authentication attempted password

No.  With WPA2, the client never actually sends the authentication password over the air to the access point.  Instead, the client derives an encryption key from the password, then the access point is able to verify that the client is in possession of the password cryptographically using the key derived from the password.  See https://superuser.com/questions/1068126/router-wifi-password-encryption-types-and-why-do-they-matter for more information.


Is it bad to use special characters in passwords?

Language-specific characters are typically avoided by password generators because they would not be universally available (US keyboards don't have accented characters, for instance). So don't take their omission from these tools as an indication that they might be weak or problematic.

The larger the symbol set (a-z, A-Z, 0-9, etc.) the larger the pool of possible characters to try to guess when bruteforcing a password. Adding language-specific characters adds to the pool, and that can be a good thing.

But be careful about how you calculate entropy. The string ààààààààààà doesn't have a lot of entropy if you are just hitting it on your keyboard because it's convenient. Entropy is about how the characters are chosen. A randomly chosen string has high entropy and a randomly chosen string from a wide pool of characters has higher entropy.


PCI SAQ C 1.2.1, 1.3.5 and remote server/software

Can't comment yet, but will answer as well as I can w/o knowing more info about your setup in regards to the Company in Denver.

That depends on what you mean by remote server.  For the 95%, are you going to a web site and entering the credit card information, or are you using remote access tool (Remote Desktop, Logmein, Teamviewer, etc..) to connect to this machine in Denver?

Will need more information on this question to give an better answer.

PCI Questionnaires are by Merchant.  Additional sites come into play, if all the sites are processing using the same merchant #.  However, generally most PCI compliance scans are per merchant #, which usually is per site.  Wal-Mart might have thousands of stores, but each store/site is a different merchant #, and each store would do it's own questionnaire.  You would want to talk to both IT departments about questions you will have during the questionnaire.  Firewalls, passwords expiration, AV updates, and so on.

However, not knowing all the information about your setup with Denver makes it difficult to give an accurate answer.

The question I can help you the most with as it can't really be any different regardless of how your setup between County and Denver is.

Any computer that processes a credit card over the internet, should be on a closed network.  The only internet send/request that device should make is for processing credit cards.  Exclusions for OS/Software/Antivirus/Firmware Updates/and some others.  Basically, no browsing & no email should happen on those computers that process a credit card.  Even if you have remote access to your computer that processes credit cards, that remote access should be turned off only when needed.  There has to be 0 chance that an employee gets on the wrong website, clicks the wrong email, anything that could allow something to happen.  Any other computers that are on the same network should not be browsing or emailing.

Analogy I give to customers about this is:

You can have a house that you've built a circular fence made of brick encompassing your house.  The brick can't be climbed, however at one point in the brick there is a small hole where you can see through it.  In PCI Compliance view.  You failed.

For hotels/restaurants it can be difficult, but that's what it's supposed to be.  Do all hotels & restaurants follow this procedure.  No.  They're supposed to.  Generally all the credit card locations my company manages (over 100), have an office computer strictly for internet use.  The computers for running credit card transactions are on a separate network with a firewall in place, and many security settings to only allow CC processing and nothing else.  There are several sites that I manage, that either don't want to pay for security, or don't care.  Those people sign a waiver, not holding my company liable.

Hope that helps somewhat, and apologize for not commenting first.  If I can get more information on your setup with Denver, I will edit my answer.


You can't.

I suspect you're behind a NAT, and hence you'll need to port-forward from the NAT to your LAN IP.

The other option is hosting something on AWS/DigitalOcean and get an actual Public IP for your server that you can use as your LHOST.


Yes this would be secure.

Why would it be secure?
Because you would be running HMAC (the underlying primitive of PBKDF) as a stream-cipher using some key-derivation function.
As this is a common approach (at least for integrated encryption schemes) using this would at least as secure as using the bare KDF and a salt.

However I'd strongly argue against using PBKDF2 for anything but password-based key derivation and as a "stream cipher" it would give people a wrong sense of security (because it's 1000x slower = 10 bits more security).

If you want to use Poly-1305 I'd recommend you using a dedicated stream cipher (e.g. ChaCha20) or a dedicated stream cipher mode (e.g. CTR).


Relative merits of Yubico OTP vs. OATH-HOTP?

Usability

The Yubico AES mode does only work with a yubikey. The device returns a long data structure, that needs to be decrypted.

HOTP returns a 6 or 8 digit code. The algorithm uses a truncation to form the digit code. The digit code is created so that a user is able to read the code and type the code into a keyboard.

Thus: With Yubico AES mode you need to have a device that creates and inputs the one time password, while with HOTP you can always rely on the keyboard.

Sync

The Yubico AES mode increases two counters. A plug counter and a usage counter. This data is encrypted, then the encrypted value is sent for validation. Validation in this case means, that the encrypted value is decrypted and than the server verifies if the sent counters are bigger than the last counters, which the sever saw.

In HOTP mode the OTP value is calculated based on the counter.
The server needs to perform the same operation like the OTP token.
This is why you have this window thing. The token could be pressed without the value being sent to the server. The server knows the last value (counter=n) it saw. But it does not know, how many blank presses were performed on the OTP token. So the server needs to //try// n+1, n+2...

The HOTP token can get "out of sync". The Yubico AES mode can not.

Standards

Yubico AES mode is well documented. But it is only implemented by the Yubikey.
HOTP is standardized in RFC 4226.

Crytpography

Well don't ask me. I suppose that both AES and HMAC-SHA1 are sill strong algorithms...

...now you can decide which aspect is a pro and which one a cons. Usually it boils down to usability and the question if you have a USB port or not.
The yubikey is one of the few hardware tokens, that can be initialized and thus you can keep control over your key material. Use a open source backend lile privacyIDEA to enroll and manage the tokens.


Public Key Recovery Options

You have limited options:

You can recover your keys from a backup of your keyrings, if you have one.
You can recover public keys from a keyserver, if you or someone else uploaded your key there.
You can ask other people on your project for a copy of your public key, which they would have needed to verify your commits.
You can see if GitHub has a backup containing your public key, and is willing to retrieve it for you.

That's about it. If you have lost your keys, include backups and copies, then you should treat them as lost for good. Pragmatically, from a Git perspective you can:

Verify the Git history against a local or archived copy of the repository to ensure its authenticity, and then resign all your commits with a new key (if necessary, which it probably isn't).
Sign the current commit once you've validated it against your sources, and then use the new key going forward.

In Future...

Key material sufficient to recreate a public or private key can't usually be recovered just from signatures, so in future make sure you:

Make regular backups.
Escrow essential keys using a tool like paperkey or ssss, or some other recovery mechanism of your choice.
Ensure your public key is uploaded to a reputable public keyserver and signed by others for validation.
Store your public key under source control, perhaps even inside your source tree if appropriate for your project.
Make sure you have revocation certificates on hand in case you lose your key.

Basically, take steps to ensure that your public key and revocation certificates are recoverable at need, and trusted by other keys besides the key itself.


How can I track IPs to block malicious users?

You need to know which proxy servers you trust, and which headers to trust (if any) for each of them.

For example, you know your load balancer uses X-FORWARDED-FOR so you can trust the rightmost value that is supplied to your web server in this header.

Your load balancer will also need to know which IPs to trust. For example, it may trust RFC 1918 private IPs. You may also want it to trust certain public IPs if, well... if you trust them.

When trusting them, you would probably want to remove them from the chain of IPs that are candidates for storage.

As soon as you hit the first untrusted IP in the chain, then you should store that as the remote IP for the connection. The reason is because anyone can set X-FORWARDED-FOR within their HTTP request. If this REMOTE_ADDR is not in your trusted list for X-FORWARDED-FOR then you have no reason to trust the header.

This may be complicated if you are trusting arbitrary internet servers with different trusted headers for each.

The above is a generic approach for finding the actual connection that is requesting your page. However, if you are trying to track malicious users you may just want to gather any IP in any header as well as REMOTE_ADDR and then store multiple IPs for each connection. Then if they start doing anything "bad" on your system, you can add a black mark against all of those IPs.

Once so many black marks have been awarded, your system could start blocking connections with that IP in either REMOTE_ADDR or an IP HTTP header. You need to be aware though that a malicious user may try to blacklist other IPs in a denial of service attack. Therefore you are back to my first description of only trusting certain hosts.

Of course, a determined malicious user will always switch hosts when they feel the need to attack your system. e.g. by Tor, by VPN services or via their botnet. It's a tricky thing to accomplish so you may wish to consider other methods of blocking malicious users - e.g. forcing them to sign up with an account before using your system which is verified by SMS. This adds a high enough barrier to block all but the most determined of malicious users.


Short Answer:

No. (at least not in the versions I have worked with)

Longer Answer:

ZAP is not an exploitation tool, it is a vulnerability detection tool. You can however fuzz with ZAP to determine if SQL injection is possible but as it already detected a possible SQL injection the next step is exploiting it in order to verify if it is a true / false positive.

If this is a legitimate security assessment, what's wrong with using SQLmap or trying to exploit it manually?


Does Windows protect MBR from being overwritten?

An administrative/SYSTEM user has complete access to the system, and can make changes to disk partitions, etc. Also, tools exist for Windows which can modify the MBR to point to a different bootloader. So I'd say, no, there is nothing protecting it.

The question makes it sound like a big deal that "any program" running with administrator privileges can overwrite the MBR. A process with SYSTEM privileges can do whatever it wants. However, that is the standard threat model; if someone else runs code on your computer, it's not your computer anymore.


Visible anti counterfeiting in a mobile app?

I'm not sure about this particular system but one thing I have seen is a simple code (could be a little more sophisticated, like a movie) which changes every few minutes. The conductor just hast to watch the most recent one and then compare.

Given

the low cost of a ticket for a single ride and
the hassle of taking a nice enough picture or video to not get caught combined with
fines for getting caught

this can be sufficient protection.

In practice at least codes are dubious. The conductors don't actually seem to check mine, ever.


Is the macOS Universal Clipboard secure?

The specific encryption algorithm used by the Universal Clipboard feature has not been disclosed but I believe that all Bluetooth connections are encrypted (on some level) by default1. Read this answer to learn a little more about possible attack vectors: Is Bluetooth 4.0 traffic encrypted by default/design?. I don't know the exact implementation details of Universal Clipboard are but what I can say is that Universal Clipboard data is not uploaded to iCloud, it stays local to your devices2. That means that Apple can't see what you are copying and pasting.

See also this answer to the same question when it was asked previously: Universal clipboard iOS10 and macOS. How secure?

Footnotes:

1: Is Bluetooth 4.0 traffic encrypted by default/design?

2: http://arstechnica.com/apple/2016/09/macos-10-12-sierra-the-ars-technica-review/4/#h3


How can a client validate a OCSP Responders Delegate Certificate and how does it obtain it?

Actually, the bullet list does not contain the certs. But directly underneath it is mentioned:

  The responder MAY include certificates in the certs field of
  BasicOCSPResponse that help the OCSP client verify the responder's
  signature.

And in B.1.  OCSP in ASN.1 - 1998 Syntax

BasicOCSPResponse ::= SEQUENCE {
  tbsResponseData          ResponseData,
  signatureAlgorithm       AlgorithmIdentifier,
  signature                BIT STRING,
  certs                [0] EXPLICIT SEQUENCE OF Certificate OPTIONAL }

Thus the answer to my question is: The client looks for a certificate in the certs list of the response that matches the ResponderID in either name or key hash:

  o  either the name of the responder or a hash of the responder's
        public key as the ResponderID;

Then it validates the chain to verify that the response was signed by a trusted or authorized responder.

I found it helpful to review the Verify existing OCSP Basic Response implementation in PKI.js


How to implement child protection firewall that cannot be bypassed?

Have you looked into using a DNS service like OpenDNS? OpenDNS has content filtering as part of their services. I'm sure there are other providers but they are the ones I'm familiar with.


There are several kinds of VPN. But if the VPN in question does not use server authentication in some way (i.e. server certificate, shared secret...) or if the client improperly validates the server than this kind of MITM is possible. This is similar to HTTPS with a client not validating the servers certificate properly.

Edit: Actually, I think your question is already answered in Are VPNS vulnerable to active man in the middle attacks. Thus please study the answers there.


What does a failed SPF record tell me from a DMARC Aggregate report?

When setting up the SPF policy you are basically listing out the domains, subdomains and IP addresses that are allowed to send email for your domain. When the aggregate report states that the source IP address failed, it means that the mail was received from an IP address that did not appear in the list of IP addresses that are authorized to send email for your domain. So, as you said, somebody tried to spoof your domain name in the email, but SPF caught it because the IP address from which the mail came was not authorized to send email for your domain. It is most probably spam.


In Real-Time, where do we deploy Extensible Authentication Protocol(EAP) in Wireless Networks?

It's not in the average home Wi-Fi, not even in SOHO, it's mostly used in enterprise environment, hence the WPA/WPA2 Enterprise...

This is done when you try to authenticate to a WPA/WPA2 Enterprise AP, which mean you need to setup a RADIUS server to process the EAP Auth.

Sometimes also called 802.1x/EAP.

UPDATE FROM COMMENTS

This is in order to enhance security now that the AAA is done trough wired communication and not trough the air (The challenge/response can't be sniffed/cracked)


If I'm the owner of VPN network, what can my ISP see?

The ISP sees everything as it is, the connection is only safe between you and the VPN. As you mentioned, this VPN allows you to browse on insecure networks, public wifi hotspots with safety, defends against eavesdropping and MITM, so still good to have.

There are no other solutions. If you try to communicate with a website with or without VPN that only supports HTTP, the transferred data will still be unencrypted. It's the matter of between who.


Why Disallow Special Characters In a Password?

One explanation I haven't seen here is that many financial institutions are tightly integrated with older systems and are bound to the limitations of those systems.

The irony of this is that I have seen systems that were built to be compatible to older systems but now the older systems are gone and the policy still must exist for compatibility with the newer system that was built to be compatible with the older system.

(the lesson here is that if you have to be compatible with an old system, allow for some future elimination of those limitations).


What are my options to store a key which unlocks encrypted data locally on a device?

No matter how you slice this, if the key is available on the device and the key is not protected then the crypto system is flawed.

You are thinking correctly trying to find a secure way to get the key but in the end, it’s checkbox encryption and you store the key on the device or you find a way to get a keyboard on it to type a key in on boot.


Is traffic not encrypted given ERR_CERT_COMMON_NAME_INVALID?

You are correct. When you get a certification name invalid error, the piece of TLS that you lose is the authentication component; knowing that you're talking to the server you think you're talking to.  The encryption component is not affected.  So, as you suspect, you may not be (or may be, it could just be a configuration issue) talking to the correct server/site, but the encryption between you and whoever it is you are talking to is intact.


Limiting number of combo attempts with Fail2ban and 128 bits of entropy

You can set any value you want. See parameter maxretry in jail.conf.

You are right. Giving 100 attempts before blocking means that an attacker can try about 7 bits (log 100). Thus remaining will be 121 bits, which means very low probability to guess the password.

On the other hand, why would one want to give an attacker more chances to guess the password?

If you ban the attacker, he will need another IP to continue his attempts. If you ban the attacker after 5 attempts, it means he will need 20 times more different IPs than in the case with allowed 100 attempts. For the most of the single working attackers this is an essential increase of resources. They will stop the attempts.

One more aspect is your resources. Each login attempt means that your server needs resources (CPU, memory) to establish a connection, session, to check the password. In for a small number of attackers like 5-10 it will not affect other processes on your server. But if there are thousands of login requests from different IPs, this can require a lot of resources from your server. Thus the processes running on it can be slowed down. That's why it makes a difference, if you ban an attacker after 3-5 attempts or after 100 attempts.


What could someone do if they knew my MAC address?

Your MAC address is your machine's hardware address. This is mostly only important when an attacker is on the same network as you. At this point an attacker could do deauth attacks or intercept your traffic by posing as the networks router. However if an attacker is on or near your network is is trivial to get your MAC address. You can also change your MAC address as you please. The people trying to hide their MAC address likely do not fully understand networking and erroneously believe it to be a risk.


Can a picturefile with stegano be malware when saved somewhere?

Files are not inherently dangerous; attack files like this are either programs that need to be executed, or make use of issues in another program when loaded into that one as data.

From your article:

  To execute the hidden payload, the malicious ads load a heavily modified version of Countly, an open-source package for measuring website traffic. That JavaScript extracts the hidden code out of the image and executes it.

That is, the image contains a small JavaScript program, which is extracted and run by some code that the ad network places on a webpage, and viewing that webpage causes your browser to execute it.

If the image were to just sit on your hard drive, nothing happens. If you open it in an image editor, nothing happens. You have to execute it.

Now, many pieces of malware will take advantage of vulnerabilities in image viewing software, audio players, etc., so it is not generally safe to open up suspicious files in them; I don't mean to imply that web browsers are the only way to run an exploit. Similarly, malware will often try to get itself added to a machine's "run at boot" list, so that the OS will execute it automatically. But in this specific case, it appears the exploit runs through a browser.

Anyways, when it is run, an exploit can do (more or less) anything it wants, so yes, it could check for the presence of a valid network connection. It's mentioned that this one performs a series of checks to try and determine if it's being run in a vulnerability-checking environment, and those are probably much more sophisticated than merely checking if it can access an external server. But again, this can only happen if it's running.


How to make sure that the public key has not been tampered with during transit?

Either

you give the public key to Bob when you physically meet him and mutually verify identities (as at a key-signing party), or
Bob verifies your public key through a trusted introducer (e.g. a Certification Authority)

This is the "Infrastructure" part of PKI.


using msfvenom to encode an exe file without any payload, like oldy msfencode (packing)

I would advise to use following command:

msfvenom -p generic/custom PAYLOADFILE=[path_to_file] -a [architecture] --platform [platform] -e [encoder]

bear in mind you can list all possible encoders with:

msfvenom --list encoders

this is an example of encoding a shellcode using add/sub encoder, which I tested working:

msfvenom -p generic/custom PAYLOADFILE=./shellcode.bin -a x86 --platform windows -e x86/add_sub -f raw -o shellcode-encoded.bin


Does End of Life Hardware comply with PCI DSS 3.2 if still under support from a Third-Party, Not the actual vendor?

I worked for a PCI ASV for five years. This isn't compliant - you can't ensure that you're patched against security vulnerabilities if there are no security patches to apply.

However, your PCI ASV may, at their own discretion, choose not to fail you for it. I have seen clients in situations where changing the software would cost multiple millions, or where there's literally no other option. The way this is usually resolved is that they apply secondary security controls to mitigate the risk, for example isolating a Windows application inside a well-configured Citrix XenApp instance on a terminal services server, with AV installed (and up to date!) and OS lockdown in place. Or, for a web application, installing a WAF and making custom WAF rules to prevent exploitation of known vulnerabilities (e.g. XSS) as and when they come to light. In all cases this involves an in-depth security assessment of the EoL'd product in order to ascertain its security posture and risk. In most cases it also involves providing your ASV with a long-term plan to replace or move away from the EoL'd product.

Your ASV absolutely must mark non-compliant components of your network or system as non-compliant, but they may also mark it as an exception (with justification) and give you an overall pass on the assessment.


Your application looks like it's mainly running client-side because the server only receives those images. You therefore have very little data you can trust. This makes it very difficult to have it tamper proof, or be able to detect the data tampering.

Let's say you expect an untampered client to upload images every second to the server. Assuming the tampering takes half a second, you expect the tampered images to be uploaded every second and a half, but the tampered client can start processing earlier and still hit the 1 second upload rate. You cannot rely on timestamps provided by the client because they can also be forged.

You said you could rely on the upload rates of the other users in order to establish a pattern of know good rates, but what if an attacker or attackers would outnumber legitimate users? There is a difficult theoretical problem about this. A Google engineer described the difficult problem of detecting spam based on user reports and building systems that could resist sybil attacks.


Do I have any assurance that the previous owner does not have a valid
  HTTPS certificate for the site?

No, you don't.

CAs can issue certificate that are valid after the expiry date of the domain (at the time of issuance). Even if they didn't, a domain could be transferred before its expiry date.

In addition, you can't possibly control all the CAs that exist and that are trusted by potential clients. Even if there was a scheme that may monitor domain registration updates, not all CAs might be part of that scheme. You can't possibly know exactly all the CAs that your potential users may trust.

I can create my own CA and issue a certificate valid for the next 20 years for a domain that doesn't even exist yet. Of course, that's an extreme and pointless example, but if I use it as an internal CA and you happen later to register that domain, you'd have no way to know about it.

You could of course restrict your assumption to the major CAs around. However, as far as I know, they don't monitor changes to the whois database, at least for domain-validated certificates (I'm not sure if there's any provision for this situation with EV certs).


What kind of certificate do I need to be able to sign my own subdomain certificates?

Wildcard certificates, i.e. certificates which contain a name with a "*" in it, are not a solution for you. A wildcard certificate is a certificate which clients will accept as "matching" several server names: a certificate which contains "*.example.com" will be good for "foo.example.com", "bar.example.com", and so on. This is precisely what you do not want: you do not want the private key and certificate contained in one instance of your product to be good for another instance with a distinct name.

To achieve what you are trying to do, you need to obtain a CA certificate, marked as such with a Basic Constraints extension with the cA flag set to TRUE. This would allow you to issue certificates yourself, and the certificates would still link up to the root CA from which you obtained your own CA certificate.

Existing, deployed CA can sell you such a certificate, but it will be expensive:

The existing CA has a business model which sums up as "selling lots of individual certificates". If you produce many certificates without the help of the CA, then it is losing business... and will want to get its money back.
That existing CA has one huge asset, which is that its root public key is included in the Web browsers and operating systems of the whole world. OS/browser vendors accept to do that only if the CA offers strong guarantees that it is serious and will issue certificates only to duly identified entities. When such a CA delegates this issuance power to a sub-CA, it tends to require the same level of seriousness. So you would need some rather strong hardware (HSM in a bunker) and heavy procedures.

In a past life, I did such a deal with GlobalSign. Is was about 10 years ago, I don't know if they still offer such deals.

Another strategy is to let your customers deal with the issue. Document how to push a certificate and key in the application, and you are done. Some customers will use their own CA (you do not have control over the browsers of the users, but the customer might have such control). Some will buy a certificate elsewhere.

Anyway, you need to allow for such a manual certificate configuration, if only to allow renewal when the certificate expires.


What are good ways to check for tampering/bad behaviour on a linux server that has already been set up but it's your first time logging into it

If you have reason to believe that the system may be compromised then wipe the server and restore from last known good backup.

By all means take an image to see if you can determine how the system was compromised and plug the vulnerability. But this is not an easy process, particularly if you have no experience at this kind of thing - indeed if you attempt this yourself then it's likely that such intervention may occlude the nature of the compromise - certainly the "evidence" would be considered severely tainted by any law enforcement / jurisdiction.

Validating the package checksums is a starting point — but not nearly as effective as a properly managed file integrity checking regime (tripwire, LIDS etc). Logs are good place to look too. But you have to understand what these tools are telling you.

There's lots of things you should be doing before a system is compromised: configuring the permissions, firewall, optionally installing additional security measures such as mod_security, suhosin, fail2ban etc. Part of the maintenance of a server should be running a rootkit detector on it regularly. For a system/cluster of significant value you should also be carrying out periodic vulnerability scanning. But there's no point in doing any of this after the event.


SSH host key: Is the public one necessary and should the key be password protected

Q2. sshd host keys should not be password protected. If they are, sshd can't read them (unless you set up a host agent and preload it which is rare). This is because sshd is designed to run in background and cannot get the password(s) from the user, especially because it is usually started automatically at boot before any user is even logged-in. Instead the files for the sshd host key(s) should be secured against access by any userid other than the one which runs sshd, normally root, and that userid should be well protected and not used unnecessarily.

Q1. sshd does not need a separate .pub file because it can automatically compute the publickey from the privatekey. In contrast the client ssh (unless using an agent) usually wants to use the publickey for a userauth 'probe' before prompting for the password to actually use the privatekey, and this requires a separate .pub file except when using OpenSSH's 'new format' keyfile, which is required for Ed25519 since IIRC 6.5, and the default for all keys since 7.8 about a month ago. Since ssh-keygen doesn't know which you want, it always generates a .pub file, which does no harm if unneeded -- even if it is not secured, since it is public.


Whitelist mail domains and users to smtp on mail server

Your smtp should be set up so that only internal clients can access it. In Postfix this looks something like the following (obviously with your internal IP range).

smtpd_relay_restrictions = permit_mynetworks
mynetworks = 127.0.0.0/8 192.168.0.0/24

In smtpd_relay_restrictions you may also want to add permit_sasl_authenticated and set up SASL authentication for clients. You can then require all clients to be authenticated to be able to send emails outside of your servers destinations (defined by mydestination). See here or here among many others for a detailed description of setting up SASL for Postfix.

If you set it up with mynetworks, an external attacker (connecting from the internet) will not be able to use your smtp as an open relay. If an internal client is compromised, you have to fix that internal client, but requiring authentication from internal clients as well mitigates that risk somewhat (but a compromised client will potentially give away the credentials to the smtp server too).

Also one common mistake is when you put our smtp behind a firewall, set up your smtp to only allow local clients to send emails to external addresses, and then have your firewall rewrite packages with the source IP set to that of the firewall as opposed to the external client - so your smtp sees external traffic from the internal firewall IP address. You can fix this either by not rewriting the source address of packages, or not having the firewall address in mynetworks.


Protecting web site with nginx 301 redirect and HTTP firewall

You can either redirect http (80) to https (443) or block port 80.

You can't do both

redirect http (80) to https (443):

Helps users by redirecting them
May leak the first request

block port 80:

Doesn't leak the first request
But the user will not be redirected

Blocking port 80 have a (very little) security advantage, and a huge negative impact in term of usability.

If you really wants to improve security, you should use HSTS and HSTS preload.

See also https://community.letsencrypt.org/t/support-for-https-only-with-http-01-challenge/15134


I have no experience running or setting up such a thing but here are my thoughts on avoiding undue risk:

You will want some external monitoring on the critical services on the VM.  An exploit that causes a crash would certainly ruin it for everyone else.  It would be OK if someone defaces the website running on the box as long as all the necessary links to other pages remain intact but if they deface it so badly that the rest of the website becomes unreachable or starts throwing 500 Internal Error codes back at visitors or even crashes the HTTP daemon altogether, that should be detected and remedied quickly.
The attacker's machine may be exposed to some threat if you aren't supervising them.  The first exploit kit found by a Google search for the product they are aiming at is just as likely to contain malware as an actual exploit tool.
Political/legal risk.  Sign waiver forms for them to hold on to, explicitly allowing them to attempt to exploit this particular machine.  Explain the consequences of performing these same actions without the signed form.

And some general thoughts about running this thing:

Many exploits show no obvious signs of success.  Of course, this is part of the learning experience, to show the developers that simply switching off the displaying of errors does not completely stymie competent attackers.  But it might be nice to have a separate interface to show the current contents of the users table in the database or the contents of a particular directory or the list of running processes on the machine.  This might be something you switch off later on as the developer get better at detecting their own successes themselves.
Keep score.  For people who aren't naturally interested in security, making it fun is the key to keeping their attention, and a little bit of friendly competition will help that.  If you're only looking at allowing a single vulnerability, it won't be a score so much as a checklist of who has done it and who hasn't, but if a developer actually finds an extra vulnerability to the one you expected to be there, they should be publicly praised in some way.
Since you're aiming this at developers rather than pen-testers, their day job is going to be the not-so-glamorous defense role.  Once they have exploited the code, a followup task might be to see if they can protect the code themselves before seeing the best solution your team has collectively come up with.


Access Key In URL - Securing The Unsecured!

All you'll need in this case is GUID's for the key. Most implementations are random, and they're always going to be unique.

Of course you could AES encrypt it, but that doesn't really add any additional security over and above using standard GUID's.


Migrating GPG master keys as subkeys to new master key

It can actually be done.

The instructions at atom.smasher.org/gpg/gpg-migrate.txt are now out of date.

Try this. As always, make backups, because it's really easy to mess it up.

So these are your 'old' keys:

$ gpg -K
----------------------------------
sec   2048R/712A2BBD 2013-01-29
uid                  Test Key 1
ssb   2048R/F0B63FDA 2013-01-29

sec   2048R/72838B89 2013-01-29
uid                  Test Key 2
ssb   2048R/A19F06EC 2013-01-29

sec   2048D/AC349218 2013-03-21
uid                  Test Key 3
ssb   2048g/179E9F47 2013-03-21

And you have created a new "master key" which has no subkeys:

sec   4096R/CB577A43 2013-03-21
uid                  Master Key

For simplicity's sake, I will only rehearse the procedure using the first of your "old" keys:

sec   2048R/712A2BBD 2013-01-29
uid                  Test Key 1
ssb   2048R/F0B63FDA 2013-01-29

Using the same procedure, you can add all of them. I recommend you add all of them at the same time, because every time you break the keys and then reassemble them you have to recreate the binding signatures, and that is a pain.

1. Use gpgsplit to split the "old" key into its constituent packets.

$ gpg --export-secret-keys 712A2BBD | gpgsplit -vp 712A2BBD_

Now look at the files it created:

$ ls
----------------------------------
712A2BBD_000001-005.secret_key
712A2BBD_000002-013.user_id
712A2BBD_000003-002.sig
712A2BBD_000004-007.secret_subkey
712A2BBD_000005-002.sig

The first file, "712A2BBD_000001-005.secret_key" contains the packet for the main private key (packet type tag 5, hence the "005") of your "old" key, "712A2BBD". (You will need this, but you will need to convert it into a subkey if you want to migrate it into your new "master key". In keys generated using standard settings on GPG, the main private key normally has usage flags "S" for sign and "C" for certify, and does not have "E" for encrypt, which is normally why the key is generated with an "E" subkey. However, because of the procedure we are using, when eventually the main key is turned into a subkey and migrated to your new "master key", it will have all usage flags ("SCEA") enabled. This probably makes it less secure, and is probably a behaviour to be avoided.)

The second file, "712A2BBD_000002-013.user_id", is the packet for the user id (packet type tag 13). (You won't need this).

The third file, "712A2BBD_000003-002.sig", is a binding signature (packet type tag 2) for these packets. (You won't need this).

The fourth file is the packet for the secret subkey (F0B63FDA) (packet type tag 7). (You will need this. Typically, a subkey like this has the "E" usage flag, meaning it is solely for encryption. However, in the process of importing it to your new "master key" it will acquire all of the usage flags, ("SCEA"), which is probably not a good thing.)

The fifth file, "712A2BBD_000005-002.sig", is the binding signature (packet type tag 2) for the subkey. You won't need it.

Save the two files you need and remove the others.

2. Change the main key (712A2BBD) into a subkey.

You don't need to do anything to the subkey (F0B63FDA) in order to migrate it to your new "master key", but the main key (712A2BBD) of your old key needs to be altered in order to make it work.

Open the main key, "712A2BBD_000001-005.secret_key", in an editor capable of making hex or binary edits. I used ghex:

$ ghex 712A2BBD_000001-005.secret_key

If you are using a hex editor, the first byte will be "95". Change it to "9D".

If you are using a binary editor, the first byte will be "10010101". Change it to "10011101".

Save it, and then use pgpdump to see if it you have been successful in changing it to a subkey. If you have been successful, you will see the following output:

$ pgpdump 712A2BBD_000001-005.secret_key | head -1
-----------------
Old: Secret Subkey Packet(tag 7)(1836 bytes)

3. Add a "dummy" subkey to your new "master key" for each of the subkeys you want to add.

Just like in the tutorial at atom.smasher.org,   you will need to create new "dummy" subkeys on your "master key" in order to make sure you have keybinding signatures on the master key which can accomodate (at least at first) the migrating subkeys.

Think of it as creating "slots" for the old subkeys to slot into. You will need as many slots as keys you want to add.

In the present example, we'll make two dummy subkeys, but if you want to add all of the keys you listed in your original post, you'll want to make six.

$ gpg --expert --edit-key CB577A43
----------------------------------------
gpg (GnuPG) 1.4.12; Copyright (C) 2012 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Secret key is available.

pub  4096R/CB577A43  created: 2013-03-21  expires: never       usage: SC
                     trust: unknown      validity: unknown

gpg>addkey

Make sure, while going through the subkey generation wizard (which is in expert mode, thanks to the --expert option above) that when you generate your new "dummy" subkey, it is of the same...

key strength (2048, 4096, etc)
type (DSA, RSA, elgamal)
usage (encrypt, sign)

...as the subkey you intend to add in that "slot".

Since we're adding two subkeys, we finish up with two new "dummy" subkeys on the "master key". Save and exit:

pub  4096R/CB577A43  created: 2013-03-21  expires: never       usage: SC
                         trust: unknown      validity: unknown
sub  2048R/AAAAAAAA  created: 2014-01-01  expires: never       usage: S
sub  2048R/BBBBBBBB  created: 2014-01-01  expires: never       usage: E

gpg> save

4. Split your new "master key" - along with its new subkeys - into its constituent packets.

$ gpg --export-secret-keys CB577A43 | gpgsplit -vp CB577A43_

You should end up with something like the following:

$ ls
----------------
CB577A43_000001-005.secret_key
CB577A43_000002-013.user_id
CB577A43_000003-002.sig
CB577A43_000004-007.secret_subkey
**CB577A43_000005-002.sig**
CB577A43_000006-007.secret_subkey
**CB577A43_000007-002.sig**

The fifth and seventh files (the ones I have put in **s) are the subkey keybinding signature packets that were generated when you generated the "dummy subkeys." You'll need these.

The fourth and sixth files are the actual "dummy subkey" packets. They can now be safely deleted (the only files you should delete.)

$ rm CB577A43_000004-007.secret_subkey CB577A43_000006-007.secret_subkey

5. "Slot in" your migrating "subkeys" into the split master key.

Steps one and two left you with two files:

712A2BBD_000001-005.secret_key (the original main key of your "old key", now edited into a subkey)
712A2BBD_000004-007.secret_subkey (the original subkey of your "old key".)

Now copy these into the folder with the split constituents of the "master key" and rename these files so that they replace the "dummy subkey" files that we just deleted.

$ mv 712A2BBD_000001-005.secret_key CB577A43_000004-007.secret_subkey
$ mv 712A2BBD_000004-007.secret_subkey CB577A43_000006-007.secret_subkey

You should once again have seven (in this example) files, but files four and six are now your old keys, all set and ready to become subkeys of your new master key.

$ ls
----------------
CB577A43_000001-005.secret_key
CB577A43_000002-013.user_id
CB577A43_000003-002.sig
**CB577A43_000004-007.secret_subkey**
CB577A43_000005-002.sig
**CB577A43_000006-007.secret_subkey**
CB577A43_000007-002.sig

6. Concatenate the files into private and public key files, ready for import

Put the files together so that they are concatenated consecutively into single files, for ease of import later.

$ cat CB577A43_000001-005.secret_key CB577A43_000002-013.user_id CB577A43_000003-002.sig CB577A43_000004-007.secret_subkey CB577A43_000005-002.sig CB577A43_000006-007.secret_subkey CB577A43_000007-002.sig > concatenated_CB577A43.secret

Use the same command, but pipe the output to gpgsplit in order to create public counterparts, before redirecting to a new file:

$ cat CB577A43_000001-005.secret_key CB577A43_000002-013.user_id CB577A43_000003-002.sig CB577A43_000004-007.secret_subkey CB577A43_000005-002.sig CB577A43_000006-007.secret_subkey CB577A43_000007-002.sig | gpgsplit --no-split --secret-to-public > concatenated_CB577A43.public

7. Back up your keyring and then delete the "old" AND "master" keys, both public and private, from your keyring.

$ cp -R ~/.gnupg/ ~/.gnupg_backup
$ gpg --delete-secret-and-public-keys CB577A43 712A2BBD

8. Download and build gnupg-1.2.0.

Recently whenever anyone tries to follow the tutorial at atom.smasher.org, the problem they run into is that, with the newest versions of gpg, once they reassemble the key and add it to their keyring, the added subkeys 1) have no usage flags, and 2) they are unable to reset the expiry date, as advised in the atom.smasher.org tutorial, a necessary step to creating new and valid keybinding signatures. The result is that after one export/import cycle, the keys are deemed invalid by GPG, and disappear from the private keyring.

In order to successfully import the keys, you have to download and build an old version of gnupg: a version from 2002 no less. One you have build it, you will be able to reset the expiry dates on the new keys, and successfully recreate valid keybindings for the keys.

Download the old gnupg.

$ wget ftp://mirror.tje.me.uk/pub/mirrors/ftp.gnupg.org/gnupg/gnupg-1.2.0.tar.bz2
$ tar -xjf gnupg-1.2.0.tar.bz2
$ cd gnupg-1.2.0/
$ ./configure
$ make

Navigate into the g10 folder, which contains your newly build gpg binary

$ cd g10

In order to use this binary, instead of the system-wide build of gpg, which is more recent, you have to type the absolute path to the binary. If you are in the same folder, use ./gpg, for example:

$ ./gpg --list-keys

8. Append the new assembled "master key" file with imported subkeys to your public and private keyrings.

Don't import the keys the normal way, because the keybinding signatures don't check out, with the result that the imported subkeys wouldn't import the normal way. Instead, cat the files to the end of your public and private keyrings.

$ cat concatenated_CB577A43.secret >> ~/.gnupg/secring.gpg
$ cat concatenated_CB577A43.public >> ~/.gnupg/pubring.gpg

Check that they have gotten in there by using the gnupg-1.2.0 to list the keys:

$ ./gpg --list-keys
-----------------
pub   4096R/CB577A43 2013-03-21
uid                  Master Key
sub   2048R/712A2BBD 2013-01-29
sub   2048R/F0B63FDA 2013-01-29

9. Edit the keys using the gnupg-1.2.0.

Make sure to navigate back to the build folder for gnupg-1.2.0/g10, and invoke the built binary to edit your newly imported, newly assembled master key, complete with your imported old subkeys:

$ ./gpg --edit-keys CB577A43
-----------------
sec  4096R/CB577A43  created: 2013-03-21 expires: never
ssb  2048R/712A2BBD  created: 2013-01-29 expires: never
ssb  2048R/F0B63FDA  created: 2013-01-29 expires: never

First, reset the passwords.

Command> passwd

It is likely that all of your old keys had different passwords from your new master key. If you reset the password now, it will prompt you for each of the passwords of the old keys in turn (along with the subkey id) and when you have inputted them all properly, it will ask you for a new key. Once this is done, they will all have the same password, avoiding weirdness later.

Second, reset the expiry date on ALL of the keys. Begin with the main master key:

Command> expiry

And then work your way through each of the subkeys, changing the expiry date on each of them.

Command> key 1
Command> expiry
Command> key 1
Command> key 2
Command> expiry
Command> key 2
Command> key 3

And so on

Make sure you change it to a value it was not on already, or there will be no change. You can change it back to the original value later on: what is required now is a change, so that gpg-1.2.0 can create new subkey binding signatures.

Once all of this is done save.

Command> save

gnupg-1.2.0 will exit and save the new keybinding signatures, validating the migration of your old keys into your new master key.

10. Now edit the keys using the standard (current) gnupg, to see if it worked.

Open the same key for editing using the standard, system-wide version of gpg, to see if it worked. If it worked, you will see two things:

The expiry dates will be the new expiry dates you have just changed to in step 9, and not the original ones.
The usage flags to the right of the subkeys will display "usage:SCEA"

.

$ gpg --edit-key CB577A43
--------------------------
pub  4096R/CB577A43  created: 2013-03-21  expires: [newdate]     usage: SC
                     trust: unknown      validity: unknown
sub  2048R/712A2BBD  created: 2013-01-29  expires: [newdate]     usage: SCEA
sub  2048R/F0B63FDA  created: 2013-01-29  expires: [newdate]     usage: SCEA

If it did not work, the usage flags will be blank, and the expiry date will now show the change you just made in step 9.

$ gpg --edit-key CB577A43
--------------------------
pub  4096R/CB577A43  created: 2013-03-21  expires: **never**     usage: SC
                     trust: unknown      validity: unknown
sub  2048R/712A2BBD  created: 2013-01-29  expires: **never**     usage:
sub  2048R/F0B63FDA  created: 2013-01-29  expires: **never**     usage:

Finally, you need to put it through one export/import cycle, and delete the keys in between. If it survives this, the procedure has worked.

$ gpg --export-secret-keys CB577A43 > CB577A43_new.secret
$ gpg --export CB577A43 > CB577A43_new.public
$ gpg --delete-secret-and-public-keys CB577A43
$ gpg --import CB577A43_new.secret CB577A43_new.public
$ gpg --edit-key CB577A43
--------------------------
pub  4096R/CB577A43  created: 2013-03-21  expires: [newdate]     usage: SC
                     trust: unknown      validity: unknown
sub  2048R/712A2BBD  created: 2013-01-29  expires: [newdate]     usage: SCEA
sub  2048R/F0B63FDA  created: 2013-01-29  expires: [newdate]     usage: SCEA

11. Final tasks.

Assuming it worked, you will now need to cross certify your subkeys, so that they can create and verify signatures.

$ gpg --edit-key CB577A43

And from the gpg edit prompt:

gpg> cross-certify

You will also need to reset trust:

gpg> trust

Finally, make sure to test the key rigorously, to make sure it works before you distribute it. I refer back to the atom.smasher.org tutorial for these final steps:

  check all expiration dates and preferences. some of these operations may have changed your expiration dates and preferences; reset as
  necessary.
  test out all key components for creating and verifying signatures, and encryption/decryption. use the bang (!) to force each (sub)key:
  create & verify signatures:

$ date | gpg -u '712A2BBD!' --clearsign | gpg -v --verify
$ date | gpg -u 'F0B63FDA!' --clearsign | gpg -v --verify

  encrypt/decrypt:

$ date | gpg -ear '712A2BBD!' | gpg -v --decrypt
$ date | gpg -ear 'F0B63FDA!' | gpg -v --decrypt

  after testing out the keys locally, send your new public key to one or two people and test all key components (sending signed/encrypted
  messages to each other using all key components). make sure that they
  first delete (from their keyrings) your old key! and make sure that
  they understand that the key should NOT be circulated until all
  functions are verified to be working!
  when putting the new key into circulation, it's probably a good idea to expire/revoke the old key. include a revocation comment that
  specifies the new key ID and instructions to delete the old key from
  the keyring.
  note on key revocation: according to the OpenPGP standards a revocation generated by a sub key will be ignored, unless that subkey
  has been designated (by the primary key) as a revocation key. GnuPG
  seems to behave correctly, but some versions of PGP(tm) may not. if
  someone is claiming that your new key is revoked, have then remove all
  of your old and current keys from their keyring: then re-import your
  current key(s).


Exchange encrypted messages on public chatroom

This is the typical issue with encryption: the trouble is not how to securely encrypt a file but rather how to handle the encryption keys.

Basically, you have several options:

Use a secure channel to exchange the encryption key. It could also be a less than secure channel that you just hope isn't intercepted by anyone monitoring you line (SMS, for instance)
Use a method to securely agree on a common secret to use as encryption key (for instance, manually performing a Diffie-Hellman key exchange although you'll have to make sure no one can actually modify your exchange since DH isn't authenticated).
Use a public/private key system (for instance, PGP) to each generate a key pair and exchange your public keys in your chat room. Again, you should watch for anyone having the ability to change the traffic each of you see because they could perfom a MITM against you (in this case, i suggest using a different channel to confirm your keys. PGP makes it easy by having a key fingerprint that is much shorter than the actual key.

In practice, I would go for PGP and confirm each key over phone (it's hard to intercept and harder to fake if you know your interlocutor). Once you have exchanged keys, you can transfer any number of file using the same keys so it's actually pretty practical.


What is the difference between Key Derivation Function and (salted) Hash?

There are actually two kinds of KDFs. One kind is designed to derive a key from high-entropy input (like another key); this can be done with a fast keyed hash like HMAC. The other kind takes a password as input. Passwords are low-entropy; they're not inherently very hard to brute-force. A good password hash thus has to be slow.

In your question, you said that adding for (i=0; i<bignum; i++); would slow the hash. This is actually completely useless. The attacker does not have to play by your rules. Hashes need to protect passwords when the attackers have a copy of the hashes. If the attacker can compute hashes quickly, it doesn't matter how slowly you compute them. Hashes need to be inherently slow; there should be no shortcuts to evaluate them faster than the legitimate server.

The "randomness properties" are because a KDF needs to produce a key. They have nothing to do with precomputation, including rainbow tables. Cryptographic algorithms typically make certain assumptions about the key; among other things, they normally assume it was selected totally at random from the set of possible keys. Keys are also expected to be a certain length; their derivation functions need arbitrary-length output.  In contrast, it's OK if a password hash has lots of structure to the output. Maybe there's a 70% chance that adjacent bits have the same value. Maybe it spreads 128 bits of entropy into 4096 bits of output. As long as it's hard to reverse, that's a fine hash, but it's unsuitable as a key.

A secure password-based key derivation function is a secure password hash (PBKDF2 is in fact one of the big 3 hashes). The reverse is not necessarily true. Which one to use is simple: use a password-based key derivation function to derive a key from a password, and a hash to store passwords.


Is there a way to accept signed requests without storing client's password in plaintext?

Strictly speaking, the use of the term "signature" about HMAC is improper; it is widespread, but still incorrect: signatures are about asymmetric algorithms with a public and a private key, such as RSA.

HMAC uses a secret key (i.e. a bunch of arbitrary bytes) which is used for computing the MAC and for verifying it (it is actually verified by recomputing it). So, in your problem, yes, the server must store either the secret key, or some data which is enough to recover it.

Note that the secret key K can be derived from the password in a non-invertible way. E.g. through a hash function or, better yet, with a Key Derivation Function. The server will store the key K "in plaintext" and knowing key K is enough to produce valid HMAC values. However, this forces an attacker to use some specific software for that, not the standard client which requests a user password. Depending on the context and your attack model, this may or may not be a security improvement.

A way to achieve what you are looking for, but with higher complexity then a plain HMAC, is to use SRP. SRP is an asymmetric key exchange protocol with mutual password-based authentication, with the following characteristics:

No need for public key distribution (PKI, certificates...).
Client authenticates server and server authenticates client, in the same pass, even in the presence of active attackers ("Man-In-The-Middle" attackers are defeated).
No active or passive attackers learns enough information to perform offline dictionary attacks (a dictionary attack is guessing the password by trying random words; the attack is offline if it can be done without interacting with the honest client or server for each guess).
What the server stores is not enough to learn the password or impersonate the client (but it allows offline dictionary attacks, which is unavoidable).

The key which results from the SRP protocol can then be used for a MAC algorithm such as HMAC. It needs not be stored: it is just kept in RAM for the duration of the session.

SRP can be integrated in TLS, as explained in RFC 5054. The easiest, most standard supported way to use SRP in your context would be to use HTTPS (i.e. HTTP within a SSL/TLS tunnel) with the TLS part using SRP for key exchange. GnuTLS is an opensource implementation of SSL/TLS which supports SRP.


Actually, it is possible to restrict root if one is prepared to define such a restriction as fundamentally trusting the operating system. This can be done using SELinux (that I know of) and presumably other such systems. The best such example I have seen of SELinux used in such a way is Russell Coker's Play SELinux Machine.

As a brief overview of how it works, the name "root" is not special in Unix. The UID 0 is. UID 0 is taken to mean "trust everything I say". This applies specifically to the standard access model used on Unices, Unixen or however you pluralise "Unix".

LSM, or Linux Security Modules, allows you to hook into just about everything and audit/deny actions as you see fit. In the case of SELinux, SELinux permissions are checked after Unix permissions, so your flow looks like this:

Event ----> Has Unix Permissions? ---> Has SELinux Permissions? ---> Let it happen

The next stage to understand is that there are, or have been historically, different versions of the SELinux Policy. Before I get into that, understand that in SELinux:

inodes have types, suffixed _t, which might also be called domains; and
users have roles, suffixed _r.

Combined, they control what action a user in a given role may do, and what a process in a given domain may do.

Now, there are three main SELinux policies:

targeted. This is the default policy for desktops like Fedora. The idea of targeted is that critical system services and daemons are run in domains, but a lot of what you the end user does is run in unconfined_u:unconfined_r:unconfined_t. No prizes for guessing what that means - if Unix permissions work, SELinux is effectively pass-through.
strict. This policy involves removing unconfined_u entirely. This is not an easy process, especially on the Linux desktop (i.e. init 5). Specifically, the X11 security model is not great and often requires unconfined_t for some applications. You can do this, but I would expect working with X11 to be harder (although not impossible) especially when executing GUI applications that require root. There is a project underway to provide support for SELinux-like functionality in X, called XACE (X Access Control Extensions).
MLS. MLS stands for multi-level security and means the end of the permission string: user_u:system_r:httpd_content_t.s0-s2:c5-c7, i.e. those s and c numbers actually mean something. Specifically they form a no read up, no write down setup such that unless the process is running as a certain level, the information they can see will be limited. The idea of this level of information is to protect classified assets—SELinux was originally developed by the NSA, presumably for this purpose.

So that's your background. Now, according to the web pages (see FAQ) root is UID 0 on the play machine; however, root is set to run as user_r and not sysadm_r in the strict policy. This means the user will not be allowed to execute administrative functions from the shell.

What would be interesting to know, then, is the status of the other processes that do require root. Presumably such processes have been appropriately labelled and have policies that allow them the access they require. The question then becomes how do you administer the system and can any of these processes launch a shell in that user's context. If that can happen, you can still manage an exploit.

Since the play machine is currently down (at time of writing), I'm working on assumptions here; but essentially, you would need a process running in sysadm_r and with UID 0 as your target for an exploit. Assuming such a process exists and is exploitable, you could still get to root. As for still being able to admin via root, there are two options that I can think of:

Either there is a trusted transition of some sort that means root then can transition to sysadm_r (less secure) or
On different runlevels, a different policy applies, so to admin the machine one sets the runlevel to 1 and the policy does not restrict root. I'm guessing here.

Summary

If your question is "can I easily and safely do this now?" the answer is no. If your question is "I am prepared to learn about SELinux, get down and dirty with my distribution and put up with quite a lot of things not working" the answer is it is possible to constrain root much more than your average install. That said, this does not in any way make you invulnerable to exploits—it does not make it impossible for a user to circumvent this extra access control either in software or physically.

So yes, you can make things invisible to root; however, the extra technical burden aside, the same caveats apply as to any access control setup on any normal user—there is no silver bullet.

Oh and blatant self-promotion: you might like my blog post on storing secrets in software. It's on the security stackexchange blog, so I don't feel so bad about promoting that. Essentially, as you can see, there are mechanisms to make life a lot more difficult for an attacker (and you), but in the end, it's a case of "turtles all the way down" i.e. fundamentally impossible to do.


Does NIST really recommend PBKDF2 for password hashing?

The recommendation is for PBKDF2 to be used as an algorithm for generating a cryptographic key from a password, not for hashing a password for safe storage for authentication purposes. (I trust you're salting as well?) So the answer is no, for your use case there is no such recommendation. This doesn't mean it's not suitable, but there isn't a NIST recommendation to cite.


Which cipher suites are proposed by client and which one chosen by server?

The client offers a list of cipher suites in the first message it sends, called ClientHello.

The server offers a single ciphersuite it chose from the list offered by the client, in the first message the server sends, called ServerHello.


Concerted attack on linked google accounts?

A few things to note: the account and password lookup features on HaveIBeenPwned only represent found from public database leaks that have been cracked or were in plaintext.  It's still possible that your account was still breached but the breach never publicly surfaced.  Most websites don't delete inactive accounts, so a recent leak could have been responsible for your old accounts being compromised.
If I had to speculate, assuming it wasn't someone targeting you specifically, your account was probably part of a database leak. They took the credentials from the leak and began credential stuffing, eventually logging into your email account.

My most likely explanation would be that another (linked) account got compromised and these e-mail addressed where extracted from there?
Or should I consider my personal computers to be compromised as this sort of information could technically have been extracted from there?

I find these two scenarios less likely for a few key reasons:

If a linked account was compromised, they wouldn't immediately have access to your other account's password unless you had it saved somewhere.
If a linked account was compromised, you likely would have seen a login alert for that account as well.
Assuming your password wasn't saved in the linked account, they would likely need to use the "password recovery" feature to reset the password and gain access.  It sounds like you still have access to the account, so that doesn't seem to be the case.
If your personal computer was compromised, it wouldn't have been your 5-year old unused email accounts being breached.

... is it also considered good practice in this case to rebuild my personal machines?

If you want to rebuild your machines, that's up to you.  However, if you are going to factory reset your machines, make sure you do that first.  If your devices are compromised, changing your passwords before doing a reset won't fix the issue.
It may not seem related, but keeping regular backups makes factory resets easier to recover from, meaning less downtime from having an account breached.
Another thing you may want to consider is that all emails in your breached accounts may have been read.  Years ago, it wasn't uncommon for websites to send you your account passwords in plaintext when you signed up.  You may want to search for "password", "welcome", or "login" in your old emails to see what other accounts may have been compromised.