Is there an algorithm to generate digitally signed URL links? Scenario:

Alice prepares an URL link to Bob's web service (e.g., http://www.bob.com/sensitiveData/123?login=mallory).
Alice gives the URL link to Mallory.
Mallory follows the link to trigger an action in Bob's system.

Alice's link is intended for Mallory. Note that Mallory is authenticated by Alice but not by Bob. Bob does not know Mallory, but he trusts Alice. So if Alice has authorized Mallory to perform the action, Bob will blindly execute it.

The problem: How can Bob be sure that it is really Alice's link and that Mallory has not modified it in step 2?

In theory, such problems can be solved by a digital signature with a public-key algorithm. Alice could generate a signature of the link with her private key and append it to the URL, for example, http://www.bob.com/sensitiveData/123?login=mallory&signiture=<SOME-SIGNITURE>).

How do you generate the signature? As an experiment, I just tried gpg --encrypt but the results are quite huge. Additionally, it has to be encoded to use it inside an URL (e.g., with Base64), which will make it even larger.

As I'm not very experienced with cryptographic algorithms and protocols, I wanted to ask whether there is an well-known solution to the problem of generating signed URLs?

If not, do you see any flaw in my approach? Do you have any recommendation for the generation of the signature that can be appended to the link? If possible, it should be relatively short.

Just for clarification: Unfortunately, it is not possible to change the overall protocol. Of course, it would be safer if Alice and Bob could directly communicate over an encrypted connection. Additionally, Mallory should have to ask Alice to tell Bob to trigger the action, or Bob should have the option to ask Alice to authorize the action.

A RSA signature has the length of the modulus: if using the usual 2048-bit key size, then signatures are 256-byte long. Since you want to fit the signature within a URL, which is text-based, you must have some sort of encoding such as Base64, which implies some size overhead; with Base64, 256 bytes will become 342 characters -- that makes for URL of quite respectable length.

DSA (and its highly fashionable elliptic-curve variant ECDSA) offers smaller signatures; you can get very decent security (as much as 2048-bit RSA) with signatures of size 60 bytes or so -- there again, Base64 encoding inflates that to 80 characters.

If you need even smaller signatures, then you must go to less mainstream algorithms, e.g. BLS, which offers signatures twice smaller than DSA. But the maths are fiendishly more complex, and there is no standard, only scientific papers.

Alternatively, you may want to use RSA with ISO 9796-2 signatures (not PKCS#1). You will get 256-byte signatures, but you may then smuggle some of your application data within the signature itself. As a rough approximation, you will get your n-bit security with a total overhead of about 3n bits; albeit with a minimal message size of 256 bytes.

Theoretical limit: if you want a security level of n bits (meaning that attackers are assumed not to be able to run computations which need more than 2n elementary operations), then the signature size cannot be less than n bits either. Indeed, an attacker could try brute force on the signature, trying out all possible signature values and using the signature verification algorithm (which uses Alice's public key, which is public, hence known to the attacker) until a matching signature is found.

However, there is no currently known secure digital signature algorithm which offers n-bit security with n-bit signatures. DSA needs to use 4n-bit signatures to offer n-bit security. BLS lowers that to 2n bits. There have been proposals for some other algorithm types which could reach down to 1.5n bits or so, but right now they all turned out to be flawed in some way.

It may be possible that you could change your model slightly. For instance, Alice and Bob may share some secret value; Bob then does not really trust "Alice" but rather "whoever knows the secret value, which happens to be known to both Alice and Bob". In that case, you can rely on a MAC algorithm instead of a digital signature; you can think of MAC algorithms as signatures where the key to sign and the key to verify are the same.

It is a change of context. Depending on who Alice and Bob are in your system, sharing a secret may or may not be possible. Using a MAC forfeits most chances at non-repudiation. It also means that Bob can himself produce URL that he will accept. Yet, if you can tolerate the use of a MAC, then you can get very short verification elements. In fact, you can even get below n bits because exhaustive search can no longer be applied: since the MAC verification key is secret, the attacker cannot try MAC values and decide whether they are correct on his own machines. To "try" a potential MAC value, the attacker MUST then send it to Bob, and see if Bob is happy with it or not. After a million tries or so, Bob may begin to suspect foul play and apply countermeasures (e.g. cease to respond to that decidedly dodgy requester).

In that sense, a MAC value of length 64 bits ought to be sufficient. That's just 8 bytes -- encodable with Base64 as 11 characters. HMAC is a widely implemented MAC algorithm with good repute.

There is no well-established standard for "signed URL". Every site designer works out his own scheme, which is unfortunate because homemade cryptography is one of the surest paths to disaster. The conceptual model, though, is sound: basically, you want Mallory to convey a message from Alice to Bob, such that Bob can know that the message really comes from Alice, and was not altered or even invented altogether by Mallory. That the message travels as a "URL" is a mere encoding constraint which has no impact on the concept.

A MAC or a signature is indeed the right tool here. Beware, though, of replay attacks: if Mallory has an Alice-signed URL that Bob will accept, Mallory may try to send that URL twice to Bob, to get twice the effect. Depending on your context, this may or may not be a problem. If it is, then Bob MUST enforce some protection mechanism, e.g. remembering past requests to reject duplicates. A time stamp within the URL data (under cover of the MAC or signature) can help Bob keep track of seen URL (i.e. Bob can forget "expired" URL since he won't accept them any more).


How can I know why Nmap identified a service or OS the way it did? Nmap uses a variety of testing methods and fingerprints to determine - or at least guess at - what services are running on each discovered port, and what operating system is running on the host. (When -sV and -O, or other appropriate options, are used.) I'm sure all of this is fairly well-documented in the Nmap online documents, and in the scripts that are run for these purposes. However, Nmap itself doesn't seem to do a very good job of exposing these internals - at least, not with default settings.

I'd like to know if there's a way to add some sort of justification for Nmap's service/OS identity selections to its output. Something that, for each service/OS ID, says something along the lines of "[Service/OS ID] chosen because [Test(s) used for fingerprinting] resulted in [Response(s) received]". Even better might be an additional list of other possible fingerprint matches that are very close to the one observed, or any responses from the target system which deviated from those normally expected of the indicated service/OS.

I think some of this may be available through tweaking verbosity and debugging output levels. However, I'd rather (if natively supported) be able to get these details without bloating the output beyond what's necessary to serve this purpose.

How do I get Nmap to provide output that provides additional transparency into its service/OS fingerprinting logic? Is there a way to do this without adding a lot of other noise to the output?

Ok, let's see if I can help:

OS Fingerprinting

At default verbosity, the OS fingerprinting engine provides the line number of the matched fingerprint in the nmap-os-db file in the XML output as the "line" attribute of the "osmatch" element. (For IPv6, this number is the ordinal group within the classifier. Because of the machine-learning approach, it's harder to see why a particular scan matched a particular OS group.)
With -v2 or higher or -d, the exact fingerprint (observation) that was used to match is printed in Normal and XML output formats.
Sometimes an OS will be printed on a "Service Info" line in the Normal output. This comes from the o// template in the nmap-service-probes file. The XML output has this info directly under the "service" tag as the "ostype" attribute. More on this in the next section...

Service fingerprinting

At default verbosity, the service fingerprint (exact probe responses) will be printed if no match is found and you did not reduce version intensity with a command-line option (--version-intensity or --version-light).
The --version-trace option turns on lots of interesting Normal-mode output regarding which probes are sent, and which ones match.
The --packet-trace option can print exact hex dumps of what is sent and returned.


Client Certificate Authentication enough multiple server-backends and multiple server-clients I have a flexible number of servers A (same type of servers but located in different regions, so serving different data) that need to authenticate api calls from a flexible number of servers B (same type of servers but located in different regions).

Any server B can call any server A. And of course, a server A in one location is actually a load balanced bunch of multiple servers.

So I need all calls from servers B to be authenticated when calling any server A, and instead of doing something equivalent to what one would do with tokens/sessions/..., I narrowed down to Client Certificate Authentication.

However, to avoid a devops deployment nightmare, what would be the best strategy?
- One unique client certificate used by all servers B (easiest?)
- One unique client certificate used by all servers B for each server A?
- One unique client certificate used for all servers A, but different per server B?
- A cartesian multiplication of unique certificate server B for each server A?

In order to give a scale, there can be >= 10 servers A and >= 30 servers B.

Thanks!

There is more than one way to do this, but if it were me doing it, this is the way I'd choose:
Servergroup A
These servers should have a server certificate, not a client one. Since they will all be accessed using the same hostname, they should use the same server certificate. This certificate must be issued by a CA that is trusted by the servers in servergroup B - this can either be a generally trusted CA, such as letsencrypt.org or Digicert, or it can be a local CA that you set up (in which case you need to distribute the root CA cert to the B servers).
Servergroup B
These servers should use client certificates. Here, I'd use one certificate per server, instead of having all of them share one. And I'd use a local CA, internal only to my own organisation, to issue them. I'd set up the A servers to allow access by any client that uses a certificate issued by my internal CA.
The reason for not using a public CA is that it's a lot easier to grant access to all certificates issued by a CA, instead of checking to see if the certificate contains e.g. the correct organization, or the correct hostname. (Obviously you also want to check that the certificate hasn't been revoked, so your CA needs to publish certificate revocation lists, or maybe even allow OCSP checks.)
I would also recommend using a configuration management system, such as puppet or ansible, to automatically enroll the B servers to the internal CA when they are installed, and to renew the certificates as necessary. There are a number of protocols that can be used for this; I'd recommend either CMP or EST. (EST is newer; it's specified in RFC 7030 and you can test it at http://testrfc7030.com).
Setting up an internal CA
If the internal CA is only used to create these client certificates, you can use pretty much any CA software you like and are confident that you can manage.

OpenSSL is available for all operating systems, but it's not very easy to use, though with some scripting it's very versatile.
If it were me setting it up, I'd use EJBCA, because that's the software I know best and it's fairly easy to use. It's open source and you can download a pre-installed virtual machine to test it out. (Note that this pre-installed machine should not be used in production without significant work done first; it's not been hardened and every virtual machine has the same admin key!)
If you're running on Windows, then Microsoft Active Directory Certificate Services is probably your best bet; it's integrated from the start.
If you're running RedHat, maybe look at RedHat Certificate System.


Information on the phpMyAdmin probe? Various IP addresses call various domains looking for various paths that might lead to PHP My Admin/scripts/setup.php

One of those IP addresses is 74.63.212.48 which I just now found out I could navigate to shows a page in Chinese titled

  Rinchen technology overseas VPS - PHP probe

A very effective translation can be achieved using Google Chrome. It will translate the live results as they come in. It is also available via domain name 8242.cn. However, although it is Chinese, it seems to be hosted within the US, and around the world. (assuming it is all the same thing)

Generally, it seems to list status information, PHP build, processor, ram, etc. linux version, uptime, clock, information on the features enabled for PHP, database. It also provides

  MySQL database connection detection

which asks for address, port, user, pass and has a "MySQL Test" button

  Function testing
  Enter the function you want to test:

and also

  Mail detection
  Please enter your email address to test:

I find this program very annoying since it floods our logs with high amounts of traffic that I consider it an attack. Actually, we don't run PHP, so I am not much concerned, but we have blocked several of these IPs anyway, especially the ones that are overseas and hardly likely to serve legitimate users.

Does anybody know who runs these systems? Somebody must be putting considerable effort into getting a list of all instances of PHP MyAdmin setup.php. I was hoping there would be more information available from you guys. This is very annoying that the program keeps popping up on new IPs. All I know so far is it is Chinese....

probes for phpmyadmin setup scripts are really just a fact of life at the moment if you run a web facing web server.  I wouldn't assign too much significance to the locale that they come from .

As far as I'm aware it's just pretty much blind scanning looking for known vulnerable software.  I get it on a regular basis on my web server even though I don't (and never have) run phpmyadmin on it.

After ensuring that you're not running it on the host(s) in question I'd recommend just filtering it out of the alerts that you're getting.


How to determine start and end bytes of LUKS header? How can I determine the exact start byte and exact end byte of a LUKS header on a block storage device?

I use Linux Unified Key Setup (LUKS) for Full Disk Encryption (FDE), so all of the data on my drive is encrypted using a strong master key that's not derived from my passpharse, and I'm working on a script that will securely wipe the drive in an panic/emergency shutdown situation (ie: someone is physically stealing your laptop).

Let's say I have a 1000T drive and time is too short (~30 seconds) in my emergency scenario to actually fill the drive with random bytes. Instead, I'd just like to overwrite the header because all of the data on the drive is worthless if the header (containing the keyslots with the master key and salts) is lost--even if the passphrase were recovered via rubber-hose cryptanalysis.

How can I safely determine the start byte and end byte of the LUKS header so I know what to overwrite?

  Note: The solution provided must be valid for both LUKS1
  (released in 2014) and LUKS2 (released in 2018).

Edit: I know I can just overwrite the first 10 MiB and be sure to get the entire LUKS header, but I'd also like the ability to restore the header in the future in-case the emergency shutdown was accidental (triggered by a false-positive). So knowing the exact start & end bytes of the LUKS header are critical to avoid the risk of data corruption if restoring the LUKS header.

LUKS Emergency Erase

To solve your problem, the best way to wipe a LUKS header is to use the luksErase command. To quote the man page:'

luksErase <device>

       Erase all keyslots and make the LUKS container permanently inaccessible.
       You do not need to provide any password for this operation.

       WARNING: This operation is irreversible.

LUKS Header Start Byte

But to answer your question, a LUKS device's header necessarily starts at byte zero.

This is distinct from a LUKS container, which could start at a non-zero offset. In any case, the LUKS container's start byte would be found by searching for the byte sequence 0x4c55 0x4b53 0xbabe, where (encoded as ASCII):

0x4c = 'L'
0x55 = 'U'
0x4b = 'K'
0x53 = 'S'

LUKS Header End Byte

The end byte is trickier, and to get it we must parse the LUKS headers.

First we parse the LUKS binary header to determine the version. Currently, there are only two versions of LUKS. LUKS1 was released in 2014 and LUKS2 was released in 2018. The encoding for both is very different, but the first 8 bytes of the LUKS header are encoded the same:

+-------+--------+-------------+
| Start | Length | Field Name  |
+-------+--------+-------------+
|     0 |      6 | magic       |
|     6 |      2 | version     |
+-------+--------+-------------+

LUKS1

For LUKS1, the easiest way to determine where the LUKS header ends is to determine where the encrypted data (aka the "payload bulk data") begins, since the encrypted data begins immediately after the LUKS header ends.

To get the start offset of the payload, we need to get the value of payload-offset. From "Figure 1: PHDR layout" of LUKS1 On-Disk Format Specification, the payload-offset field starts in the LUKS header at an offset of 104 bytes and is 4 bytes long.

For example, the following hex dump of a LUKS container shows the version is 00 01 (LUKS1) and the payload-offset is 0x1000 (hex) = 4096 (decimal).

root@disp4117:~# hexdump -Cs 6 -n 2 luksVol1
00000006  00 01                                             |..|
00000008
root@disp4117:~# hexdump -Cs 104 -n 4 luksVol1
00000068  00 00 10 00                                       |....|
0000006c
root@disp4117:~#

The payload-offset just defines the number of sectors (in 512 bytes sectors) after the LUKS header start where the encrypted data begins (note that the location where the encrypted data begins is exactly the same as the location where the LUKS header ends), so in order to get the bytes where the payload begins, we have to multiply payload-offset by 512. 4096 * 512 = 2097152

Therefore, for the example above, the LUKS1 header ends at byte 2097152.

LUKS2

For LUKS2, there is no payload-offset field in the binary header. Rather than iterating through all of the keyslot objects, the closet equivalent to the payload-offset is the offset attribute for the first segment object defined in the JSON metadata (this is new to LUKS2).

But before we can parse the JSON object, we have to find it! In LUKS2, there's six distinct header areas:

  \/ primary binary header          alignment padding \/
+----+----------+----+----------+-------------------+----+
| /\ | 1st JSON | \/ | 2nd JSON |   Keyslots area   | /\ |
+----+----------+----+----------+-------------------+----+
                  /\ secondary binary header

As you can see above, LUKS2 actually has two copies of the plaintext metadata. It's stored in the "primary binary header" and the "1st JSON" area (the second copy is called the "secondary binary header" and "2nd JSON" area, respectfully). In most cases the data in the "primary binary header" exactly matches the "secondary binary header" and the data in the "1st JSON" area exactly matches the "2nd JSON" area. The data is stored twice to aid in recovery and protect against corruption.

The "1st JSON" area always starts at byte 4096 (immediately after the "primary binary header" area), but its length is variable. The length can be determined by looking at the binary header's hdr_size. From "Figure 2: LUKS2 binary header on-disk structure" in LUKS2 On-Disk Format Specification, hdr_size is 8 bytes long and starts at byte 8 in the binary header.

For example, the following hex dump of a LUKS container shows the version is 00 02 (LUKS2) and the hdr_size is 0x4000 (hex) = 16384 (decimal).

root@disp4117:~# hexdump -Cs 6 -n 2 luksVol2
00000006  00 02                                             |..|
00000008
root@disp4117:~# hexdump -Cs 8 -n 8 luksVol2
00000008  00 00 00 00 00 00 40 00                           |......@.|
00000010
root@disp4117:~#

From the above hex dump, we clearly see the hdr_size field has a value of 0x4000 (hex) = 16384 (decimal). But the hdr_size field defines the size of the the "primary binary header" plus the "1st JSON" area. So the length of the "1st JSON" area is the hdr_size value minus the "primary binary header" length. In this case, that's 16384 - 4096 = 122881. Depending on the length of the LUKS2 container's metadata stored in the JSON object, the JSON area could be up to 4194304 - 4096 = 4190208 bytes long.

So here's a hexdump of the "1st JSON" area, starting at byte offset 4096 with length 122881:

root@disp4117:~# hexdump -Cs 4096 -n 12288 luksVol2
00001000  7b 22 6b 65 79 73 6c 6f  74 73 22 3a 7b 22 30 22  |{"keyslots":{"0"|
00001010  3a 7b 22 74 79 70 65 22  3a 22 6c 75 6b 73 32 22  |:{"type":"luks2"|
00001020  2c 22 6b 65 79 5f 73 69  7a 65 22 3a 36 34 2c 22  |,"key_size":64,"|
00001030  61 66 22 3a 7b 22 74 79  70 65 22 3a 22 6c 75 6b  |af":{"type":"luk|
00001040  73 31 22 2c 22 73 74 72  69 70 65 73 22 3a 34 30  |s1","stripes":40|
00001050  30 30 2c 22 68 61 73 68  22 3a 22 73 68 61 32 35  |00,"hash":"sha25|
00001060  36 22 7d 2c 22 61 72 65  61 22 3a 7b 22 74 79 70  |6"},"area":{"typ|
00001070  65 22 3a 22 72 61 77 22  2c 22 6f 66 66 73 65 74  |e":"raw","offset|
00001080  22 3a 22 33 32 37 36 38  22 2c 22 73 69 7a 65 22  |":"32768","size"|
00001090  3a 22 32 35 38 30 34 38  22 2c 22 65 6e 63 72 79  |:"258048","encry|
000010a0  70 74 69 6f 6e 22 3a 22  61 65 73 2d 78 74 73 2d  |ption":"aes-xts-|
000010b0  70 6c 61 69 6e 36 34 22  2c 22 6b 65 79 5f 73 69  |plain64","key_si|
000010c0  7a 65 22 3a 36 34 7d 2c  22 6b 64 66 22 3a 7b 22  |ze":64},"kdf":{"|
000010d0  74 79 70 65 22 3a 22 61  72 67 6f 6e 32 69 22 2c  |type":"argon2i",|
000010e0  22 74 69 6d 65 22 3a 34  2c 22 6d 65 6d 6f 72 79  |"time":4,"memory|
000010f0  22 3a 32 37 34 35 33 30  2c 22 63 70 75 73 22 3a  |":274530,"cpus":|
00001100  32 2c 22 73 61 6c 74 22  3a 22 71 4a 6e 79 2b 4a  |2,"salt":"qJny+J|
00001110  5c 2f 6f 35 71 77 57 77  35 78 2b 57 31 30 7a 47  |\/o5qwWw5x+W10zG|
00001120  59 54 6f 64 44 64 57 6f  39 6e 74 5c 2f 6c 67 49  |YTodDdWo9nt\/lgI|
00001130  41 61 61 6f 78 5c 2f 45  3d 22 7d 7d 7d 2c 22 74  |Aaaox\/E="}}},"t|
00001140  6f 6b 65 6e 73 22 3a 7b  7d 2c 22 73 65 67 6d 65  |okens":{},"segme|
00001150  6e 74 73 22 3a 7b 22 30  22 3a 7b 22 74 79 70 65  |nts":{"0":{"type|
00001160  22 3a 22 63 72 79 70 74  22 2c 22 6f 66 66 73 65  |":"crypt","offse|
00001170  74 22 3a 22 31 36 37 37  37 32 31 36 22 2c 22 69  |t":"16777216","i|
00001180  76 5f 74 77 65 61 6b 22  3a 22 30 22 2c 22 73 69  |v_tweak":"0","si|
00001190  7a 65 22 3a 22 64 79 6e  61 6d 69 63 22 2c 22 65  |ze":"dynamic","e|
000011a0  6e 63 72 79 70 74 69 6f  6e 22 3a 22 61 65 73 2d  |ncryption":"aes-|
000011b0  78 74 73 2d 70 6c 61 69  6e 36 34 22 2c 22 73 65  |xts-plain64","se|
000011c0  63 74 6f 72 5f 73 69 7a  65 22 3a 35 31 32 7d 7d  |ctor_size":512}}|
000011d0  2c 22 64 69 67 65 73 74  73 22 3a 7b 22 30 22 3a  |,"digests":{"0":|
000011e0  7b 22 74 79 70 65 22 3a  22 70 62 6b 64 66 32 22  |{"type":"pbkdf2"|
000011f0  2c 22 6b 65 79 73 6c 6f  74 73 22 3a 5b 22 30 22  |,"keyslots":["0"|
00001200  5d 2c 22 73 65 67 6d 65  6e 74 73 22 3a 5b 22 30  |],"segments":["0|
00001210  22 5d 2c 22 68 61 73 68  22 3a 22 73 68 61 32 35  |"],"hash":"sha25|
00001220  36 22 2c 22 69 74 65 72  61 74 69 6f 6e 73 22 3a  |6","iterations":|
00001230  36 31 39 34 33 2c 22 73  61 6c 74 22 3a 22 46 69  |61943,"salt":"Fi|
00001240  4c 67 31 35 56 5c 2f 55  56 4b 47 72 72 4e 39 4f  |Lg15V\/UVKGrrN9O|
00001250  52 2b 5c 2f 69 59 46 51  70 38 38 59 44 77 50 4c  |R+\/iYFQp88YDwPL|
00001260  6a 4f 6f 4c 70 6a 77 6d  78 58 77 3d 22 2c 22 64  |jOoLpjwmxXw=","d|
00001270  69 67 65 73 74 22 3a 22  49 70 34 31 5a 58 70 44  |igest":"Ip41ZXpD|
00001280  76 77 52 76 6d 41 73 33  30 58 69 72 6c 48 65 6d  |vwRvmAs30XirlHem|
00001290  57 72 44 67 6c 5c 2f 44  4a 31 36 79 33 31 41 71  |WrDgl\/DJ16y31Aq|
000012a0  66 42 55 6f 3d 22 7d 7d  2c 22 63 6f 6e 66 69 67  |fBUo="}},"config|
000012b0  22 3a 7b 22 6a 73 6f 6e  5f 73 69 7a 65 22 3a 22  |":{"json_size":"|
000012c0  31 32 32 38 38 22 2c 22  6b 65 79 73 6c 6f 74 73  |12288","keyslots|
000012d0  5f 73 69 7a 65 22 3a 22  31 36 37 34 34 34 34 38  |_size":"16744448|
000012e0  22 7d 7d 00 00 00 00 00  00 00 00 00 00 00 00 00  |"}}.............|
000012f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00004000
root@disp4117:~#

To get the offset of the first data segment we need to check the segments object in the above JSON. Here's the same segments section from above, but formatted for readability:

  "segments": {
    "0": {
      "type": "crypt",
      "offset": "16777216",
      "iv_tweak": "0",
      "size": "dynamic",
      "encryption": "aes-xts-plain64",
      "sector_size": 512
    }
  },

We can see above that the first data segment begins at offset = 16777216.

Therefore, for the example above, the LUKS2 header ends at byte 16777216.


Is it fine to store the key used to encrypt data, in encrypted form alongside the encrypted data? There is a need to encrypt data and upload somewhere on a periodic basis. We have a master key that is used to generate data keys. On each instance of upload, a new data key is generated and the data is encrypted using the key and the key is encrypted using the master key.

Is it fine to store the encrypted data and the corresponding key (also encrypted) alongside? This will obviate the need to store the mapping between the upload and the corresponding key used for encryption, again in a data store. But if this is not a good practice, it is better avoided. Any thoughts?

There's nothing wrong with this model, so far as your question goes. Lots of systems store an encrypted key alongside the ciphertext (examples include most disk encryption tools and many file or email encryption tools, such as PGP).

Whether it's a good idea to do things this way depends on what you're trying to achieve.

If you need to make it possible to rotate the master key without decrypting and re-encrypting everything, this approach makes sense.
If you're just trying to ensure that each message is encrypted independently, so that two messages of the same content will have different ciphertexts, that's what Initialization Vectors (IVs) / Nonces are for, and you don't need separate keys at all (the IV/nonce is usually stored, in plain text, next to the ciphertext; it's not a secret at all you just need to not ever reuse it with a given key).
If you're trying to store data on behalf of another party (for example, a message you encrypted for the recipient), your scheme still works but you should be encrypting the per-message keys with the recipient's public key, rather than with a shared symmetric key (this is how GPG and S/MIME typically work).

Bear in mind that you'll still need integrity protection on that data. Encryption, by itself, does not prevent tampering with the data, regardless of how well the key is protected. Use a message authentication code of some sort (HMAC, a keyed hash construction, is standard) or use authenticated encryption (Galois/Counter Mode, or GCM, is standard) and store the integrity/authentication data alongside the ciphertext.


Suitable cipher for SMS encryption I would like to create application for encrypting SMS messages. After long enough discussions I decided to use symmetric encryption, since handling public/private keys issue is too difficult for most of users.

So I suppose that secret key has to be transferred over voice interaction between parties or other method.

My question is: what will be suitable algorithm for encrypting SMS? Cipher must consider  max length of message 160 chars and accepting only printable symbols. Message chunking and all kind of around issues - are unacceptable. I know that some phones can automatically merge chunked messages, but keeping in mind great diversity of makes/models - it's unworkable.

Any thoughts, advices?

I would urge you to revisit your premise.  If both endpoints will have an app on it, then your criteria for choice of algorithm are not valid: the app will be handling the encryption/decryption, not the user.  Public-key crypto can potentially provide better security in this context; for instance, it may enable trust-on-first-use (SSH-like) key management.  Also, I'd suggest that you start by first thinking about the threat model.  The first question should not be, what crypto algorithm do I use?  Your first question should be, What threats am I trying to defend against?

If you must use symmetric encryption for SMS texts, AES in CBC mode with ciphertext stealing is a plausible choice.  Ciphertext stealing will avoid waste if the message to be encrypted isn't an even multiple of 16 bytes.  You will still need an IV, which will take up some space.  To save space, you could send a counter, and use the AES encryption of the counter as your IV.

If there's any way you can avoid transporting ciphertexts over SMS, avoid it.  160 characters is very limiting.  Can you transport messages over the Internet?

If you can avoid tunneling over SMS, you might explore a model where the app has its own private/public key, and maintains a public key for each contact.  When contacting someone new, the app sends its own public key, receives a public key from the other person, remembers that public key, and sends the message encrypted with that public key.  This is of course vulnerable to man-in-the-middle attacks but may be easier to use.


Windows file to look for to test LFI For Linux, if we want to test local file inclusion we always search for /etc/passwd
For old versions of Windows,  (e.g. Windows XP) we search for win.ini
What file should I search for in later versions of Windows that is readable and always there?

C:\Windows\System32\drivers\etc\hosts is pretty commonly used to check for read access to the file system while pentesting. If I remember correctly, this file exists on Windows 7 and later, but I'm not sure if it exists on earlier versions.


Should I log that a user changed their password? Are there any security concerns with logging that a user changed their password? I'm already logging whenever an admin changes a users password for audit purposes, but is there a reason to not have a log of when each user changed their own password?

Edit: Answers to questions below

  What is your expected gain from this?

Mainly forensic. The ability to see who (user_id of admin or self) changed the password in case the user claims they were hacked.

We are not using this to support any password managing schemes such as forced password change or disallowing password reuse.

  Who would have access to these logs?

System administrators and possibly a small support team.

  About what kind of accounts are we talking about?

User accounts on an e-learning platform i.e. teachers and students.

  Also, do you have password expiration rules?

No. I'm talking about storing when every change of password was done, not only the last change.

  What info are you storing? (not actually asked but implied)

We are storing the user id of the user whos password was changed, the user id of the user doing the change (it could be an admin or a students teacher), the time the password was changed and the URI used to change the password.

To, answer your question, Yes, you can and SHOULD log password-changes, and there's nothing fundamentally wrong with doing so, as long as you don't e.g. record the password itself"

What to log?

When designing logging for Security purposes you want to address these questions:

When did the event happen?

The date and time the event occurred (Use the common log format)

What was the event?

A short description of the event (e.g Password Change)

Who triggered the event?

The user id, name, email or some unique identifier

Why was the event triggered?

This is not the same as the "What" even though many people use it that way. This is the reason the event was executed. (e.g. Password changed due to policy, User manually changed password, etc). This can be really good for weeding out noise.

Scenarios

One of the best methods for discussing what to see is via scenarios and asking the team:

What information does the event provide?
Is the event required for compliance / legal?
Are we logging for detective reason? (e.g. Triggering a SIEM) or for corrective? (e.g. Forensics after the fact)
Who will be looking at the logs?
How will we protect the logs?

Example:

James is part of the IR team, which is responsible for Made-Up Company's critical application 'Non-Existence'. James want to be able to see all password changes in order to detect changes that occur outside the normal policy process. These events will trigger and investigation if a password change happens without an incident logged by the support team. Logs will be sent to the IR SIEM appliance which will use a rule set to trigger warnings to the IR team when an incident cannot be correlated to a password change outside of a required policy change.

(Obligatorily caution for using this at a workplace. I just made this example up.)

[edit]
- Updated the initial answer to be more clear. Thanks to @SeldomNeedy for the suggestion.


Cryptographic hash function vs. non-cryptographic hash function examples and compare I would love to understand at the most basic level what their differences are.
When is each one used?

Any advantages and disadvantages?

An example of a cryptographic hash function is SHA256. An example of a non-cryptographic hash function is CRC32.

A cryptographic hash could be used, for example, for a cryptographically strong Message Authentication Code (MAC, or, in practice, as a component of HMAC for key-based integrity) or as a component in an authentication encryption (AE or AEAD) scheme. Here, the meaning of "cryptographically strong" is that the hash function satisfies certain properties such as, for example, the hash value (the output of the hash function) doesn't have any collisions that, for example, could be exploited by an attacker. For more information, the book "Introduction to Modern Cryptography" by Katz and Lindell discusses properties of cryptographic hash functions.

A non-cryptographic hash could be used, for example, as a checksum for a small amount of data if you are not worried about an attacker trying to modify it. E.g., some data stored in an EEPROM only accessed by other trusted components on a circuitboard.


Can a hypervisor rootkit enable hardware-assisted virtualization when it has been disabled by the BIOS? I was recently reading An Introduction to Hardware-Assisted Virtual Machine (HVM) Rootkits (PDF), and it wasn't entirely clear about the conditions under which the hypervisor rootkit can start virtualization.  If virtualization is enabled then everything is fine, but what happens if virtualization is disabled?

Specifically, does either Intel VT-x or AMD-v (VMX and SVM) support unlocking and enabling HAV after the BIOS (or a later hypervisor) has disabled and locked it?  I am not interested in modifying the BIOS or changing its settings, purely in whether a hypervisor can change these settings.

(If anyone is wondering: No, I am not planning on creating a malicious rootkit :P)

To quote the Intel Software Manual, Volume 3c:

  VMXON is also controlled by the IA32_FEATURE_CONTROL MSR (MSR address 3AH).
  This MSR is cleared to zero when a logical processor is reset. The relevant bits of the
  MSR are:

  Bit 0 is the lock bit. If this bit is clear, VMXON causes a general-protection exception. If the lock bit is set, WRMSR to this MSR causes a general-protection exception; the MSR cannot be modified until a power-up reset condition. System BIOS can use this bit to provide a setup option for BIOS to disable support for VMX. To enable VMX support in a platform, BIOS must set bit 1, bit 2, or both(see below), as well as the lock bit.
  Bit 1 enables VMXON in SMX operation. If this bit is clear, execution of
  VMXON in SMX operation causes a general-protection exception. Attempts to set
  this bit on logical processors that do not support both VMX operation (see Section 23.6) and SMX operation (see Chapter 6, “Safer Mode Extensions Reference,” in Intel® 64 and IA-32 Architectures Software Developer’s Manual, Volume 2B) cause general-protection exceptions.
  Bit 2 enables VMXON outside SMX operation. If this bit is clear, execution of    VMXON outside SMX operation causes a general-protection exception. Attempts to set this bit on logical processors that do not support VMX operation (see Section 23.6) cause general-protection exceptions.

Interpretation, for those who aren't au fait with what's going on here:

MSR stands for machine specific register and refers to registers / configuration settings that alter between implementations of the x86 architecture.
Some of these are used to configure features for that family of processor.
VMXON is the (Intel) x86 instruction to enter hypervisor management mode - the point where you get access to hardware assisted virtualisation features including extended page tables, etc. Here, you create virtual machines and execute vmenter and vmexit and so on.
A general protection exception is an exception handler built into the processor that occurs when you do something you're not allowed to do. Examples would include executing privileged instructions outside of a privileged level, e.g. the sidt instruction as a user mode application (ring 3).

As you can see from the documentation, BIOSes have a choice to enable or disable VMX and also may use the lock bit; if the lock bit is set any attempt to write to alter the configuration will cause a GP exception.

So it really depends how those bits are set by your BIOS, in answer to your question. If the BIOS sets all of the bits to 000, then software ought to be able to enable the hypervisor via WRMSR - see the discussion here. I think WRMSR is a privileged instruction, meaning you'd need to do it from the kernel. However, once the lock bit is set, you're required to reboot - essentially the circuitry will check the state of the reset flag, then throw an exception at your operating system - which can either carry on as normal or crash, depending on whether it understands the exception (most likely not).


How do you practically manage your never-use-personal-accounts-on-a-work-machine-ever double life, when your personal machine is a limited iOS device? This is follow-up to

Starting a new job after working at home for years. How to best protect data, secrets, and privacy in the workplace?

The answer to that question is

Follow company security protocol.
Never use a personal account on a work machine, ever...which means only on a personal device, and only over a cellular connection.

Thoughts:

Bring a personal, cellular-only laptop to work (not necessarily safe, practical, or prudent)
Use a non-sensitive Dropbox account to transfer large files
So now I'll always have two phones in my pocket  :(
What if you work for a cellular phone company? Is a contract with another carrier good enough, or should you expect that you are compromised? Do large corporations have their own cell towers that they can intercept traffic from an arbitrary carrier?

How do you practically manage this double life, when your personal computer is a limited iOS device?

More and more people are using iOS devices as their only computers for several reasons. The popularity, expense, and maintenance costs have been outlandishly high for years now. My gf has gone through the iPhone 4 to 4S to 5 to 6 in the past 2.5 years -- and she does not own any other computers. She can't really afford them and always has a broken screen she is trying to save up money to replace. This is the new normal.

Can you use an iOS device for all personal computer needs? Certainly you can, and depending on your pocketbook, you can pick up accessories such as the ClamCase or apps described in books such as #iPadOnly. The first real post-PC book. How to use only your iPad to work, play and everything in between.

Can you secure a personal iOS device against nearly every drive-by threat? Yes, and this is very easy to accomplish if you have a few strong controls (e.g., 12-character-length mixed-case mixed-character password, never use public WiFi, leave Bluetooth and NFC off, never install unofficial apps, etc). Some of these have been described in the NSA hardening guides or via the Center for Internet Security benchmarks.

For protection against carrier networks, I agree that we put too much trust in them. SIM cards, just like USB and SDcards, have been known to contain spyware such as NSA implants (many countries can perform the equivalent). Some devices shipped with rootkit-like technology such as found in the CarrierIQ fiasco. Many cell towers utilize stingray technology (i.e., IMSI catchers), especially near malls, airports, and government buildings. In these areas, you will also find law enforcement use of automatic license plate recognition, facial recognition, and many other privacy-busting technologies. Perhaps these are only used to combat terrorist activity or child kidnappings. It is a bit presumptuous to think that these national security or localized law-enforcement technologies will ever be used against you personally, however, so most people tend to fight their overuse by way of the ACLU, the EFF, or other legitimate channels.

The technology that I use is simple: a data-only SIM card via the Straight Talk BYOT program. You call Straight Talk, order the SIM, and plug it into your GSM (preferably AT&T-compatible with LTE) device. If you are very concerned about your data, or have a specific need for your most-secure data, such as intellectual property (e.g., say you are working on a patent), then you might want to avoid cloud, virtual, and outsourced services. For example, you could setup a private infrastructure (e.g., ownCloud) and pay for insurance (techinsurance.com for the baseline, plus cyber insurance) to cover residual risk.


Why don't we use hash based digital signatures everywhere? If it is known that existing digital signature schemes like elliptic curve crypto or RSA are vulnerable to quantum computers, why haven't we already switched to Lamport signatures or something like that? Is there something that prevents people from deploying this type of signature right now?

I understand that we still believe/know that nobody has a good working quantum computer yet but why not make the switch anyway?

If it is known that existing digital signature schemes like elliptic curve crypto or RSA are vulnerable to quantum computers, why haven't we already switched to Lamport signatures or something like that?

Quantum computers were not a realistic threat when these algorithms became standardized and widely deployed. As indicated in the comments by @Steffen Ullrich, hash-based signatures carry some disadvantages. The size of the public/private keys and the size of the signatures can be a non-negligible downside if quantum computers aren't relevant.
Additionally, some hash-based signature schemes are stateful. Stateless schemes carry a big advantage over stateful schemes in regards to implementation simplicity.

Is there something that prevents people from deploying this type of signature right now?
I understand that we still believe/know that nobody has a good working quantum computer yet but why not make the switch anyway?

For a cryptographic algorithm to be adopted for use in the real world, it needs to be standardized.
Hash-based signature have been submitted to NISTs call for post-quantum algorithms.. It is entirely possible that hash-based signatures will become standardized and recommended for use - assuming that they survive the analysis. The deadline for submissions was just closed (November 30, 2017) and it will probably be a few years before anything is standardized.


Password protect an excel file for a period of time? I'm trying to find a way to password protect an Excel file (.xlsm) beyond the built-in password encryption options.

Ideally:

Enter the password
File works for a week(/other period of time)
After a week, file requires password again

The file is used consistently, so I don't think that encrypting a .zip or similar would solve my problem.

There are VBA options to do this, but they can be easily circumvented by restricting macro security or similar.

Help! Thanks!

A file is just a collection of bytes. It cannot do something to itself on its own.

You need an access restricting application, or use what is available.

Because it is an excel document, any weird solution you come up with can be circumvented just by saving it locally unprotected or copy pasting.

But if you still want to at least try, even if it isn't perfect, try using access controls through sharepoint. You can hand out temporary access or manually turn it off for someone/everyone after a certain period of time.


Intentionally slow decryption We need to store the user ID of authors of messages in a database.

It should be possible to find an author of a problematic message i.e. on police request in minutes, but it should be very hard to find all the messages of an author (because we have no legal obligation (yet) to do this and we want to save our users privacy). So we need an encryption which allows us to decrypt but needs a lot of time to do so.

Therefore I was considering to encrypt the associated user ID of the message with the message as key - so that an attacker may need a lot of CPU time to decrypt the user IDs of many many millions of messages to create an index.
This would be even harder if we could implement a really time consuming decryption method. Is there such?

Background:
The Austrian government will introduce a law which will make all providers of public forums, news site's comments, chats, social media, etc. obliged to have all their user registered by validating their real ID with a document like a passport. The police and intelligence agencies will be allowed to access this (to find the author of a message) - "to fight online harassment".
This makes us fear, that once all this data is collected, the police and intelligence agencies will demand access the whole data - but the other way around. Not to fight harassment by one message but to get all contributions a user did in the last years on all media platforms all together. To get an exact political profile of everyone (history teaches us this is always one of the first steps in applied fascism).

What you need is a slow KDF which processes a stored or memorized password and converts it into a key for decryption. The actual decryption process can proceed rapidly, as long as the key-derivation takes a long time. There are a number of algorithms that can do this, such as PBKDF2, Argon2, and scrypt. If each stored record contains its own password that must be first put through a strong KDF before the record can be decrypted, then you will be able to enforce a configurable delay to access.

I strongly suggest, however, that you contact a lawyer rather than trying to implement a technical solution yourself. There may be better legal alternatives that would allow you to get away with not storing any data. Never assume that your own interpretation of the law is the only possible interpretation.


Securely transmit sensitive data I am creating a java application which is dependent on  a server. On startup the application will transmit sensitive information (passwords and emails) which will be sent to the server and from there encrypted and never be touched again.

My question is how can I secure the client>server transmission from MITM attacks or other attacks that can make the attacker able to read the transmitted data?

You can use a protocol designed for exactly this: SSL/TLS.

As usual, don't roll your own cryptography. Find an existing implementation that takes care of the heavy lifting for you. For Java, that will probably be the Java Secure Socket Extension (JSSE).


What type of PCI Compliance do I need if I don't hold any card data I am working with a company that is "PCI Compliant" and in order for them to give me any of their data (customer name, email) they ask that I be "PCI Compliant" even though this data is not credit card related. Very new to PCI but in reading I guess I fit in the SAQ D but not sure.

Edit: This is a spreadsheet with 2 columns (name, email) uploaded to me by a web form that's run on port 80

Edit2: Even if you are not doing any processing can you still say that your environment meets PCI DSS 2.0? Like AWS says that their EC2 environment has been validated by a QSA?

This is a bit of a weird request.  Only the Primary Account Number (PAN), Cardholder Name, Expiration Date and Service code are Cardholder Data.

Directly from PCI-DSS under the header PCI DSS Applicability Information

  The primary account number is the defining factor in the applicability of PCI DSS requirements.  PCI DSS requirements are applicable if a primary account number (PAN) is stored, processed, or transmitted.  If PAN is not stored, processed, or transmitted, PCI DSS requirements do not apply.

Thus, if you are not storing, processing or transmitting the Primary Account Number, then you are automatically PCI DSS compliant as no requirements apply to you.


How can illegal downloads (not file sharing) be detected? I understand that copyright holders can identify illegal filesharing over BitTorrent by simply taking part in the sharing process and noting IPs of peers.

But how do they detect illegal downloads from file hosters?

If you use a public hosting service, that service could of course also keep track of what it's hosting and who uploaded and downloaded it.

If the filehost is not a honeypot, if you use an unsecured or semisecured connection with a known protocol to transfer your files, the ISP could track whenever you initiate a file transfer. If the protocol works with an open checksum, the ISP could take that checksum and compare it to one of its databases. If they find a match, they have strong evidence that something that wasn't supposed to be shared was shared.

Now this could also work for encrypted file sharing systems such as the one hosted by MEGA, if the ISP got a hold of the keys and inspected the file, deemed it illegal, and then started tracking whoever is downloading it. (this is not hard considering most people leave the keys in the link - the ISP could backtrace your steps and inspect the pages for the hashtag keys, or defer that task to a crawling service (would probably be faster))

Alternatively, a shill (cop, digital rights holder, etc) could transfer you the files or request that you transfer the files and then rat you out.

Beyond that, a spying agency, a bored hacker or anyone being able to install software on your device (e.g. the company you have your operating system from) could have installed spyware on your computer to track what you are doing.

Finally, somebody could be looking over your shoulder.


Encryption-Laws in Iran According to this article Iran has criminalized the usage of encryption and VPNs.
According to this blog post  it is still possible to use SSH tunnels in Iran.

For me SSH is encryption and thus forbidden in the Iran, but I wonder how online banking, which requires SSL, works over there and if OTR (eg for pidgin) and PGP/GPG are forbidden, too.

The reason why I ask this is pretty simple, I know some people traveling through asia who upload their videos to my server via sftp. I have no ftp installed because of security concerns, sftp was fine for me and them.

I don't want to bring them in any danger because of using sftp and would rather istall ftp and open ports till they are in india (I heard pakistan handles it pretty much the same way).

To clarify: I do not want any way to circumvent the iranian regulations, just a clear information on how encryption laws are applied in iran and under which circumstances.

If you're just accepting files, opening FTP to help them comply with Iranian law is a good idea.

If your friends are taking videos and images which are even unintentionally slightly political in nature, then they should keep a low profile.  I can't fathom why they'd enter the country with computers and start sending encrypted videos outside the country if that was their intent.

http://www.nytimes.com/2012/01/26/world/middleeast/iran-steps-up-arrests-of-journalists-and-bloggers.html

I was travelling in Iran a couple years ago and I didn't dare bring a computer into the country.  They need to ensure that their computers are squeaky clean.  Material offensive to Islam in the eyes of a corrupt government official could be anything.  Concealing their encrypted volumes is not an option.  If they didn't back up their data before crossing the border... I don't know what to say.

Iran's a nice place, but like anywhere in the world, you are subject to their laws in their country.

Without a computer, I carried a small stack of SD cards, had a USB card reader with a read-only flag and sent stuff out from cybercafes.


Why can't you steal the synchroniser token to do CSRF? I'm reading up on CSRF and I came across synchroniser tokens. I don't understand why you can't do a CSRF to get the token to do a real CSRF.

Example: bank.com has a form like this at https://bank.com/transfer:

<form action="https://bank.com/do_transfer" method="post">
  <input type="hidden" name="synchroniser_token" value="j/DcoJ2VZvr7vdf8CHKsvjdlDbmiizaOb5B8DMALg6s=" >
  <input type="hidden" name="to"                 value="ciro">
  <input type="hidden" name="amount"             value="100">
  <button type="submit">Send 100$ to Ciro.</button>
</form>

This code adapted from https://stackoverflow.com/a/26895980/1512962

What would stop malicioussite.com from GETing https://bank.com/transfer with JavaScript on the client (using the session ID already stored in the cookie) and scrapping the page for the authenticity_token? Then, they could use that ID to do a real CSRF, essentially a CSRF to do another CSRF.

It'd require JavaScript to execute, so it'd be harder to do then a simple 0px X 0px image, but it's more than doable.

What stops a malicious site from obtaining the anti-CSRF token is the Same Origin Policy. The Same Origin Policy, or SOP, is at the browser level, and defines where JavaScript is allowed to communicate.

JavaScript on example.com cannot call example.org to get data. Also, JavaScript on http://www.example.com/ cannot call http://www.example.com:8080/ as the port differs. There are further restrictions which are set and can be found here: https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy.

There are exceptions to this with Cross-Origin Resource Sharing, but that has its own rules that browsers have to follow. CORS can be found here: https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS.


Designing a Capture The Flag target and game management system I've attended several CTF contests and I'd like to implement a CTF system for my students. I asked Google about designing such system but it's fruitless. Any ideas?

As mentioned by this.josh, you can look at multiple already existing vulnerable applications in this question or you might try to find existing CTF source codes - for example, look at the OWASP hackademic challenges. You might just take one of these and modify them slightly for your students. Also, try to contact authors of past CTF's - they might help you by giving their source codes. Here's a handy calendar of various CTF challenges with appropriate links.


Why doesn't asn1parse parse all DER encoding strings? I have a file - crl.txt - that contains the following:

MDYwNKAyoDCGLmh0dHA6Ly93d3cyLnB1YmxpYy10cnVzdC5jb20vY3JsL2N0L2N0cm9vdC5jcmw=

It's from one of the extensions in a particular X.509 cert.  Why is it that when I run the following command nothing happens?:

openssl asn1parse -inform PEM -in crl.txt -i

Here's the hex encoding of that string:

00000000  30:36:30:34:a0:32:a0:30:86:2e:68:74:74:70:3a:2f  0604.2.0..http:/
00000010  2f:77:77:77:32:2e:70:75:62:6c:69:63:2d:74:72:75  /www2.public-tru
00000020  73:74:2e:63:6f:6d:2f:63:72:6c:2f:63:74:2f:63:74  st.com/crl/ct/ct
00000030  72:6f:6f:74:2e:63:72:6c                          root.crl

The DER decoding (unless I'm mistaken) is as follows:

SEQUENCE {
  SEQUENCE {
    [0] {
      [0] {
        [6] 'http://www2.public-trust.com/crl/ct/ctroot.crl'
        }
      }
    }
  }

It's the id-ce-cRLDistributionPoints extension.  The first [0] is for distributionPoint, the second [0] is for fullName and the [6] is for uniformResourceIdentifier (an implicitly encoded IA5String).

With OpenSSL 0.9.8r, I get this:

$ cat crl.txt
MDYwNKAyoDCGLmh0dHA6Ly93d3cyLnB1YmxpYy10cnVzdC5jb20vY3JsL2N0L2N0cm9vdC5jcmw=
$ /usr/bin/openssl asn1parse -inform PEM -in crl.txt -i
    0:d=0  hl=2 l=  54 cons: SEQUENCE
    2:d=1  hl=2 l=  52 cons:  SEQUENCE
    4:d=2  hl=2 l=  50 cons:   cont [ 0 ]
    6:d=3  hl=2 l=  48 cons:    cont [ 0 ]
    8:d=4  hl=2 l=  46 prim:     cont [ 6 ]

If your OpenSSL does "nothing" then there is something wrong with it or with your computer.

Note that "PEM", stricto sensu, is for Base64 with header and footer: it shall begin with -----BEGIN FOO----- (from some "FOO") and end with -----END FOO-----. OpenSSL is quite urbane to accept processing your headerless Base64 (or maybe it does not, in your case ?).


Minimum access required to access LSA Secrets I would like to know what the minimum access is to be able to dump from the lsass process on a Windows machine (any version) to try and grab LSA Secrets.

The reason I am asking is because I had thought it was restricted to admin and higher, but accross multiple penetration tests and different versions of windows, I have in the past been able to obtain LSA Secrets without getting local admin first, sometimes while logged in via RDP.

Now, is this because my user had admin access and I didn't realize it? Is that the only possibility? Or are earlier versions of Windows less restricted in allowing access to LSA Secrets?

You can dump lassas direcly from task manager or use procdump,in this case you are a local administrator on machine but you have not full admin token,i.e task manager is not running as administrator on windows 7 and you did not invoke UAC (consent.exe),then you can use mimikatz to grab LSA secrets.Use this link it is useful https://forums.hak5.org/index.php?/topic/29657-payload-ducky-script-using-mimikatz-to-dump-passwords-from-memory/


Timing attack - Is safe to check if strings have the same length? In Node, you can use crypto.timingSafeEqual() to check if two strings are equal in a timing-attack safe way. But, they must have the same length, so you have to do something like that:

return stringOne.length === stringTwo.length && crypto.timingSafeEqual(Buffer.from(stringOne), Buffer.from(stringTwo))

Is this approach safe? In my opinion, it's not, because it can leak the length of the string! Surprisingly, I discovered that hash_equals() of PHP works in a similar way:

// Source code from php-src/ext/hash/hash.c

if (Z_STRLEN_P(known_zval) != Z_STRLEN_P(user_zval)) {
        RETURN_FALSE;
}

known_str = Z_STRVAL_P(known_zval);
user_str = Z_STRVAL_P(user_zval);

/* This is security sensitive code. Do not optimize this for speed. */
for (j = 0; j < Z_STRLEN_P(known_zval); j++) {
    result |= known_str[j] ^ user_str[j];
}

Do you consider this approach safe?

You are correct this implementation of constant-time string comparison will  leak information about the length of some string that is being compared against an attacker controlled string.

However, if this is checking strings for authentication purposes, you never should be comparing raw strings.  You would first hash (preferably with a salted key-stretched hashing function like bcrypt or PBKDF2) the entered password with the salt taken from the database and compare against the hashed password from the database.

In such a scenario where a password login could potentially work, the lengths of the compared hashes will always be identical (regardless of what string the attacker enters).  Now, there may be scenarios where the lengths aren't identical; e.g., maybe you used to use sha256crypt but decided to migrate user hashes to bcrypt.  So you explicitly check whether the user-entered password works against either hash which have different lengths, and there's no vulnerability if different lengthed hashes fail quickly.  (Obviously it makes more sense to use an identifier to track the hashing algorithm you used).  Or you use client side code to hash first and then transmit the hash and the attacker tweaks the sent hash length.  If they send an incorrect length, there's no problem to fail quickly (it's not like the correct hash length is a secret).

You do have to be careful about writing code like the following python code:

def bad_constant_time_str_compare(str1, str2):
   ret_val = 0
   for c1, c2 in zip(str1, str2):
       ret_val |= ord(c1) ^ ord(c2)
   return ret_val == 0

which seems like it should work, until you recognize that bad_constant_time_str_compare('hello', 'h') returns True because of how zip works (stopping at the length of the shortest string).

You could fix this with say:

def better_constant_time_str_compare(str1, str2):
   ret_val = 0
   for c1, c2 in zip(str1, str2):
       ret_val |= ord(c1) ^ ord(c2)
   return ret_val == 0 and len(str1) == len(str2)

but if you think about it, it is still vulnerable to timing attacks leaking the length of the string.

Finally, you should recognize that constant time string comparison in principle leaks information about the length of the two strings being compared -- the running time will be proportional to the number of comparisons run.  You could try to mask that (e.g., add random wait time), but in principle any secure system will not be weakened in any practical way by knowing the length of the strings being compared.


how to get cookies from aspx site to use it with hydra I've been playing around with Hydra and .aspx site  and I've hit a bit of a snag - Hydra responds letting me know that the first 16 passwords in my password list are correct when none of them are.

Syntax :

hydra 192.168.88.196 -l admin -P /root/lower http-post-form "/mmm/index.aspx:username=^USER^&password=^PASS^&Login=Login:Username and/or password incorrect."

Output

Hydra v7.3 (c)2012 by van Hauser/THC & David Maciejak - for legal purposes only

Hydra (http://www.thc.org/thc-hydra) starting at 2015-05-05 22:30:51

[DATA] 16 tasks, 1 server, 815 login tries (l:1/p:815), ~50 tries per task

[DATA] attacking service http-get-form on port 80

[80][www-form] host: 192.168.88.196   login: admin   password: adrianna

[STATUS] attack finished for 192.168.88.196 (waiting for children to finish)

[80][www-form] host: 192.168.88.196   login: admin   password: adrian

[80][www-form] host: 192.168.88.196   login: admin   password: aerobics

[80][www-form] host: 192.168.88.196   login: admin   password: academic

[80][www-form] host: 192.168.88.196   login: admin   password: access

[80][www-form] host: 192.168.88.196   login: admin   password: abc

[80][www-form] host: 192.168.88.196   login: admin   password: admin

[80][www-form] host: 192.168.88.196   login: admin   password: academia

[80][www-form] host: 192.168.88.196   login: admin   password: albatross

[80][www-form] host: 192.168.88.196   login: admin   password: alex

[80][www-form] host: 192.168.88.196   login: admin   password: airplane

[80][www-form] host: 192.168.88.196   login: admin   password: albany

[80][www-form] host: 192.168.88.196   login: admin   password: ada

[80][www-form] host: 192.168.88.196   login: admin   password: aaa

[80][www-form] host: 192.168.88.196   login: admin   password: albert

[80][www-form] host: 192.168.88.196   login: admin   password: alexander

1 of 1 target successfuly completed, 16 valid passwords found
Hydra (http://www.thc.org/thc-hydra) finished at 2015-05-05 22:30:51

I have read that we can add cookies to hydra okay: "H=Cookie: security;low;PHPSESSID=<value for PHP SESSID cookie"

How do you get cookies from aspx?
How do you provide it to hydra?

Use a browser plugin such as Cookies Manager Plus on Firefox, then you can simply copy the value to the clipboard.

As you say, simply paste the value into the command line for Hydra.

See my answer here for the syntax that worked for me.

"/dvwa/vulnerabilities/brute/:username=^USER^&password=^PASS^&Login=Login:F=incorrect:H=Cookie: PHPSESSID=07b7ebb2faea96f8471ecdb759e68108; security=low"

So instead of PHPSESSID and the value you include the name of the ASP.NET session ID and value.


Building a Security Training Simulation environment My university is keen to build a security training simulation environment/platform and it will be a good learning experience for students. I Googled around to find out how it is built, but I only ended up with recommendations for a VirtualBox or VMWare lab environment been built using pwnOS, Metasploitable, etc...

What I'm looking for is to build an environment with a couple of vulnerable systems and network setup such that that students won't cross/interfere each other's work. For instance, the system could create a private simulation for each student to practice their class labs, I think some refer this as 'sandboxing'.

We are looking for something similar to Offensive Security training simulation or Hera Labs, another example is XNET from CERT, they have a very interesting training simulation for Forensics.

If you could give us some direction on how the system is actually built and if there any open source application that we could use to build the system, otherwise an organization who can build it for us (keep in mind this is university budget) we would really appreciate your kind input.

We are aware that Offensive Security offers this service but I need to get quotes from different vendors before can I proceed (at the end I prefer to build the system and might as well give it as project for the security lab students)

We are also aware it may require a couple of servers and OSs to build it, and we do have unused hardware.

Thank you and Happy New Year

Start with a network diagram based on how a corporate network would look like. For example the DMZ is going to contain the webservers and mailservers. Then there will be firewalls to protect the userland from the DMZ, also you are going to have domian controllers, database servers which are going to be part of a secure network. Hope you get an idea on what am trying to descibe. You can find a sample network diagram here on a for a pentest lab. You need not follow the same thing, but use it to design your own.

As for actually building the lab I would suggest using vSphere server which allows you to virtualize the operating systems and build a network. Check this blog on setting up a virtual environment.

Check out the following resources to help you in setting up a lab

http://ist.bk.psu.edu/cvclab/wp-content/uploads/2011/12/BuildingLab1.pdf

http://www.irongeek.com/i.php?page=videos/pen-testing-practice-in-a-box-how-to-assemble-a-virtual-network

http://www.jasonjfrank.com/wp-content/uploads/2012/10/Building-your-Own-Penetration-Testing-Lab-on-the.pptm


What kinds of attacks can VLAN's mitigate? Assuming these very basic kinds of attacks:

Header based
Protocol based
Authentication based
Traffic based

What would using a VLAN mitigate?

Several people seem to be overthinking the question. The attacks mentioned above are the 4 attack types in the network security attack taxonomy. I realize they are generic, but I was hoping for a moderately generic answer.

Traffic based. An example of an traffic based attack would be sniffing. By implementing VLANs, you seggragate traffic into different VLANs so that traffic that belongs to one VLAN is only visible in that VLAN. If there were no VLANs, an ARP spoofing attack could affect the whole switched network. In case there are VLANs, it would affect only the VLAN on which it is happening.


Unlock hard drive vs full-disk encryption When I turn my notebook on, it asks me for the password to unlock the BIOS and than it asks me for the password to unlock the hard drive.

How different is this 'password to unlock the hard drive' from the so-called full-disk encryption in terms of offered protection?

If someone could reset the BIOS (which is said to be easy even if password protection) given that this hard drive password was itself set in the BIOS, wouldn't the person have access to the drive? I'm not sure that this lock means encrypting.

Because if it works as Xander suggest then it makes the whole discussion about what software to use in full-disk encryption seem silly, since the solution is as trivial as setting a password in the BIOS.

I'm confused. :)

There are (typically) three types on on-boot authentication:

BIOS boot password
Drive locking mechanisms (e.g. HP DriveLock)
Full disk encryption (e.g. TrueCrypt / OpenPGP)

The BIOS boot password is simply a logical check inside the BIOS chip, which can be bypassed by flashing the BIOS manually or replacing the chip. It's a soft protection mechanism.

Full disk encryption is a proper security mechanism that involves actively encrypting the entire disk, and using a password or other authentication materials to decrypt the disk data on boot. This is typically performed via 3rd party software, but may also be integrated into the disk hardware.

Drive locking mechanisms are (usually) a proprietary feature of certain laptops, e.g. HP DriveLock. These involve encrypting the low sectors of the hard disk, i.e. the sectors that contain things like telemetry and bad sector mapping. When these are scrambled, a traditional drive controller cannot access the disk. This doesn't encrypt the data of the disk. It is possible to do a full hardware-level image of the sectors from a known set of telemetry and manually carve the file systems out, but it's a significant barrier against all but the best equipped attackers.


Hashing a key: less entropy than the key itself A web API needs to store a 'key' for authentication, in much the same fashion as a password but at 128 characters. My concern is that the salted SHA1 hash for the key has less entropy than the key itself (40 characters vs. 128 characters).

I understand that the hash is necessary in case the database is compromised, but in this case from the outside, assuming no database breach is this setup more susceptible to brute-forcing than if I were to store and match the unhashed key?

Note: this horrible setup does not replace the real password setup. Someone who 'calls the shots' wants a second password to access a certain feature, and he wants it to be long and to be called a 'key'. After coding it I will have opportunity to demonstrate how silly this is. My question here only pertains to the susceptibility of a 128-character string being brute-forced vs. its 40-character salted hash being brute-forced.

Entropy is a big word for a mathematical concept (in the context of cryptography), which is thus named out of an approximate analogy with the "entropy" as used in physics. Here, "n bits of entropy" means, more or less, that there are 2n possible values for the key.

For security, there is no practical difference beyond 100 bits of entropy. We want our keys to be safe from exhaustive search and this is achieved when trying out all possible keys, or at least a substantial portion of the key space, is ludicrously unfeasible with existing technology. Cryptographers have long used "80 bits" as the threshold for that; 100 bits ought to account for improvements in technology and a comfortable margin (at these sizes, energy dominates, not Moore's law). Beyond that size, exhaustive search is utterly defeated, and there is no bigger defeat than that.

Therefore, while your "128-character key" has potentially 512 bits of entropy (assuming the "characters" are hexadecimal digits) and the SHA-1 hash has "only" 160 bits (the output size of SHA-1), both are still very far into the "cannot do it" realm, and it does not make sense, from a security point of view, to say that one is "more secure" than the other. Both are immune from exhaustive search.

Correspondingly, there is no need for a salt here. Salts are about preventing parallelism and precomputations: they assume that the attacker can run an exhaustive search on one key, and will want to run the same attack on several keys. Salts ensure that the attack cost for ten keys will be ten times the cost for one key. With a 100+ bits key, the cost for one attack is way beyond what can be done, so there is little point in preventing parallelism. Said otherwise, if the salts change anything to the security situation, then the attacker is a god, and you should sacrifice oxen to appease him, not try to counter him with your puny hash functions.


How to detect an image virus? How can someone detect a malicious code, executable or not, inside an image file ?
What techniques and what programs are out there that will help you for the detection procedure ?

This question is a sequel to the question: Can you get infected by viewing an image?

The nature of an exploit using an image often is a buffer overrun, for instance:

Buffer Overrun in JPEG Processing (GDI+) Could Allow Code Execution (833987)

To defend against such an attack, all practices and techniques to detect attacks on the stack and heap of your image processing programs will help. Rigorously checking the source code is the first step. Other means include hardening OS and even using special hardware is an option.

A nice overview can be found in this paper on buffer overruns.


Are GNU coreutils SHA digest functions FIPS-validated (in NIST's Cryptographic Module Validation Program)? My Challenge
My project has a requirement that we use only FIPS-validated modules to do anything cryptographic, including generating checksums for binaries. We've been using the SHA-2 utilities provided by GNU coreutils for a long time to generate checksums. Now we need to be compliant. SHA-2 comes from FIPS-180-2, and I can't find a validation program like NIST CMVP for FIPS-180. I did see on some CMVP certifications that a Secure Hash Standard (FIPS-180) certificate was also listed, so I had hoped to find GNU holding a certification for their implementation in NIST's CMVP database, but I couldn't find one.
My Question
Can anyone point me to documentation that shows GNU coretuils is validated for FIPS-180-2 compliance? If not, I'd love suggestions for where to look for an alternative provider. I did find that Red Hat holds a certification for an OpenSSL implementation, which also lists SHA-3 and SHS certificates, Cert #3781, but that's for RHEL8 and we're on RHEL7.
Barring alternatives, is there a way to prove GNU coreutils's SHA-2 implementations are "sufficiently secure"? I feel that question is asking for trouble and a whole lot of math proofs... The GNU website doesn't say much about SHA-2. I read the source code for the utility here, but found nothing interesting.

I can't find a validation program like NIST CMVP for FIPS-180.

You did, actually. CMVP covers compliance for all cryptographic mechanisms approved by NIST, including the hashes defined by FIPS 180. FIPS 180 is the definition of some hash algorithms. FIPS 140 is the standard that defines what it means to be compliant. All “FIPS compliance” is compliance to FIPS 140.

I did see on some CMVP certifications that a Secure Hash Standard (FIPS-180) certificate was also listed, so I had hoped to find GNU holding a certification for their implementation in NIST's CMVP database, but I couldn't find one.

You looked in the right place. You couldn't find one because there isn't one.
FIPS 140 validation is expensive. You don't just do it on a whim: you do it because you have a clear business requirement. Furthermore, you can only validate one specific version of a program, so it's bad for maintenance (no bug fixing, no new features, no porting to a different environment, …) and it's bad for security (no bug fixing). So it's no surprise that very few programs are FIPS certified.

If not, I'd love suggestions for where to look for an alternative provider. I did find that Red Hat holds a certification for an OpenSSL implementation, which also lists SHA-3 and SHS certificates, Cert #3781, but that's for RHEL8 and we're on RHEL7.

Several vendors have certified a version of OpenSSL, including RHEL 7, which does cover the NIST hash algorithms. So you could use that.

Barring alternatives, is there a way to prove GNU coreutils's SHA-2 implementations are "sufficiently secure"? I feel that question is asking for trouble and a whole lot of math proofs...

FIPS 140 certification has nothing to do with security. In fact, it's detrimental to security, since it forbids applying security updates. If you want FIPS certification, use the certified version of OpenSSL. If you want security, use an up-to-date system.
If you care about security, the software that calculates hashes is the least of your worry: of all cryptographic primitives, they're the hardest to mess up. What you use the hashes for is more important and can be susceptible to subtle pitfalls.


What are the possible security issues when making a checkout system in Javascript? I'm pretty new to JS and I am currently making a simple step-wise checkout system using purely JS and some Ajax and PHP. I have been wondering about the possible security issues that might pose with such a system.

The way it works is as follows:

Select item, press Next(Hide current div, show next div)
Use increment/decrement and pick the amount for the item, press Next(Hide current div, show next div)
Same as 2 just with another item
Same as 2 and 3 with another item
Generate invoice
Proceed to payment(fill in card details and pay using external credit card gateway API)

Now, the thing that bothers me is number 6. What is someone edits the JS and sends different values to the external payment processor? Is that possible? And if it is, how to I securely make sure this does not happen?

Withing my Javascript I make use of several values that I fetch from a simple table from my database, such as price for the items and shipping cost. But I use these values straight in my Javacsript file for the computations and generating the invoice. I have a feeling that I'm approaching the problem in a wrong way. Is there a possibility that the "client" can change the computations as well?

How do I make sure that once the invoice is generated, the correct values are used in calculating the total price so the proper price is sent through the payment gateway, no matter if someone simply messes up with the "front-end" numbers that show up?

Note
I have been told that this would be the best place to post this question, please let me know if it's not and I will remove it.

If you want to see the checkout I made, here it is(I haven't put the payment gateway API yet as I'm worried about the security):
http://goo.gl/jdN4T0

Thanks :-)

The user's client should submit only the order's list of items (and quantities) to the server. Anything else about the order -- especially how much the entire order costs -- should be computed on the server, based on the list of items that were ordered. If the user mucks around with the JavaScript that submits which items the user has ordered, there's nothing really wrong with that: it is equivalent to the user having normally ordered some different items.

Optionally, you may have the client also compute the order total, purely as a UX convenience for the user. This value should be thrown away when submitting the order to the server, since the user may have altered it after it was calculated by your script.


Why require new users to change password? I have noticed that it is common practice when setting up a user account (Windows users accounts and Google Apps user accounts in my case) to require new users to create their own password immediately after their first log on. In searching for an answer to this question, I have found many posts explaining how to require new users to change their password, but I can't find anywhere why this is considered "best practice." What problems are created if I generate and assign a unique password to each new user and they don't change it?

To clarify, I'm not asking why it is good practice to require users to change passwords periodically. My question is what are the risks if brand-new users keep using the password I set for them instead of creating their own password?

It would be convenient for me to have them keep their initial password because I am in a smaller business setting, and if I know their password I can set up Google Drive Sync and Google Apps Sync for them, instead of having to write instructions for them on how to do this themselves.

It would be convenient for me to have them keep their initial password because I am in a smaller business setting, and if I know their password I can set up Google Drive Sync and Google Apps Sync for them, instead of having to write instructions for them on how to do this themselves.

This is exactly why it is good practice to require them to change passwords after first login;  they should be the only ones who know their passwords.  It eliminates any potential for administrative abuse, or the perception of abuse even if none exists.   Additionally, passwords they choose are less likely to be forgotten and require reset, or to be written down where they can be stolen and (again) abused.

Your second question about period password changes has already been answered in the following question: How does changing your password every 90 days increase security?


Cold-boot attack example in the real world I wrote a paper with a countermeasure for a cold-boot attack and am currently in rebuttal process.

One reviewer struggles with the motivation behind cold-boot attacks. Specifically, he asks if there is an example of a cold-boot attack used in the real-world.

To answer him, I searched for forensic companies which use the cold-boot attack, but it was not easy to find.

Are there forensic companies or forensic examples that use the cold-boot attack?

Yes, there are examples of cold boot attacks in the wild. An archived presentation gives an example of live memory acquisition on a server when DMA is not possible and when the native BIOS automatically initializes ECC memory at boot. The forensic investigators ended up writing a custom version of the open source coreboot BIOS and flashing it on the target hardware. This custom BIOS was designed to read memory and export its contents, lightly compressed, over a serial device. In order to get the new BIOS to run, they had to reset the CPU. I believe they were able to do that using the LPC bus. The end result was a low-memory footprint cold boot attack.

There are also examples of cold boot attacks being used on hardware you own in order to obtain valuable information to assist reverse engineering. Cold boot attacks have been used against various game consoles, and even prompted Microsoft to implement transparent memory encryption on their Xbox 360 console. The attack has also been attempted to view memory (UMA) reserved by the Intel Management Engine which is blocked from software access.


Possibility of business calls/leads being professionally stolen? Is it possible our VoIP calls being redirected from our intended phone number to another phone number unknown to us?  Our VoIP number is currently forwarded to mobile phones and computer phones.  Would this be an issue with our VoIP provider?  Our local network or our mobile phones?

In a situation like this, you don't know what attack vector is being used, but it seems that you may have some anecdotal evidence from customers?

The best solution is to log all occurrences and then attempt to determine the threat(attack) vector. You may have to reach out to the companies that provide your phone service or the client's. This is very dependent on how the attack takes place, considering multiple client's are potentially vulnerable with an unknown breach. Once you have a log of what user's are doing, and what results they are getting, you can quickly identify what companies are doing it and can, with the help of ISP's or phone companies, determine where the attacks are coming from.

You can start by testing using google voice numbers or track phones you activate and call or attempt to contact yourself with.

I would also invest in a VPN service and turn off location services while you do this so that an attacker might not know that you are the one making the calls if they have you blacklisted for the malware.

Once you have even a small record of where the attack is coming from, you can begin to figure out how it is happening. Vague suspicion isn't enough to figure out what is actually happening.

The more specific the threat, the more specific the solution will be.


What full-disk secure erase solutions exist for SSDs (OS X)? I want to back up the contents of my Macbook before encrypting it, but after I'm done, I would like to sanitize the SSD that I have backed up my files to, since it will be unencrypted (no point encrypting your laptop if someone can just steal your backup drive and get your files from there).  I think I read on Apple's website that OS X's diskutil secureerase command is not available for SSDs.  Or was that just diskutil secureerase freespace that's not available for SSDs?  I don't know.  I can't seem to find the page.  I don't see why secure erasure of an entire volume wouldn't be available for SSDs though.  What are some good options that I can use?  I don't feel any need to use Gutmann, since that's only necessary for HDDs.  A one-pass random bit secure erase should be sufficient.

EDIT: I would delete all files on the SSD before doing the disk sanitation of course, so all blocks would be available for reprogramming.

It's notoriously difficult to erase data from SSD drives, as explained here:
http://arstechnica.com/security/2011/03/ask-ars-how-can-i-safely-erase-the-data-from-my-ssd-drive/

The overwriting procedure that works so well for HDDs doesn't work as well for SSDs for a couple of reasons. One is that many SSDs have extra storage space that's not accessible by users. This is called over-provisioning, and some deletion tools won't give you access to this area (if you can go through the BIOS and uninstall your drive, you can get full access). While the research paper noted above that overwriting did obscure all of the data in some of the SSDs they tested, some still had old readable data on them after twenty overwrites. This can happen because of firmware bugs, and unless you're able to physically confirm that this procedure works on your SSD of choice, it's not reliable enough for a real secure erase.
Likewise, it is almost impossible to securely delete an individual file on an SSD, because the way that SSDs write and delete files is scattered, and a user has no control over what an SSD is doing where.

For this reason, it looks like Apple has disabled its secure erase support for SSD. This source has some more up-to-date information on securely erasing from SSDs:
http://www.zdnet.com/article/how-to-securely-erase-hard-drives-hdds-and-solid-state-drives-ssds/
That source will tell you how to use the manufacturer utilities to securely erase your SSD.


Can JavaScript defeat source port randomization? Can malicious JavaScript defeat source-port randomization in browsers?

I am reading a research paper which seems to claim that the answer is yes:

  many browsers seem to always assign a random local port number
  for different web pages which makes [source] port prediction
  very difficult. To overcome the challenge, we design a
  simple strategy to intentionally occupy as many local ports
  as possible so that the next port used is selected from a
  much smaller pool.

  Specifically, the malicious website can instruct the client
  to open many connections to the malicious site (or any
  other server) to consume a large number of local ports. In
  addition, the occupied port numbers tend to be contiguous
  according to our experiment likely due to the origination
  from the same JavaScript. One challenge is that the OS
  may limit the total number of ports that an application
  can occupy, thus preventing the attacker from opening too
  many concurrent connections. Nevertheless, we found such
  limit can be bypassed if the established connections are
  immediately closed (which no longer counts towards the
  limit). The local port numbers are still not released since
  the closed connections enter the TCP TIME WAIT state
  for a duration of 1–2 minutes. If an attacker can manage to
  open enough connections, he can easily use brute force
  [to guess the source port of the next request].

Conceptually, the attack seems to go like this.  Suppose the victim visits a malicious web page, evil.com, controlled by the attacker.  evil.com serves up some JavaScript that opens many connections to evil.com, in an attempt to exhaust the pool of local source ports.  Then, the JavaScript redirects the browser to www.amazon.com (or opens an iframe to www.amazon.com).  The claim is that their method enables them to predict the TCP source port number that will be used in that connection to www.amazon.com.

Is this right?  Does this attack work?

If the attack works, I have some questions about how/why the attack works:

Do browsers manage a pool of local ports themselves, or do they ask the OS to allocate a local port for them?  What pool is getting exhausted here?
If the OS is allocating the source port, isn't the source port specific to the destination server?  (I can open many connections to evil.com and exhaust all available local ports to evil.com, but I'd expect that a new connection to www.amazon.com could use any source port, even one that is currently being used with www.evil.com or has recently been used with www.evil.com.  Am I misunderstanding something?)
Or is the browser somehow manually managing a pool of local ports it is prepared to use for connections?  If so, what strategy does it use?  Is this browser-dependent?
How does the attacker predict what the source port number will be used for the connection to www.amazon.com?

I was hoping someone might be able to help me understand what's going on here, and fill in the missing details.

As far as I know, the kernel will allocate a new local port for each outgoing connection (unless the application enforced a specific port value with bind() -- this can be done even for a TCP client) and will not allow two outgoing connections to use the same local port simultaneously. Indeed, if you try to force two client connections to use the same local port number, with explicit bind() calls, the second bind() should fail.

A Linux will allocate local ports in the 32768-61000 range (see this page). This is system-wide.

I doubt Web browsers select specific local ports. I do not see what this would buy them, and, moreover, this will be lost when traversing a router which does some NAT -- a very common thing, both for home users and enterprises.

Guessing the local port in advance does not give an advantage per se to the attacker -- it can be part of a larger attack which also guesses TCP sequence numbers  (which seems harder).

Edit: actually Linux will allow a port to be reused for another connection while the first one is still active, but this does not happen often. Linux randomizes ephemeral ports (see RFC 6056) so, generally, a new port will be used. When the application does an explicit pre-connect bind(), a new port is necessarily allocated, since at that time, the kernel does not know the target address yet. For a bind-less connect(), another algorithm is used, which tends to prefer using a new port but may reuse an older. See this blog post for an analysis.


How important is it to delete disk partitions when wiping devices? The command lsblk returns the following output that lists 4 partitions.

sda      8:0    0 931.5G  0 disk
├─sda1   8:1    0     8M  0 part
├─sda2   8:2    0    40G  0 part /
├─sda3   8:3    0 883.8G  0 part /home
└─sda4   8:4    0   7.7G  0 part [SWAP]

When executing one or more of the following commands, is the necessary to delete the partitions prior?

If no, why and if yes, why?

If partitions are not deleted prior to the commands below being run, what is the risk of data remanence?

When executing one or more of the commands below, the partitions are not deleted.

dd if=/dev/urandom of=/dev/sda bs=2MB conv=fsync
hdparm --user-master u --security-erase password /dev/sda
pv /dev/urandom > /dev/sda

What you are seeing is a cached view of the previous partitions.

If they appear to persist even after a wipe, this is a result of the kernel caching the last known partition table in memory. It does this because it does not expect the layout to change at runtime, so rather than re-read the table each time the partition layout is queried, it reads it once at boot. If you want to force it to re-read the partition table, you can run a command like partprobe, reconnect the drive, or reboot.

The partitioning information is contained in the partition table, which is a small (64 bytes) section of the drive that specifies the location, type, and length of each partition. MBR partitions keep this table at the very beginning of the drive. GPT partitions keep an additional backup at the end of the drive and add support for more features. When you overwrite an entire block device, the partition table is also wiped.

As another answer mentions, the partition layout itself is unimportant and contains no sensitive data. They contain essentially nothing other than the bare minimum of information required to split a disk up into multiple partitions. You can see exactly what is stored for both MBR and GPT tables:

https://en.wikipedia.org/wiki/Master_boot_record#PTE
https://en.wikipedia.org/wiki/GUID_Partition_Table#Partition_table_header_(LBA_1)


Security of downloaded softwares I downloaded Rufus and Win32 Disk Imager from Sourceforge and used them on a flash drive to boot another operating system. I'd like to know if it's possible that one of these infected my OS or firmware and how to find out if the firmware or OS are infected besides Antivirus for the OS?

The way to find out if you have a virus is to use an antivirus.  Basically, in the question you ruled-out the only real workable solution.
A second idea would be to make a reference system that you know is safe. You can compare the real system with the reference system. Use other tools than Rufus & disk imager. If the reference system is different, you still are not sure that the problems are caused by Rufus or Win32 Disk Imager.
If you want to know if one of the two specific programs has given you malware, you must compare them with known-good versions.  You can do that after they have been used, but theoretically, they could have changed themselves when they ran.