Using PGP to encrypt emailed password reset token Even if I am careful not to lose/forget my password for some web account, other people are not and so most sites have a password reset mechanism, often being a reset token sent to the registered email address.  Unless there is a way to opt out of this (which I have never seen), that leaves my account at their site vulnerable to email take over or interception.

Instead of allowing the security conscious to opt out of password reset emails altogether, I've pondered the idea of letting people optionally register a PGP public key, and then using that key to encrypt the reset token that gets emailed.

I think it would theoretically be better to have them send a signed password reset request, rather than decrypt a token, because this seems more in the spirit of having separate keys for separate purposes.  But encrypting the token would conceptually be a very simple change to one part of the existing code:

if (defined key) encrypt(token,key);
email_send(token);

Whereas a signature based system would be an entirely new system, so would have more opportunities to mess it up.

I have never seen anything like public key encrypted resets done.  This lets the account owner calibrate for themselves exactly how much effort they want to go through to secure their private key.  Is there precedent for this?  Is there some reason it wouldn't work?

(This is for my curiosity, I don't plan to implement such a system.)

Keep it simple. Sending signed mails with given, parseable format makes things rather complicated. If a user opts for encrypted mail, send him (all, including all other) mail encrypted, including password resets.

Selbstauskunft.net (German language only), a service for sending requests to companies request information they store about you based on privacy law, lets you do exactly this: you can select an OpenPGP key (which they list based on a keyserver query), and if you do so, they will also encrypt the password reset mail.

Regarding OpenPGP mail interfaces, only Hetzner, a German hosting company offering an e-mail API using OpenPGP signed message for domain registration.


Is an API password safe in a mobile application? I want to let customers use our API in their mobile applications i.e ones that they develop.

I have never developed for mobile so don't know if having the clients secretAPI key stored in the app is secure. I am assuming they will be able to connect using SSL from the app so transmission will be secure.

From what I am reading online everywhere is suggesting using oAuth. But oAuth requires a sign in from the client's user which is not possible in this case as the clients users wont have an account with our service.

Anything stored on a user device should be assumed readable by that user. This is regardless of whether its stored in file storage, OS registry, compiled code or even just kept in RAM. It also makes no difference whether the application is a mobile app, desktop application or web app (for data available on the client).

With compiled code you can either inspect the binary directly, use a tool to decompile it or execute the code and take a memory dump then inspect that.

If you want to restrict calls to your API solely to code controlled by your clients then they will have to authenticate their users and forward requests via their own servers.


So, how dead is antivirus exactly? Symantec recently made a loud statement that antivirus is dead (http://online.wsj.com/news/article_email/SB10001424052702303417104579542140235850578-lMyQjAxMTA0MDAwNTEwNDUyWj ) and that they don’t really consider it to be a source of profit. Some companies said the same afterwards; some other suggested that Symantec just wants a bit of free media attention. Some companies just silently recommend using advanced information protection and press is full of data on antivirus efficiency being quite low. A notable example would be the Zeus banking Trojan and how only 40% of its versions can be stopped by antiviruses (http://www.bankinfosecurity.eu/banking-malware-new-challenger-to-zeus-a-7006/p-2 ). Arms race of protection and malware developers is probably not going to stop, so this situation will remain.

On the other hand, nobody was thinking too much of antivirus anyway for a long time already, so it’s hardly surprising. It’s not a panacea; the only question that remains is just how exactly should antivirus operate in modern security solutions. Should it be one of the key parts or protection solution or it should be reduced to protection against only the easiest and already well known threats?

It’s not only about dealing with threats, too, there are also performance concerns. Processors get better and interaction with hard drives becomes faster but at the same time antiviruses require more and more of that power. Real time file scanning, constant updates and regular checks on the whole system only mean one thing – as long as antivirus is thorough, productivity while using this computer go down severely. And this situation is not going to change, ever, so we have to deal with it.

But how exactly? Is the massive migration of everything, from workstations to automatic control systems in industry, even possible? Or maybe using whitelisting protection on windows-based machines is the answer? Or we should all just sit and hope for Microsoft to give us a new windows with good integrated protection like windows 8 is stated to have? Any other ways to deal with it?

This is kind of an opinionated question.  I also think antiviruses are overrated, but its an interesting topic.

I think that malware/virus behavior is where things will go.  Creating signatures based on behavior rather than the binary itself.  So I think some kind of analytics will be the way antiviruses will go, whether it be in the cloud or not.

Essentially automating malware analysis by doing analytics on registry accesses, function calls, system file accesses.  Obviously this would take a lot of processing power if you're doing live scans.  So whether nightly logs of these get uploaded to a cloud for analysis, or you just have a ridiculous machine and run it anyway it'd be an interesting metric.  I'm not sure how much of this is going on in current antivirus software, and if it is please comment and let me know.

But signatures based on binaries is pointless in the ever changing/obfuscating malware world.  Looking for common behavior patterns is an interesting topic to go down.


Can password-protected zip files be broken without brute force? You have a zip file that you created with 7z to password-protect it with AES 128. Can a smart adversary extract the data only through brute force, or is the file vulnerable to other attacks - such as, I don't know, being able to bypass the password and extract the data?

In direct meaning, you mentioned (extracting data), no. But keep in mind, however, that any program (or user), in any password-protected ZIP archive, can (without knowing the password):

browse list of files,
check file types1.

ZIP files can also be "broken" in the meaning, that you can overwrite exisiting, password-protected file, stored inside ZIP file, with another file, named the same, without knowing the password.

All these mentioned operations does not give attacker access to contents (he or she is still unable to extract data), only list it or destroy it. So I'm mentioning this only as a possible side effects, you may be not aware of. Some people treats this as security leaks, stating that for this reasons, ZIP files are insecure, but other my assume that security is not violated, even with these mentioned "additions".

Since contents of ZIP file can be changed, without knowing the password, by replacing a password-protected file with another one, we can speak about complete insecurity in psychological terms (social engineering, hacking etc.). The attacker can alter contents of ZIP file, without knowing the password and claim to its victim that it is unchanged. Victim, without knowing above mentioned side-effects, may incorrectly assume, that archive hasn't been changed or its contents aren't altered, since he or she (the victim) is the only person, that knows the password.

Here you have my simple question about security of ZIP files, which turned out to be a very good readout, with a lot of useful answers and comments.

1Try to send encrypted, password-protected ZIP containing EXE file via Gmail to see this "in action". Gmail will be able to detect, that you're sending an executable file (and prevent you from doing so) even though, it does not know password, your ZIP is protected with. Side note: It is also both funny and surprising, that renaming the very same file, by giving it different extension (i.e. file.not-zip) can completely disarm this "protection" mechanism and allow you to send archive containing executable file. Funny, because Gmail is able to "break" encrypted, password-protected ZIP to "save" user from sending executable files, but "dies" completely on file rename.


Could embedding website links in an email lead to more hack attempts? My organization currently sends out emails directing our customers to login and view information on a customer portal site we have developed.  The content of these emails is usually pretty generic since we prefer to send all confidential information to our customers through this secure website.

I have recently been asked if we should eliminate all URLs that link to our customer portal from the emails that we send out.  The justification for this request is to deter attempts to hack our customer portal.  Since this request came from executives and not security experts I would like your opinions.  Would this in any way decrease our chances of our customer portal being attacked?

One more piece of potentially useful information is that a link to this customer portal is published at the bottom of our public website.

I see at least one way an attacker can take advantage of these e-mails.  The attacker (perhaps one of your customers) might attempt to "claim to be you" and send an e-mail to your customers with a very similar link, to a faked version of your website.  When the victim user "logs in" to the faked site, the attacker would then have their username and password, and could then log in as the victim user on the real site.


SSL terminates on webserver instead of load balancer. PCI compliance question We have a website that processes credit card data and uses a load balancer for our two web servers. The SSL connection terminates on the webservers and not the load balancer. Is the load balancer in scope for PCI?

First, IANAQSA (I am not a QSA), but I do deal with PCI a lot.

Using the open scoping toolkit as guidance, the Load Balancer would not be a category 1 system. That is because it does not directly store, process, or transmit unencrypted cardholder information. Given that it opening network connections to the web server (which is a category 1 system), then it would likely be a category 2 system. A category 2 system is still in scope for PCI as it can affect the security of the cardholder data environment (CDE).

I would suggest following the scoping toolkit yourself or engaging a QSA for more information.


SAML 2.0 Multiple AuthnStatements If I interpret the SAML 2.0-protocol correctly you can have multiple AuthnStatements. What is the purpose of this? I cannot see a use case of having multiple AuthnStatements really.

As per SAML 2.0 specifications, an authentication statement specifies that the assertion subject was authenticated by a particular means at a particular time.

One of the use cases with some IDPs is when a multi-factor authentication has been performed. In that case, each authentication statement specifies one step in the authentication process. For example -

AuthnStatement one - Subject was authenticated via password at
12:34:35
AuthnStatement two - Subject was authenticated via OTP/Biometric at 12:34:56

SAML 2.0 has a vast set of features but not every implementations support all of them. Most implementations support only single authentication statement which covers a major use case.


JavaScript resource files are available without authentication - is this a security risk? I am testing an application and come across few javascript pages that are publicly accessible. I have earlier also found few websites which allows .js pages public access.
So as per me this can be considered as a security issue, since one of the ngapp.js page contains entire logic of the angularjs page.
And if wants to secure such pages post authentication, is it possible?
Can someone please clarify this?

So I'm guessing the drive of your question here isn't whether the JavaScript files are obfuscated or not but rather whether they're accessible pre-authentication?

If that's the case then I'd say it shouldn't be an issue as the JavaScript is sent client-side and therefore no sensitive information should be embedded in it.

With that said, as a hardening measure I generally recommend not providing any information to an attacker that I don't have to , so I would restrict access to post-authentication wherever possible.  Things like Angular apps can disclose a lot of information like valid application paths which is kind of useful to attackers in trying to find other issues.

In terms of how to achieve this, I'd suggest that you could separate the templates of the site, such that the main Angular (or similar) app is accessed post-authentication only.


Stopping users from spoofing an IP I have a MikroTik router (that has firewall and scripting capabilities) that is acting as a DHCP server for the 192.168.1.0/24 subnet. Is there any way I can stop it from passing traffic from a user who has statically assigned himself an address within that range? If so, what is it called so that I can google how to implement it? Do I have to create a script that dynamically modifies firewall rules to match DHCP leases?

My end goal shown as an example in case my question is not clear enough:
A user gives themselves the address of 192.168.1.2. My router sees that this is not in his DHCP lease table. He drops all traffic from 192.168.1.2.

No,  what you are asking is fundamentally not possible.   The primary function of the MAC address sublayer is allow for the communication of numerous devices(OSI Layer 2) via a physical medium (OSI Layer 1).  Nothing is preventing an attacker from assigning their NIC an arbitrary MAC address value.  One side effect is that your DHCP server will then issue the same IP address to the same MAC for each request,  even if the request originates from an attacker's machine. This is not to be confused with ARP spoofing,  which is useful for MITM attacks, and can be prevented.

Really what you are looking for is a VPN. This can be used to allow only trusted hosts to communicate.


It's possible to make an account in mainstream social media without giving them my phone number or have a phone number that can identify you? I'm learning about privacy and security and one of the tips everyone always talk about is compartmentalization, I'm trying to do this and get rid of my old profiles in social media (one of them have 10 years since I made it), I'm planing to make new ones where I won't be giving up too much information about myself.

The problem is, in Brazil (I don't know about other parts of the world), our phone number are attached to our 'CPF'(Cadastro de pessoas físicas), it's like the ID in US I think, this is the main way to identify a person, and we don't have anything like a burn phone.

Today gmail and facebook always ask for phone numbers, so, what now?

Today gmail and facebook always ask for phone numbers, so, what now?

Get an anonymous phone number. Whilst your country may not allow them, plenty do, and as modern cell networks allow roaming, you can use a foreign SIM card.

You could even use a VoIP provider, such as google Voice (which requires an american phone number to register, but not use for receiving SMS and VoIP calls). USA notably does not require any registration of telco customers.

You can order such SIM cards on eBay.

If you want to be even more anonymous, only use the SIM card in a phone bought with cash, and in public areas in a large city.


Using buffer overflow on word with macro injection I know how a buffer overflow works on a local network against an application running on a TCP port, assuming it doesn't drop the connection, but I would like some advice on an "exploit" I would like to try to develop. I would like to do a buffer overflow on word by injecting the appropriate script into the macro of a word doc and then sending it to the victim. I am wondering a few things. One, is this even possible? In answering this question, please disregard the fact that Word almost certainly has techniques of preventing this (ie disabling macros). Two, what software do I need to inject my script to the macro? And three, is is possible to write the program in Python and still have it work against a victim without Python on their computer?

Why would you bother using a bufferoverflow when using Macros? It would be easier to just inject some VB script which does calls to, for instance, powershell (there is a really good exploitation framework here named PowerSploit).

Exploits which do target bufferoverflows within Word, are a lot harder to find and exploit. If the target has EMET installed, you're immediately completely out of luck.


How to decrypt a message using only session key? I can view a session key to the encrypted data using option gpg --show-session-key <file>.
Knowing the session key, how can i decrypt the data without using my private key?

To encrypt a file for a recipient:

$ gpg -ea -r foo@example.com < file > file.gpg

To get the session key for an encrypted file when you have the private key of the recipient:

$ gpg --show-session-key < file.gpg
...
gpg: session key: `9:901D6ED579AFF935F9F157A5198BCE48B50AD87345DEADBA06F42C5D018C78CC'
...

To use this session key to decrypt the file, without needing the private key:

$ gpg --override-session-key 9:901D6ED579AFF935F9F157A5198BCE48B50AD87345DEADBA06F42C5D018C78CC -d < file.gpg

This information was obtained by simply having a look at the man page for gpg:

--show-session-key
   Display the session key used for one message.
   See --override-session-key for the counterpart of this option
   ...
--override-session-key string
  Don't use the public key but the session key string.


Whitelist-based integrity scanner for Windows EXEs, DLLs, etc I am looking for some kind of security software that scans your computer for all EXE and DLL files, calculates a cryptographic hash for them, and looks each one up in a database, logging all the ones it doesn't recognize to be manually reviewed later.

Or if someone could recommend me a good, well-maintained database of known EXE and DLL file-hashes from a wide source of reputable vendors, that would also be helpful.

I haven't had any luck finding something like this.

You could try another approach, or maybe mix different strategies. From my experience with many customers I can tell you that we have acchieved security by classifyng EXE's with the following methods:

Allow EXE/DLL from certain directory (not very often and only combined with other safety strategies like 'no admin rights')
Allow EXE/DLL which has a specific certificate (self-signed, Microsoft etc.)
Allow EXE/DLL by hash (only for special software)

The software I have used had also the possibility to switch on some 'learning mode'. Afterwards the user starts working for a couple of days and the software 'learns' which EXE/DLL's are safe.

Give it a try: http://egosecure.com/en/


If I typed username and password, but wrongly clicked another button to go to another page instead of the login button, is there any security risk? For example, suppose I already typed username and password at this Facebook page:

https://m.facebook.com

But I wrongly clicked 'Create New Account' instead of Log In. My question is, does it increase any security risk?

I admit this question seemed like it had a straightforward answer at first but then after looking into Facebook's page and the button functionality it definitely does beg the question.

I discovered that if you have the username/password fields filled in with the credentials of a legitimate account, or have auto-fill username/password in place pulling from either Chrome itself or a password manager- which may/may not be a good practice depending on your level of security paranoia- then selecting the 'Create New Account' button actually does send a POST message that includes details of your login information from the previous page, including username/password. If you're curious and aren't already aware, this can be done by simply monitoring the network requests in your browser.

Despite this, I would still say that it depends on the application you are interacting with. In this case, since it's Facebook and- though this may be a point of contention within the community- we can trust Facebook with our data, then as long as the POST request that had my username/password information in its payload was made over secure transmission I can presume to be unworried about a security risk. I would only imagine that at this point the information being sent to Facebook via this POST request would need to be equally as secure as the login button itself.

I have never encountered another site that does this, but am hoping to see other answers on this based on other experiences.


Is the iPhone vulnerable to the BadUSB malware? Supposedly, the BadUSB malware can infect any USB device. But I find this hard to believe since I don't see why most USB devices would have reprogrammable firmware and I find it even harder to believe that the iPhone could be vulnerable to this.
Is there any evidence that the iPhone is or is not vulnerable to BadUSB?

It often isn't about reprogramming the firmware of an USB-device/gadget.

You might just create a USB device that looks like a charger, but is an evil device.

Example:

You want to attack a Windows computer with an USB-stick
You get an Arduino Leonardo, which can act as a keyboard, but has the size of an USB-Stick
You program the arduino to output the keystrokes to do  the following things:

Open a command prompt
Activate remote desktop access with password "1234" on command line
Send IP address to the attacker(e.g. by opening a web page on the attacker's server)

This example was for Windows, but because iOS devices also support external keyoard and mouse they might also be vulnerable.

P.S.: Protection against such an attack might be done, e.g. by giving explicit permission for an external device before it is acccepted by the host.


Windows Failed Logon Type 2 by A Remote Computer Account! I woke up to a creepy Windows event where the Server B used the Computer account$ of Server A to log on to Server B (itself).

The logon type is 2 and the destination process is of course "lsass.exe"

Why would a computer account request an interactive logon attempt type 2 instead of over the network type 3?

The failure reason is "The user has not been granted the requested logon type at this machine."

There are a couple of instances where a computer would use a type 2 logon instead of type 3:

KVM over IP is viewed as a type 2 logon.
Using "RunAs" without "/netonly" is a type 2 logon.
Before Logon Type 10 came into existence, RDP logons were type 2 logons.  How old are these machines?


WPA password cracking setup let's say i am trying to crack a WPA password and i need to create a virtual machine setup for it. What would the setup be if i limit myself with a specific number of hours ( let's say 24 hours and password is 8 alphanumeric characters).

does any formulas i can use to determine what my set up would be exits? like how to calculatethe number of GPUs, CPUs and RAM i need to finish in the given time limit

Assuming that the passphrase was randomly generated, the two pieces of information needed to perform the calculation for a fully brute force attack are:

Keyspace. This is how many possible combinations there are. In this case, 26 upper-case, 26 lower-case, and 10 digits = 62 characters, in eight positions (so 62^8, 62 to the power of 8, or 62x62x62x62x62x62x62x62).

Real-world GPU cracking speed. This is highly variable depending on the platform. Benchmark numbers for tools like hashcat are generated under ideal conditions which can't usually be achieved in real-world attacks, so running the actual attack is the gold standard for determining real-world throughput, and then doing the math to match your target scale.

Here is an attack under the parameters in your question, running on 6 GTX 1080s:
$ hashcat -a 3 -m 2500 -2 ?l?u?d hashcat-wpa2.hccapx ?2?2?2?2?2?2?2?2
hashcat (v4.1.0) starting...

OpenCL Platform #1: NVIDIA Corporation
======================================
* Device #1: GeForce GTX 1080, 2029/8119 MB allocatable, 20MCU
* Device #2: GeForce GTX 1080, 2029/8119 MB allocatable, 20MCU
* Device #3: GeForce GTX 1080, 2029/8119 MB allocatable, 20MCU
* Device #4: GeForce GTX 1080, 2029/8119 MB allocatable, 20MCU
* Device #5: GeForce GTX 1080, 2029/8119 MB allocatable, 20MCU
* Device #6: GeForce GTX 1080, 2029/8119 MB allocatable, 20MCU

[...]

Hashes: 1 digests; 1 unique digests, 1 unique salts
Bitmaps: 16 bits, 65536 entries, 0x0000ffff mask, 262144 bytes, 5/13 rotates

Applicable optimizers:
* Zero-Byte
* Single-Hash
* Single-Salt
* Brute-Force
* Slow-Hash-SIMD-LOOP

Password length minimum: 8
Password length maximum: 63

Watchdog: Temperature abort trigger set to 90c

[...]

Session..........: hashcat
Status...........: Running
Hash.Type........: WPA/WPA2
Hash.Target......: 8381533406003807685881523 (AP:ae:f5:0f:22:80:1c STA:98:7b:dc:f9:f9:50)
Time.Started.....: Wed Apr 25 06:39:06 2018 (4 mins, 40 secs)
Time.Estimated...: Sat Oct 23 07:02:24 2021 (3 years, 181 days)
Guess.Mask.......: ?2?2?2?2?2?2?2?2 [8]
Guess.Charset....: -1 Undefined, -2 ?l?u?d, -3 Undefined, -4 Undefined
Guess.Queue......: 1/1 (100.00%)
Speed.Dev.#1.....:   332.4 kH/s (6.93ms)
Speed.Dev.#2.....:   327.8 kH/s (7.06ms)
Speed.Dev.#3.....:   329.5 kH/s (7.00ms)
Speed.Dev.#4.....:   328.7 kH/s (7.01ms)
Speed.Dev.#5.....:   330.4 kH/s (6.96ms)
Speed.Dev.#6.....:   330.1 kH/s (6.98ms)
Speed.Dev.#*.....:  1979.0 kH/s
Recovered........: 0/1 (0.00%) Digests, 0/1 (0.00%) Salts
Progress.........: 553451520/218340105584896 (0.00%)
Rejected.........: 0/553451520 (0.00%)
Restore.Point....: 8437760/3521614606208 (0.00%)
Candidates.#1....: etET7778 -> exQC5678
Candidates.#2....: 1BDpERIN -> 1r8tBONA
Candidates.#3....: l5MUQURL -> lEeqNANA
Candidates.#4....: bjHuRANA -> bCLfNANA
Candidates.#5....: 2Rh7MANA -> 2VLbLESS
Candidates.#6....: rFrR7778 -> r0xsANAN

Once you know the performance of your GPU for that hash and that attack, you can multiply that speed (or divide that time estimate) by the number of GPUs you'll have. If you need to scale out across multiple chassis, tools like Hashtopolis or hashview can manage a cluster of systems. If I had ten systems like the above, it would take 1275 days / 10 = ~4 months to fully exhaust the keyspace (and ~2 months to crack the average case). You get the idea.
For your other capacity questions, in practice, you don't need a lot of CPU for the attack itself when using hashcat. I'm told that recent versions of hashcat do work best when the host system has roughly much RAM as the total amount of GPU memory, but this isn't exact. For example, the system above collectively has 48G of GPU memory, but only 32GB of RAM and works fine for WPA2. But this memory rule of thumb is good for general system-build advice.
As should be clear by now, if you only have 24 hours, and your passphrase was randomly generated, you won't get very far. But the above should help you perform the calculations.


What good does the Kerberos Authentication Server I currently try to understand the Kerberos protocol. And I think I understand the parts of it. How Alice gets a ticket from the Authentication Server to authenticate herself to the ticket granting service. And how that one does the same, so Alice can authenticate herself to Bob.

But what I don't understand is, why aren't the Kerberos Authentication Server and the Ticket Granting Server the same?

In fact they are the same, and that is called the Key Distribution Center (KDC).

The kerberos has three heads (like a cerberus):

The KDC that consists in the auth and the ticket granting system.
The Service.
The user of the service.

The issue here is that what is really happening is:

Alice authenticates herself against only one machine.
Alice gets a ticket for an specific service (here is one of the tricks, she is authenticated but he hasn't access to all services, just the ones she asks the tickets for so she may be able to access the printer servers without compromising the "production servers" which she is also an admin). You are establishing boundaries.
The ticket grants you access to the servers.

So, in fact, the servers "doesn't allow anybody" to get into, they just allow valid tickets. Think of it like going to the movies:

You buy your tickets at home online (this would be like the auth phase, where you present your credentials, name, card number...):
You go to the ticket machine and "auth" yourself with your credit card or an id number (the #1 step token), so it gives you a ticket.
The guy at the front check your ticket and allow you to go into if it is the right session and the right place.
You watch the movie.

As you can see "1" and "2" are very different operations and you will never have to show the guy at the front your credit card to let you go into the movie =)

Hope this helps a bit =) (Also, I have checked my notes and I have this link for you. )


How cryptographic hardware accelerator is secure? I am using cryptographic hardware accelerator of am335x. As per documentation it unloads the cryptographic operations from the main CPU and hence increases the performance. But from the security point of view how is it useful. In the documentation of 'self encrypted disks'(SED) it is given that the encryption key can not be accessed outside the dedicated processing chip. Hence any malware can not access the keys from RAM. Is it true in the case of am335x? Or are those keys stored in RAM?

Simple hardware crypto accelerators have no security benefits, only performance ones (we could argue that this allows for larger key sizes thus increased security).

Hardware security module, on the other hand are similar devices but in addition they can also store the keys securely and never reveal them to the computer itself; when the HSM is set up the keys are either imported into it or generated on it directly and they have no way of leaving the module later (the only way to get the keys out would be physical access, and even then they're tamper-proof).

The computer just gives it the plaintext, instructions on what to do with it, and the module spits out ciphertext.

About the AM335x, in the datasheet of the AM3352, there is the following :

  Crypto Hardware Accelerators (AES, SHA, PKA, RNG)

So it can do AES, SHA in hardware (not sure what PKA stands for), as well as generate cryptographically-secure random numbers. There is no mention of key management however, so the keys will still be stored on the computer itself and the security of those keys can't be compared to an HSM for example.


Why sign session cookies? I'm using express-session middleware to store sessions on the server. The client's browser gets a cookie with the session ID. A secret is required in the configuration; it is used to sign the session ID cookie.

https://github.com/expressjs/session#secret

What's the value of this signature? What's the threat it's mitigating? Even if a threat actor knows a session ID, they have no access to the session store. The odds of guessing someone else's ID should be astronomical.

Apparently, the software permits developers to provide their own session IDs, which (because people are often bad at security) might be short and/or predictable. The signature is thus added to create a minimum amount of entropy in the token to prevent guessing or brute-forcing another user's token if the token itself is low-entropy.

Or, as you put it, the odds [against] guessing somebody else's session ID should be astronomical... but you probably generate session IDs using something like "16 bytes from a CSPRNG" and some people instead use Random.nextInt() or even just an incrementing counter or something. In those situations, guessing or even brute-forcing other IDs is pretty easy, so the token is signed as, essentially, a way to "mix in" the entropy of the secret with the (possibly low) entropy of the token without revealing the secret itself.

Obviously, if both the token and the secret itself are guessable and the attacker realizes this, this will still be insecure. But, at some level, there's nothing you can do about a dev who is obstinate about being insecure. No matter how many footguns you take away, somebody will nail themselves to the floor with a pencil.


Use elliptic curves certificate only if client support them Is it possible to send an EC certificate to a client if it supports it or send a regular certificate if it doesn't?

ECDSA certificates can only be used together with the *_ECDSA_* ciphers, i.e. ciphers like TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA_P256. The client sends the list of supported ciphers within the  ClientHello initiating the TLS handshake, so it is in theory possible for a server to decide based on these ciphers which certificate to send.

In practice support for this feature varies, but there is support in Apache 2.4 and support was added to nginx in version 1.11.0. In both cases you need at OpenSSL with at least version 1.0.2.


Can a QR code steal my mobile's data? For example https://web.wechat.com

By scanning the QR code from my mobile, it automatically logs in with my wechat account.

Here, can QR web sever or QR devices steal the mobile data?

Short answer: No

Complementing what schroeder said, you can verify the content of that QR code with any QR code scanner (there are some for PCs). I verified and the content is an URL, like:

https://login.weixin.qq.com/l/YYEwfl9Y-A==

By itself those qrcodes are not dangerous for your mobile. However, whatever is in that site is what you need to care of. Some sites may have exploits that could potentially access data in your mobile (specially if your browser/os is not updated or old).

If you check the content of a QRcode and looks like Javascript code, then I would be worried about, however in order that to happen, the QRcode itself would have to have a lot of information in it (making it hard to pack in a "common" size QR code).

So your next question should be: is "wechat.com" safe? If you downloaded their app, you should also ask if that app is safe. That app has much more potential to access your data (depending on what permissions you allowed it), which in that case, if they wanted to steal your data, they don't need the QRcode for that, they already have the app in your phone.


Sniff network packets that don't talk to my computer? How would one sniff wireless network packets that do not pass directly through my computer- without access the the AP or router?

I want to be able to sniff for network traffic (even while encrypted) when authenticated to the network and when not authenticated.

Thanks

You need to run in monitor mode; promiscuous mode doesn't always work with Wi-Fi adapters (and may not even be supported for Wi-Fi on some OSes or with some Wi-Fi adapters).

Whether Wireshark can do it by itself depends on the OS.  On OS X, it can; on Linux, it could, in theory, but for complicated reasons it can't, on most if not all distributions, in practice, so you need aircrack-ng's airmon-ng script to turn monitor mode on, and you can capture after that; on Windows, it can't do it at all except with an AirPcap adapter; on other OSes, maybe, maybe not.  See the Wireshark Wiki's page on WLAN capturing for details.


How to mitigate SIP Reflection Attacks? SIP has a practical need to be widely available, but its services are prone to being spoofed as reflectors used in DDoS attacks.

A freeswitch server I manage has seen heaps of registration requests on UDP (~300/second), each one generating a 401 unauthorized response.  We only caught these when performance issues led me to do a dump on the wire and investigate the traffic.

Being UDP traffic, blocking it in iptables has a limited effect.  The incoming traffic still wastes bandwidth, but the SIP server doesn't have to respond to it and the reply traffic isn't generated.

Because it's UDP, it's probably being spoofed.  The real victim is probably the "source", which is being bombarded with my 401 messages.

This kind of problem must be very common.  Is there a current best practice for hosting SIP services so that they aren't valuable to attackers as sources for reflection attacks?

Thanks for any insights.

Update:  24 hours after blocking the inbound SIP traffic, it stopped.

Most SIP daemons (e.g. Asterisk) have an option for whitelisting IP addresses that can be used to access the server, so if possible I'd use that and ban all others. Keep in mind that's it's usually possible to allow incoming calls from any IP address but whitelist authentication requests to a set of IPs.

I'd probably also look into something like fail2ban, which is a service that monitors logs for failed login attempts and adds iptables rules based on various rules. Automatically banning IPs for incrementing time periods after a number of failed attempts will quickly discourage attackers, because even reflected attacks require a pool of proxy machines. If you're identified as a target that doesn't quickly block malicious attempts, it'll only encourage attackers to continue.

One thing to watch out for with fail2ban is that attackers may spoof the UDP source address and ban various addresses. Thankfully you can use jail.conf in the fail2ban config to set a whitelist of IP addresses that will never be banned. As long as the attacker never discovers one of these IPs, their attacks will be mitigated pretty successfully.

You could also use a script to add and remove iptables rules to block all traffic outside office hours, or change the SIP port to avoid automated drive-by attacks. Both of these are a little bit of an inconvenience, but can be useful when avoiding attacks that scan across the internet for open SIP instances.

Further reading:

Seven steps for better SIP security
How secure is your PBX?


What are the dangers of a mailto link? I was looking through my spam folder, and there's a 100% sure spam email, that asks me to confirm that I want to unsubscribe by clicking some big unsubscribe button. That button is simply a mailto: link, similar to the one below
mailto:user@inactivedomain.com;someotheruser@gmail.com;moreusers@mailbox.org;etc@inbox.ru;etc@list.ru;etc@bk.ru?subject=Unsubscribe

There are no images in the email, so no pixel tracking.
What is the attack here?
Is the attacker's hope that I would click on the mailto link, and then click send, and then they'd know that my email address is of a gullible person, so they'd better prioritize their real spamming resources, or is there more to it?
I find the above attack odd, because it puts quite some burden on the attacked. I need to ignore the fact that I never subscribed to require clicking on unsubscribe, then I need to click on Unsubscribe, then the mailto: protocol needs to be correctly associated with whatever I use for email, then I also need to click send, then the email client would ask me to confirm that I want to send a message without any content, then I would either confirm, or actually write some text in the content, and then the message would be sent, and the attack would be successful. That's a lot of work and I can change my mind at any time in this process and the attack would be unsuccessful.
Can a mailto link be somehow exploited?

Aside from the rare direct mailto-related exploit (H/T @nobody), an attacker learns what your preferred email identity is when you use an email reply or mailto. Perhaps they got to you through a list or a forwarding address (if you're like me, you customize addresses to services so you can track how your address is shared).
An actual email reply says you're a real person, verifying and/or revealing a new address to the attacker.
It's also likely a ham indicator for some anti-spam systems. The unsubscribe model by systems like (iirc) Microsoft's requires mailto:, which may help tip spam into a bulk designation instead of a junk folder.


How to test for zone transfer? I heard that it is possible to test for zone transfer attacks on a web application using host and dig commands in Linux. How it is done?

The syntax for host is:

host -t axfr domain.name dns-server

For dig:

dig axfr @dns-server domain.name

Replace dns-server with the authoritative DNS server and domain.name with your target domain name.

Keep in mind that this has very little to do with web applications, the above has to do with DNS. If we're talking about penetration testing, DNS zone transfers are a check one is more likely to perform while doing a network (rather than an application) pentest.

DigiNinja had put up a domain name zonetransfer.me for testing. Example:

$ host -t axfr zonetransfer.me nsztm1.digi.ninja.
Trying "zonetransfer.me"
Using domain server:
Name: nsztm1.digi.ninja.
Address: 167.88.42.94#53
Aliases:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 15449
;; flags: qr aa; QUERY: 1, ANSWER: 41, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;zonetransfer.me.               IN      AXFR

;; ANSWER SECTION:
zonetransfer.me.        7200    IN      SOA     nsztm1.digi.ninja. robin.digi.ninja. 2014101001 172800 900 1209600 3600
zonetransfer.me.        301     IN      TXT     "google-site-verification=tyP28J7JAUHA9fw2sHXMgcCC0I6XBmmoVi04Vl
... etc


Can a virus that's in a file stream infect a computer? I have a java web application that accepts base64 image strings from users. These strings are first converted to a byte stream after which the stream is scanned for viruses. After that the file is stored as a resized version of the actual image.

My question is, when the file has been transformed from a base64 string to an in-memory byte stream and it happens to contain a virus, will that virus be able to escape the byte stream and infect the rest of the server? Or can it only do that after the stream has been transformed to a file stored on disk?

Neither.

A virus sitting on a disk as a file is harmless. A virus sitting in memory as data is harmless.  It can only infect a machine when something tries to execute it or interprets it improperly.  This can happen because of user action (tricking someone to download and run a .exe file) or because of application weaknesses which is what you are asking about (e.g. hiding malicious data in a PDF to trigger a vulnerability in a PDF reader and trick the application into executing malicious instructions).

So to pick some examples from your situation:

If your webserver has a RCE vulnerability then malicious data in the request may execute a virus before it even reaches your application (examples)
Improper input handling may allow an attacker to trick your application into executing malicious commands (examples, more examples)
An error in your programming language's stream processing system may allow a malicious payload to execute code (which is so unlikely I don't even have any examples of it)
If the location you store your file on disk is also a location where your application may find and execute code, then an attacker may trick your application into saving a virus and then trick your application into executing it (examples)
If your image processing library has a known vulnerability then a malicious image may be able to execute commands on your server (examples)

However, a virus can't cause trouble simply by being on your server. It has to find a way to execute itself. The only defense is to practice proper application security and keep everything up-to-date.


what does key size (64 bit) in DES means I im taking cryptography classes and learned that the key size of DES is 64 bits.
So
if the key is array of Alphabetic char, and 1 char takes 8 bits
then
64/8 = 8 .
So the key can be only 8 char long maximum.
Example :
aaaaaaaa or acacacac

But when i used openssl to encrypt data using DES it accepts keys more then 8 char long.

root@io:~# openssl enc -des -in data -out data.enc
enter des-cbc encryption password: aseaxsasceaceasxaeaxsaceac
Verifying - enter des-cbc encryption password:aseaxsasceaceasxaeaxsaceac
root@io:~# openssl enc -d -des -in data.enc -out data.plain
enter des-cbc decryption password:aseaxsasceaceasxaeaxsaceac

And it accepts even more long keys then i posted here.

1) So my question is what does key size (64 bit) in DES means what point im missing .
And
2)If key can be just 64 bit long really then how much time does it takes to crack a 64 bit key if plain text and cipher text is known

First, just to get this out of the way: the key size of DES is not 64 bits in any meaningful sense; only 56 bits are actually used in the encryption or decryption (see Wikipedia). The other 8 bits are for parity (error checking). In practice, they are usually ignored.

The openssl utility's enc operation (or, simply, the enc command itself) expects a passphrase, not a key (unless you use the -K option). The passphrase is turned into a key (of the appropriate length) using a key derivation function. This happens internally; you the user don't really control the process.

Breaking DES is actually pretty easy, due to its 56-bit key length being short enough to brute-force search (trying every key until one works). Modern personal computers will still take months (possibly only weeks for the fastest) in a typical-case scenario. However, dedicated hardware designed for the purpose, or a distributed network of computers, can break a DES key in under a day. Bear in mind that those machines are now a few years old, and Moore's Law is extremely applicable to this kind of brute-force operation; what took tens of thousands of dollars and a day of work eight years ago would today take under a thousand dollars to build, or only a few hours per key.


Is it a bad idea to forward a suspicious email with an attachment - even to a secure address for analysis? The more servers the message passes through the more potential of the attack to execute successfully? (0-day etc). Or is this paranoia?

Instead: Delete, disconnect from network, and phone security team?

Servers passing messages is really not a problem.

Your mail client has already opened the email so you forwarding it on is not opening your machine up to any further risk.

The mailbox to send these things to is usually on a machine that has strong security and sand boxing capabilities (or can be created on the fly as a VM)

If concerned, always delete, however forwarding on to your analysis team is essential if they are to understand it.


Storing SAML Idp certificates in APP_Data folder We provided a SAML2 Service Provider to our client, so they can use that to configure their Identity Provider to our Service Provider.

My questions is about our client certificate, I'm worried that if I stored their certificate to our APP_Data folder so that it could be accessed in our codebase for configuring both Service Provider and Identity Provider. So I'm thinking to setup an new bucket in our AWS and stored those certificate in that bucket. Any suggestion to this approach or is there a better way to at least secure the certificates?

Which certificate is this? Is it just the public key used to validate the token?

If it's just the public key, then the only thing you need to be worried about is someone overwriting it with a different key.

If you're that concerned with it, you can instead store it in the certificate store, and ACL it such that it require admin permissions to change. The practical issue is that you'd then need to store the thumbprint or subject of the cert somewhere so you can look it up in the store. Storing it in AWS would make it slightly more difficult for someone to mess with it. You'd want to make it readonly from your service so an attacker on the box can't use the same creds to overwrite it.

On the other hand if you're talking about the private key used to decrypt an encrypted token... that's another story. You really need to consider what sort of attacks you're looking to prevent.

You can store it locally on disk and anyone with read access can, well, read the key.
You can store it as a PFX file on disk with a password, but you need the password to decrypt it, and that needs to be stored somewhere securely, and down we go the rabbit hole.
You can ACL the key so only the web app can read it, but odds are it's running as network service, so that's pretty open.
You can switch the app to use a named account with password, and ACL the file so only that user can read the key, but an administrator can still read it, because: admin.
You can store it in the certificate store, but the previous 2 points still apply.
You can store it in AWS, but this requires a key to connect, and that key needs to be stored somewhere securely. Again down the rabbit hole. Additionally, this key needs to be pulled down to be used, which can either be stored on disk or in memory. On disk makes this all moot.
Lastly, you can store it in a proper key container like an HSM (see e.g. Amazon HSM, or Azure Key Vault) where the key never leaves the store, and you send encrypt/decrypt commands to it. At this point you still need a way to connect, but you can't extract they key, though you can send arbitrary data to decrypt.

This is where a threat model comes in. You evaluate the risks associated with each method, and only then can you determine whether the risk justifies the effort and impact of method.


Patterns for searching a source repository for private data I recently discovered a case where a colleague had accidentally committed their login credentials (host, username, and password) to a local source code repository, and then pushed these changes to a public repository on GitHub.  Of course, this was not an isolated incident – a few years back, GitHub killed its full-code search feature after people discovered hundreds of private keys and other credentials in public repositories.

I'd like to make sure that this sort of thing hasn't happened in the past with any of our other public-facing repositories (and, in case it has, to scrub the private data, change the exposed passwords, revoke the exposed keys, etc.).  It's no problem for me to cobble together a shell script to pull past commits to a given Git or Subversion repository so that I can scan them for private data.  But what sort of filename and text patterns should I use? For example, I want to catch files whose name suggests that they contain private keys or credentials (password.txt, id_dsa, id_rsa, secring.gpg, .netrc, and probably several more standard ones that I'm forgetting or am not even aware of).  Is there a list somewhere covering the most common cases? Similarly, I'd like to scan the contents of text and source files for patterns that indicate hard-coded login credentials.  Perhaps someone has already produced a list of regular expressions to start from?

The important files vary by programming language, and environment. For example, if you're running nginx, .htaccess files, by default, won't affect the behaviour of the server. However, those same files could really mess things up if someone loaded your application into an Apache environment. Therefore, you need to customise any list to your own needs.

There are some files which are probably always considered sensitive though:

Private keys (id_rsa, id_dsa, *.pfx)
Shadow files (/etc/shadow) - if you're checking these into source control without a very good reason, you're doing something wrong!
History files (.bash_history and similar) - these often have passwords which were mistyped, or used in command lines for interactive tools stored
Log files (/var/log/*) - again, they often have details you might forget to look for in

More specific files that shouldn't be in source control:

.htaccess, .htpasswd - Apache directory specific configuration files
web.config - IIS directory specific config file
wp-config.php - Wordpress config
sites/*/*settings*.php - Drupal config files
*.jks - Keystore files
and so on...

Github have a good sample of gitignore file contents, although these also cover things that shouldn't be in source control due to other reasons (e.g. compiled output shouldn't usually be in source control, due to not being source...)


Preventing Session Fixation: MAC or Hash? This blog post describes a method for preventing session fixation attacks (in ASP.Net in particular).  The idea is that the session id should be tied to the user's identity in a verifiable way, which means that a given session id can't be valid for both the attacker and the victim.  Their construction uses:

select random nonce
session_id = nonce || MAC(nonce || username).

For an application that runs on a cluster, distributing or changing the MAC key can be inconvenient.  What is lost if the MAC is replaced by an unkeyed hash?

select random nonce
session_id = nonce || Hash(nonce || username).

It does mean that an attacker can generate session ids that are valid for a given identity, but it's still true that a session id won't be accepted for both attacker and victim.  Is there a weakness with this approach?

Q1 - Preventing Session Fixation - MAC or Hash?

Q2 - What is lost if the MAC is replaced by unkeyed hash?

Q3 - Is there a weakness with this approach (of replacing MAC with a hash)?

You identified the drawback of key management with using MAC across multiple servers whereas with unkeyed hash the only drawback is increased risk of session forgery due to the public nature of a hash function.  So in terms of the more secure option, the MAC option is better for preventing forged sessions.  However, in both cases as you pointed out, session fixation can still occur.  Other approaches I have seen to strengthen the session id is to adjust the session expiry and to include the user agent as input to the MAC function. I think Q1, Q2, and Q3 are partially asserting or assuming that using MAC in session id creation is the "solution" to preventing session fixation. I think that it is important to be mindful of a comprehensive set of strategies to prevent session fixation. However, a very important issue in the ASP.Net environment is to not reuse old session ids after a user which is a huge vulnerability for session fixation exploits.

See https://support.microsoft.com/en-us/kb/899918


What to do if caught in a physical pentest? I've seen a lot of people talk about how to pentest and how NOT to get caught during engagements but have a hard time finding "How to behave when caught during a Red Team engagement".

Red Teams are to simulate adversaries attacking systems. Many actions can't be done (or at least very hard to) with just some computers and Red Teams often have to go on-site and break in (legally). What I've seen so far is people succeeding in not getting caught. However, I haven't seen anyone talk about what to do when caught. It may just be some suspicion or even being chased by security (possibly armed).

In cases where a Red Teamer is caught during an engagement, what should he/she do?

Say "I'm a security tester. You've caught me so I'll just leave."
Run away like a criminal with their stolen data (which sounds fun but dangerous) to be more like an actual criminal attacker
Contact the employer to report it and get a "just continue" pass
Quietly come along for some possible interrogation (I think this would be the safest)

Update: I've made another question here which covers the 3rd parties not discussed in this question.

Always have your slip with you!
This is the golden rule of Red Teaming! If you don't have your Permission to Attack with you, it's like driving without a driver's license. That said, if you are caught during an engagement, I recommend the following:

Present a forged Permission to Attack. This way, you can see if criminals could possibly trick a security guard to letting them do their thing with a fake Permission to Attack.

Present the real Permission to Attack. If a guard has not bought your fake slip, then it's time to hand in the real slip. If the guard believes you, it's time to pick up and leave the perimeter. A real attacker would have been stopped at this point. If the guard did not believe you, ask them kindly to talk to their supervisor. If they insist on not believing you and calling the police, so be it. You're not a criminal, so don't worry about it.

Follow the police's orders. They'll take you with them to the station, where you can explain to the police that you are part of a Red Team Engagement, and that you have a permission to break into the company. They will double-check that, calling whoever is listed as the person who signed your Permission to Attack. In the happy case, they'll pick up the phone, explain that you are really hired to do that, and you'll be free to go.
In the not-so-happy case, they won't pick up because it's 4 in the morning and their phone has no battery. Should this happen, you will probably spend the night in the police station. Worse things have happened. Call your employer in the morning, and they will reach the contact at the customer's company for you.

What about the other options?

Saying "I'm a security researcher. You've caught me so I'll just leave"

will not be very helpful. In the eyes of a security guard, you're a criminal, caught in the middle of a crime. You will not have the choice of "just leaving".

Run away like a criminal.

A very bad idea. Probably the worst you could do. If the guard calls the police (they likely will), the costs could rise quite a lot and it would not make the customer happy to know they now have to pay the police for an unnecessary manhunt as well. However, you should absolutely include in your report if getting away from the perimeter after getting caught would have been a trivial effort or not.

Contact the employer to get a "Just continue pass".

That would miss the point of a Red Team Engagement. Once you have a "Just continue"-pass, you are not simulating how a real attacker would act. You would just go through the stuff of the company with their permission.


Encrypt long text with 2048 bits I'm following advice from here about how to encrypt a long text in Java.
As encrypting a long text directly using a 2048 bit key is not possible, the advice is to create a 128 bit key encryption key, then use it with AES to encrypt the long text. Then use the 2048 bit key from keystore to encrypt the 128 bit AES key and send/save the encrypted key and the long encrypted text together.
My question is: Isn't the weakest link in the chain the 128 bits? Why can't an attacker (brute force) try to break the 128 bit key directly and just decrypt the long text with it? How to encrypt the long text with 2048 bit key so that the long text would have 2048 bits of security?

No, 2048 bit RSA key is weaker than 128 bit AES key. It may sound that the RSA key should be stronger, because it has more bits, but that is not the case. The reason for that is RSA key has to be product of two prime numbers while AES key can be completely random.


Is it alright to put .htpasswd in the protected directory if there's just one user? I'm setting up a directory on an Apache server protected with mod_authn_file. The docs for AuthUserFile say not to put the .htpasswd file in the protected directory:

  Make sure that the AuthUserFile is stored outside the document tree of the web-server. Do not put it in the directory that it protects. Otherwise, clients may be able to download the AuthUserFile.

It's not clear to me exactly what this means. Is it only authorized users that may be able to download it, or is there a risk that others may have access as well?

The directory I am protecting will only ever have one authorized user. He already knows his username and password. Is there any reason not to keep the .htpasswd file in the protected directory right next to .htaccess?

By default, Apache is suppose to deny access to .htaccess and .htpasswd files. For some weird reason, you have the ability to disable it (accidentally or otherwise) in your configuration. If you have disabled this protection somehow (or if, for some reason, it wasn't setup in the first place), then any malicious attacker could exploit this to download the .htpasswd file.

Normally the suggestion is to have the file outside of your web directory, preventing it from ever being served to a user, while still giving Apache access to it.


How Were FireEye's Tools Exfiltrated? What was the mechanism for exfiltrating FireEye's redhat tools in the recent SolarWinds hack? I understand it was via HTTP (small packets to many servers)? Are there any further details?
Is this a likely ongoing systemic threat or can it be mitigated? If malware is active for long enough, can it split confidential data into small packets and post via small http requests to many IPs, 'slipping by' DLP?
I am assuming FireEye's DLP was about as good as you could reasonably expect.

From what it seems like it was a Nation State Attack in which Solar Winds was compromised and some of their codebase in certain listed products were trojanized so that when customers patched and updated Orion in their environments they had unknowingly introduced malicious code that could basically take control of network devices that were managed by the malicious and trojanized product. The C2C traffic was disguised (and highly evasive), as traffic related to the Orion Improvement Program (OIP).  After this the attackers were able to engage with the target, perform reconnaissance and take advantage of other weaknesses within the environment to potentially try to exfiltrate data and maintain persistence.  It was an attack that even used steganography.  If FireEye hadn't caught it, it could have gone on much longer.
They have a really good article about it here where they list of detection opportunities:
https://www.fireeye.com/blog/threat-research/2020/12/evasive-attacker-leverages-solarwinds-supply-chain-compromises-with-sunburst-backdoor.html
CISA Issued this emergency directive https://cyber.dhs.gov/ed/21-01/
Orgs running the vulnerable version need to shut down their Orion and rebuild their Orion and in some cases their net environment.  Patching Orion to the updated level is not enough.  Only though a rebuild and patching and credential changes can orgs get back to safety.  Additional monitoring is wise.
Fireeye Published their Red Team tool Countermeasures as a result and have made available on github:  https://github.com/fireeye/red_team_tool_countermeasures
It is estimated that 18,000 customers potentially affected by the incident, although not all of those were actively being targeted.


encryption key rotation I am creating an web application which processes user information. As a part of application we are  storing this input user information and generated output on server. Since this input and output is sensitive, I am encrypting it using AES 128 bit symmetric key for encryption.

I want to manage the key and its rotation policy. Can somebody help me or point me at the key rotation article.

Where the key should be stored in database or file?
In case of key rotation should I re-encrypt all old input-output files?
How should key rotation be achieved?

With symmetric encryption, the level of confidentiality depends on the protection of the keys.  Storing the keys on a file is easier than storing the key in a db but you'd have to have protections for the file.  Database can provide a more secure storage facility but then you have to manage much more than the contents of the file.  When you take into consideration key rotation, storing keys in a database will likely reduce headaches of key management in the future.
It depends - if you have the facility to maintain multiple keys and a way to tie back ciphertext to key, then you wouldn't have to decrypt and re-encrypt data with the new key.  This method also requires some sort of key store (see above).  However, if you want to maintain just one key, each key rotation will require you decrypt data with the old key and
reencrypt with a new key.
Again, it depends.  I'd recommend you figure out how important the data is to an adversary now and over the course of its useful life and how much effort you want to exert in this process.  I wouldn't recommend rotating keys more frequently than necessary for the sake of security.  That'll lead to a management nightmare and you could lock yourself out.

NIST has a great write-up on this topic.  The NIST docs lays out what, how, and why.  Since the NIST docs and frameworks are published for government use, it's very comprehensive and well structured.

Check out NIST's Special Publications for additional details:

SP 800-57 (Recommendations for Key Management) 800-57
SP 800-130 (A framework for designing cryptographic key management systems)
800-130


How much does a security audit cost? For a PHP CMS, what should I expect to budget for a security audit, both whitebox and blackbox? The codebase is about 85,000 LOC ("Lines of Code") and I would probably use a North American company for testing. I really have no idea if an audit would cost $10-20k or well over $100k. I'm not asking for an exact quote, just a general guesstimate so that I know what to expect. If you could separate your estimates between blackbox and whitebox testing, that would also be helpful.

Edit:

I'll try to list as many factors as I can.

Type of app: A web content management
system similar to Wordpress, Joomla,
or Drupal.
Types of testing: Broad penetration
testing and a scan for common
vulnerabilities. Code review for
additional vulnerabilities as the
source code will be publicly
available.
LOC: Roughly 85,000.
Languages: PHP, JavaScript.
Audience for report: Developers.
Location of testing: It can be done
remotely.
User roles are variable. They are assigned to groups and each group can be given any number of permissions. Any number of groups can be created.

I don't know what other information might be relevant. Really, I'm not looking for an ultra-specific number, just a ballpark figure like, "Based on the info you gave, you could probably expect to budget between $X and $X for a security audit." Even just a baseline price would be extremely helpful as I really have no idea what to expect.

Here's my rough guesstimate:

Code review:

1000 LOC = 1 hour
85000 LOC = 85 hours

Hourly rate: 100$/hr

85 hours * 100$/hr = 8500$

If your software uses an ORM and a well documented MVC framework, it can speed up the code review significantly.


POST Method Anti-CSRF Help With Burp Suite I am testing a web application and encountering anti-CSRF tokens within forms which is hampering fuzzing attempts using Burp Suite intruder. An anti-CSRF token appears as csrf-token within a HTML meta field.
On submission of a form, the same token is URI encoded and sent as an authenticity-token parameter within the POST request body.

The application terminates the user session if the token is invalid, which is the case when running Intruder as the token is single use as expected (not for the life of the user session).

Although it seems straightforward, I've not been able to set with the guides below:
https://support.portswigger.net/customer/portal/articles/2906338-using-burp-s-session-handling-rules-with-anti-csrf-tokens
https://plusplussecurity.com/2016/11/14/csrf-tokens-using-macros-in-burp-suite-iii/

Selecting the request and the appropriate token does not seem to do much but testing the macro results in a new response with a fresh csrf-token.
Particularly, authenticity-token is not updated between each request and there is no evidence of the session handling rules working.

I would appreciate a good guide or assistance on this, as I'm not sure what I am missing at this point.

Here is the route I took eventually, thanks to the video at https://www.youtube.com/watch?v=GWLMmkv3z38:

Identify the anti-CSRF token (e.g. hidden csrf_token field within
the form)
Go to Project options > Sessions > Add to record a new macro.
In Macro Recorder, select the HTTP request from the proxy history
and click "OK".
Select the Macro Item and hit configure item.
In the bottom right, Add Custom Parameter, highlighting the string
that will be extracted for use as the anti-CSRF token. The 'Define
start and end' fields should be updated with the appropriate
search terms. Populate the Parameter name field, with the description like
csrf-token.
Add session handling rule.
Add a rule action (Run a macro) from the dropdown list.
Select Update current request with parameters matched from final macro response.
Select update only the following parameters and enter the name of the parameter e.g. authentication_token.
Finally, click the Scope tab within the Session handling rule editor and    configure scope to the Burp Tools, Parameters and scope URLs desired.

I spent more time than I wanted on this issue so I hope this helps someone save some pain; if it's not clear let me know and I'll try to fill in the gaps.


What bits of data is revoked in a CRL? Is it the hash(CSR) or hash(CSR + PKI signature) My understanding is that a CSR contains the public key from the client, and the response is a signed version of that CSR, with certain attributes (EKU, expiration) and other attributes the CA decides to arbitrarily add.

When a key is revoked in a CSR, is the hash of the public key mentioned in the CSR or just the PKI-stamped version of the Public key?

For argument's sake, suppose I want to use the same RSA pub/private key pair for many usages, even though this may be a bad idea in reality.  As a result I generate 3 certs, with the respective EKU below

Signing
Authentication
Encryption

Will a revocation affect all three usages of the same private key? Or will the revocation only affect a given cert?

Based on this answer from Thomas Pornin:

  Revocation is on per-certificate basis. What is revoked is the certificate (i.e. the binding between your key and your identity), not the key. Of course, if several CA have issued certificates that bind your key to your name, and your key gets stolen, then you would like all these certificates to be revoked. However, if one CA revokes your certificate for some other reason (e.g. you failed to pay a yearly "certificate maintenance" fee), then this has no impact on the revocation status of certificates issued by other CA, even if they contain the same name (yours) and the same public key (also yours) as the revoked certificate.


Limitations of TLS Inspection when using Diffie-Hellman It is my understanding that TLS Inspection can be carried out of two different ways:

1st Approach: Using a proxy server which negotiates TLS with both ends.

In this scenario, there would be 2 different TLS channels (Client <-> Proxy <-> Server)
If non-standard TLS implementations are used by the client/server or specific requirements need to be met in TLS negotiation, the Proxy would need to have specific features enabling that.

2nd Approach: Using an intermediate element containing the secrets needed to decrypt TLS shared key negotiation (e.g. private key associated to server's certificate) with the ability to re-encrypt (by using server's certificate) and forward the traffic to the server.

In this scenario, Client and Server would not see the intermediate server and it would essentially be a unique TLS channel, even if the intermediate element can see and block suspicious traffic.
Intermediate element would not need to implement further features, as it only acts by listening and forwarding/blocking.

When a cryptosuite implementing DH or ECDH is used over a TLS communication, is it possible to make use of the 2nd Approach? From what I know, Key Exchange in DH makes infeasible for an "eavesdropper" to figure what the shared secret is, as RSA private keys are only used as a mean to authenticate the server.

Depending on the cryptosuite used, would TLS Inspection be restricted to the 1st Approach?

Both of your approaches essentially describe the same thing: There is some intermediary system which terminates TLS and you have essentially two TLS connections: one from this middlebox to the client and one from the middlebox to the server.

The only difference is that in your second scenario the middlebox has access to the certificate and private key of the server and thus can fully impersonate the server without changes to the client. While you don't describe the first case in more detail my guess is that in this case certificates gets created dynamically by the proxy and signed by a CA explicitly trusted by the client. This first case can be usually found at a firewall which should give a client access to many sites on the internet while the second case is used to protect a specific server (for example by using a web application firewall).

Since in both cases you have two separate and independent TLS connections there is nothing special with DH, i.e. both connections can use DH key exchange.

But, there is a third scenario where an intrusion detection system (IDS)
gets the traffic exchanged between client and server and should passively detect attacks against this server, i.e. without being the TLS endpoint. In this case there is only a single end-to-end TLS connection between client and server. And to decrypt the traffic for analysis the IDS needs traditionally have access to the private key server - but this works only with RSA key exchange. To make such passive analysis possible with DH the IDS would need to have access to the master secret of each new connection, i.e. somehow the server must provide the IDS with this information. And this is far more complex than just providing the IDS with the static private key of the certificate.


Anomaly intrusion detection Anyone know which opensource network IDSs are using anomaly techniques? My project is to compare them, so if you could give me some hints, that would be much appreciated.

Currently, I am looking at Snort with SPADE and Snort.AD pre-processor. Bro also claims to be able to detect novel attacks (anomaly detection function), but I can't find any documents explaining how Bro achieved that.

That's funny, I was just reading about SPADE yesterday because I had a similar question. I'd be curious to hear more about what you come up with.

Meanwhile, I can point you to some other resources. There has been some talk about anomaly detection on the dailydave mailing list over the past two months, but no particular tool or technique stood out and there was a bit of a blast over a missing video. Perhaps worth checking out, though.

My introduction to anomaly detection was around a decade ago via Ourmon (free, open-source software) and Lancope (commercial). Shortly thereafter, SPADE and Bro started to appear on the scene as potentials in the open-source space. I had always been curiously into NetFlow data as an anomaly source, and today you have FOSS tools such as FlowMatrix or FlowBAT if you prefer the CERT SiLK tools.

I have run across one use of Bro (via the SecurityOnion project) to perform DNS anomaly detection. Typically, others are integrating their detective capabilities into Splunk (as there are many books written) -- and in particular, the Prelert toolsuites are focused in that direction (with demos, much like Splunk). To that end, I also found this page on syslog anomaly-based log analyzers. Finally, a tool called Skyline appeared interesting, albeit starting from a more ITOps perspective.


Can a user have a certificate or does only a machine have a certificate? If say you want to set up a PKI and use EAP-TLS to authenticate all users and computers.

A computer's certificate is stored on that computer. But how can a user have a certificate? Where is it stored? What if a user logs in from another computer, how will he provide his certificate for client authentication?

Personal Certificates are very common in a PKI infrastructure. And I would say that most often it's stored on the computer of that user/person.

It would of course be preferable to have the certificate and private keys stored on a smart card as mentioned in another answer but there is limited support for smart cards in for example smart phones.

One way to do this is to have unique keys and certificates (with the
same DN) on several devices. Then you can easily revoke a certificate
if a device is stolen. The problem with this approach is for example
encryption. If there was an email encrypted with the public key
corresponding to a certificate on your smart phone, that email cannot
be read on your laptop and vice versa.
One middle ground would be to have different certificates on every
device but with the same private key, with this approach you can
still read encrypted messages on all units. But if one unit becomes
compromised you will need to revoke all certificates.
And the simplest solution is to have the same certificate and keys
distributed to all your units. This have the same drawbacks as the
option above.

These certificates and keys are stored differently depending on OS and application. MacOS stores them in the key ring for most of the applications. Firefox can stored them in its own key ring, and so on.


Modern SOHO router protection Much like the entropy 'myth' of '1337' speak passwords compared to overall password length, it seems that multiple modern 'PC advice' articles on the web still suggest hiding SSIDs and as MAC filtering.

Aside from those dubious points, they suggest using strong encryption (WPA2-PSK), changing the default router admin details (obviously) and the default router IP (to mitigate against drive-by internet exploits) but after reading - multiple - recent - answers, it seems every other bit of knowledge I previously had about router protection is completely wrong. It also seems that most of this knowledge has been known for over a decade...

I see plenty of answers explaining why these are bad, but few suggesting how to effectively remedy them.

What are the modern methods of protecting SOHO routers - from the technical to the social engineering point of view - against internet and wireless attacks?

Are there other methods of encryption that can be used on top of WPA2-PSK? Should NAS devices on such wireless networks be considered compromised? How can router firewalls be used to mitigate such attacks?

Things that you should do:

Update the firmware
Use a strong password and WPA2 for wireless connections.  Don't use WEP.
Change the SSID of the router to something non-standard.  This is because the SSID is used as a salt for the wireless password, and having a non-standard SSID protects against an attacker using rainbow tables generated for standard SSIDs.
Disable WPS - it is inherently insecure.
Disable remote administration.  This includes access from any mobile apps - do you really need to remotely change settings on your router from an app?
Use a strong password for the router admin webpage, and if possible change the admin username.  I use a long random password saved in a password manager.
Remember to log out of the admin webpage when finished.
If possible, enable HTTPS for admin webpage connections.
Disable uPnP.  Review any other services that your router may be running (FTP, SMB, telnet, ssh) and disable if necessary.
Be very careful about opening up ports for incoming connections.

Things that you may consider:

Set up a guest network for visitors to keep them segregated from your main network.  If your visitors can only access the internet and not any other machines on your network this should prevent them from infecting any of your devices with malware.  This also means that you don't have to share your main WPA2 key.
Use a non-standard network range (i.e. not 192.168.0.x or 192.168.1.x).  This may help against a scripted attack that uses a default network range and it will help in future if you ever need to set up VPN connections.
Turn on logging and review the logs.

Things that are a probably waste of time:

Hiding the SSID.  This is security by obscurity, and finding a hidden SSID is an easy task
Use MAC filtering.  A determined attacker can easily spoof a MAC address.


Content-Security-Policy hash of script <?php
header("Content-Security-Policy: default-src 'sha256-".base64_encode(hash('sha256', 'console.log("Hello world");', true))."'");
?>
<script>console.log("Hello world");</script>

However I still receive in Chrome:

  Refused to execute inline script because it violates the following
  Content Security Policy directive: "default-src
  'sha256-1DCfk1NYWuHM8DgTqlkOta97gzK+oBDDv4s7woGaPIY='". Either the
  'unsafe-inline' keyword, a hash ('sha256-...'), or a nonce
  ('nonce-...') is required to enable inline execution. Note also that
  'script-src' was not explicitly set, so 'default-src' is used as a
  fallback.

I've toyed with this for over an hour but still am unable to generate a hash that matches examples eg.

http://software-security.sans.org/downloads/appsec-2014-files/building-a-content-security-policy-csp-eric-johnson.pdf
Claims <script>alert('Allowed to execute');</script> (hard to determine original spacing) has hash of sha256-MmM3YjgyNzI5MDc5NTA0ZTdiCWViZGExZDkxMDhlZWIw NDIwNzU2YWE5N2E4YWRjNWQ0ZmEyMDUyYjVkNjE0NTk=

Which doesn't make much sense: the last part doesn't start with sha256-, but at least the first hash is the correct length. I get sha256-nbFv/38jW7zf8mQirwFemFjDwp5CwIaorxe4Z3yycn0= as the hash for alert('Allowed to execute');

http://nmatatal.blogspot.com/2013/09/how-my-script-hash-poc-works.html
Claims:
<script>console.log("Hello world");</script> should have a csp of
script-src 'sha256-y/mJvKQC/3H1UwsYAtTR7Q==' eyeballing it, that looks too short.

What am I doing wrong?

This is still incredibly confusing. I'm running Chrome 40 and like you I've just spent far longer than I would have liked figuring out what's going on.

The CSP 2 spec says this about hashing <script> elements:

  For example, the SHA-256 digest of alert('Hello, world.'); is YWIzOWNiNzJjNDRlYzc4MTgwMDhmZDlkOWI0NTAyMjgyY2MyMWJlMWUyNjc1ODJlYWJhNjU5MGU4NmZmNGU3OAo=.

I've managed to generate that hash by running:

$ echo -n "alert('Hello, world.');" | openssl dgst -sha256 | base64
YWIzOWNiNzJjNDRlYzc4MTgwMDhmZDlkOWI0NTAyMjgyY2MyMWJlMWUyNjc1ODJlYWJhNjU5MGU4NmZmNGU3OAo=

But this does not work in Chrome 40.

The editor's draft of CSP says this:

  For example, the SHA-256 digest of alert('Hello, world.'); is qznLcsROx4GACP2dm0UCKCzCG+HiZ1guq6ZZDob/Tng=.

The example they give is generated with:

$ echo -n "alert('Hello, world.');" | openssl dgst -sha256 -binary | base64
qznLcsROx4GACP2dm0UCKCzCG+HiZ1guq6ZZDob/Tng=

The addition of the -binary flag to openssl is the difference between the two commands.

This does work in Chrome 40 (stable) and Chrome Canary, but I believe it still has issues in Firefox 36.


Are MariaDB's encryption and compression functions secure? MariaDB [(none)]> SET @key_str = SHA2('Is it secure?',512);
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> SET @crypt_str = AES_ENCRYPT('cleartext',@key_str);
Query OK, 0 rows affected (0.01 sec)

MariaDB [(none)]> select @crypt_str from dual;
+------------------+
| @crypt_str       |
+------------------+
| ���5��!$�l           |
+------------------+
1 row in set (0.00 sec)

MariaDB [(none)]> SELECT AES_DECRYPT(@crypt_str,@key_str) from dual;
+----------------------------------+
| AES_DECRYPT(@crypt_str,@key_str) |
+----------------------------------+
| cleartext                        |
+----------------------------------+
1 row in set (0.01 sec)

Docs in MariaDB KB Encryption, hashing and compression functions, such as ENCRYPT, DECRYPT, COMPRESS, PASSWORD. MariaDB KB doesn't write about how secure those functions are.
Do you recommend a cloud native app (12 factor) to offload Encryption to the database (in my case MariaDB)? Can my example code be improved to be more secure (but only with MariaDB functions)?
I found a (maybe) biased review of a product vendor MyDiamo (they wish to sell):

Why MySQL Internal Encryption Functions are not Sufficient
1. MySQL Internal Encryption Algorithms are not Safe
MySQL provides many algorithms such as AES, DES, SHA-1 and MD5
algorithms. Among these algorithms, MD5 and SHA-1 are proven to be not
safe as they have been cracked. Also, for AES algorithms, the current
MySQL 5.6 version generally available provides only the ECB operation
mode which is also proven to be unsafe.
2. MySQL Internal Encryption Functions does not Provide Key Management
When encrypting data, not only is the encryption itself important, but
also managing key is important. The importance of the key cannot be
stressed strongly enough when encryption matters. Most encryption
solutions however do not pay much attention to the key. When using
MySQL internal encryption functions, the key is exposed to the source,
thus no proper key management exists.

While MySQL currently provides other encryption modes, MariaDB appears to still only support the insecure ECB mode. In MySQL, you can set the mode to the slightly better CBC mode with:

SET @@session.block_encryption_mode = 'aes-256-cbc';

If you cannot do this on MariaDB, then it does not support any alternate mode. Even on MySQL where it is supported, CBC is not a great mode. What is really needed is authenticated encryption. Since neither MySQL nor MariaDB support that, it should be up to the application using the DB.

The password hashing is also poor. By default, you are (unfortunately) encouraged by the documentation to hash the password with SHA-1 or SHA-2, which are fast hashes but are not designed to be a KDF. Due to the speed, they can be attacked very effectively. This danger is amplified because no salt is used by default. Unless the "password" is a randomly generated string, this is inadequate for security. You must use a secure KDF, which is not natively provided.

As for the compression functions, what would make them secure or insecure? They use whatever compression library MariaDB has been compiled with and just return compressed data. Security in terms of arbitrary code execution should lie primarily with the linked library being used (e.g. zlib).

  Do you recommend a cloud native app (12 factor) to offload Encryption to the database (in my case MariaDB)? Can my example code be improved to be more secure (but only with MariaDB functions)?

Unfortunately, product recommendations are off-topic for this site. You should design the application which uses the database such that it supports solid, authenticated encryption with a strong KDF. If you merely need to encrypt all of your database with the same key and only care about data-at-rest security, you can simply put the database files in an encrypted directory.


UPnP Open port on local device? I ran Zenmap on a local IP address and found that it had one port running UPnP open and the guessed OS was a Windows distribution. Is this normal? Should I be concerned about it? In what ways could it be exploited if it is a risk?

Yes an open UPnP-port is a potential vulnerability. There is a broad range on how to exploit the UPnP-protocol.Read this link for more information: UPnP Networking Flaws Expose Tens Of Millions Of IPs To Hack Attacks
Read more at http://www.silicon.co.uk/workspace/upnp-attacks-over-networks-rapid7-security-105786#Wi3XEMHgsbw0VXKw.99
If you don't need it to be open for some reason close it.


Is nc -l unexploitable? Suppose I want to set up a server as a test, and I want it exposed to the Internet, giving any IP access.  It could be for any reason or no reason.

I don't want it to be at risk of a buffer overflow and the execution of arbitrary code on my machine.

nc -l is fine. the server needn't do anything.  It might be done within cygwin.

How safe is that?

And if it's not safe, is there something safer?

Netcat does not appear to have much bugs such as buffer overflows. Apparently it had one such bug, but a very minor one: if you invoked it as a client with a port number beyond 999999 (which makes no sense, since valid port numbers are in the 0..65535 range), then you could get a buffer overflow leading to a crash; bug report is there.

However, there is apparently more than one nc command around. On my server, which runs a pretty standard Ubuntu (13.10 currently), the nc command is provided by the netcat-openbsd package. Apparently:

The OpenBSD people decided to put a nc command in their base system, and rewrote the complete utility for various reasons, the official one being "IPv6 support" (that's a valid reason, but I suspect a case of NIH syndrome as well).
The Debian people then extracted the OpenBSD code to package it as a Debian package (which also found its way to Ubuntu, since Ubuntu feeds on Debian packages), and then proceeded to overhaul the code for reasons which are somewhat explained in this file. In particular, they claim that "the code has been massively cleaned up, and important functionality has been added". They also removed calls to strlcpy(), a non-standard function that OpenBSD people are quite fond of, but which is not part of the GNU libc that Debian systems use.

So whether a specific version has a buffer overflow is a hit-and-miss game. This is a rather simple utility; if any software may be reasonably bug-free, then that must be netcat. I suspect that if a buffer overflow is ever found in the OpenBSD implementation of netcat, then some OpenBSD developers would commit honourable seppuku.

However, there may be other security issues than buffer overflows. The main potential issue I see here is that your nc -l will run in some sort of text console or terminal, and output the bytes as they are received. Therefore, some bad guy could send control sequences which will be interpreted by your terminal. This issue is explored at length in this question.

If you run nc -l with output redirected to a file, then that problem is avoided -- as long as you take care to read the file with some tool who filters out escape sequences, e.g. a text editor.


Missing DKIM, but having digital signature, will this help mitigate ending up in spam folder? According to this mail tester: https://www.mail-tester.com

I am quite fine with sending emails from my domain's email info@vlastimilburian.cz

According to this mail tester: http://mailtester.com/testmail.php

I am also fine, regarding SPF.

And finally I am also listed in MailCleaner list:
https://www.mailcleaner.net/tools/test_spf.html

I am missing DKIM, though. My hosting provider just does not yet support it.

All that is looking good, but I was told by a mail specialist that since I don't have DKIM, my emails will probably end up in spam in some cases.

Note that I am signing each email with Class III digital email signature, does this help?

So far, I did not have problems, but I need a different opinion in this matter of securing my email and ensuring my mails won't end up in a spam folder.

A final note is that I send everything from Thunderbird, if that matters. And through TLS protocol that is.

While Vini7 is right about DKIM not being required, I'm afraid he is incorrect as to why.

First, understand that spam filtering is not a technique that is described in an RFC. This means that pretty much everyone does something different, resulting in a large fragmentation about what possible solution to adopt.

First, pretty much all systems works as a Bayesian decision-making tree with a few shortcuts built-in: you start with a base probability that a mail is spam (called credence) and feed that mail through a series of tests that will  update that probability. In the end, depending on the probability you end up with, the mail is rejected, accepted but flagged for further vetting (placed in the spam folder) or accepted as legitimate. The shortcuts are some of the tests that can, on their own, have the mail rejected altogether.

What test are used, exactly, vary from domain to domain but here are the most frequently used ones:

IP reputation check: this is typically done at SMTP level before the mail is even transmitted. The receiving server will check the "reputation" of the sender system IP address against one or more online databases and will typically (but not always) reject connections coming from IP that have been sending spam recently altogether.
Headers check: The headers that comes with he message are checked for consistency. This is where DKIM and SPF work: if setup properly, it can allow the receiving system to know if the sender email is valid or not. Again, depending on how everything is setup, it can result in an outright rejection or simply update the credence (positively or negatively).
Content analysis The full content of the mail is analyzed. How it is done varies a lot but, basically, this step typically updates the credence one way or another.
Rule-based filtering This is usually a last step where a user can specify keywords, senders or other properties of the email that will tell the system to treat it as ham, spam directly or update the credence.

(there are plenty of other tests that are performed in some situations and domains. I couldn't possibly list them even if I knew them all. Just expect some of them to be really, really bonkers).

After all check have been evaluated and, unless the mail has already bee rejected by a shortcut rule, the final credence will be compared with configured value: under a specific threshold, it will be rejected. Between this value and another threshold, it will be redirected to the spam folder and higher than that it will be treated as legitimate.

Some system have a last step: the user can manually mark a message as "ham" or "spam" and the system will use that information to update its analysis engine: it will attempt to learn from the result of its estimations.

So, how do you optimize the chances of your mail not being sent to the spam folder?

First, be sure to have a "good" IP reputation for your outgoing SMTP gateway. You can easily check the reputation of your IP using SMTP blacklist checker services (for instance, this one or this one). If your IP is in a blacklist, make sure you understand why (and, if possible, fix it).

Second, make sure your infrastructure is 100% clean: many system have weird or dated rules that will, for instance, reject your mail if the sending system does not have a reverse DNS entry or if that reverse does not match your SMTP MX record (the fact that this is idiotic does not prevent people from doing it).

Third, use all possible ways for your recipient to check that your mail is legitimate: if possible, use SPF and DKIM. If you can't do both, do at least one. As you mentioned, digitally signing your mails will sometimes help. Avoid sending the same mail to a large number of recipients. make sure that your return path (the email address that would be used if someone replied to you email) is somewhat valid (at least make sure the domain resolves and that it has an MX record in your DNS).

When all else fails, contact the admins of the recipient's domain and ask them for assistance: it is their job to make sure legitimate email is not misqualified.


How to ensure that files uploaded by users do not contain any Javascript I'm trying to make a file uploader using PrimeFaces, but it is vulnerable to scripts: If someone tries to upload a file (e.g. a résumé) that includes JavaScript, this could get executed once the file upload is completed.

Are there efficient ways to detect JavaScript code? I know parsing the file can detect the code, but I consider parsing every file that's uploaded a bad idea because it consumes processing power.

You should not be attempting to do this as there is no way you will detect all malicious script. There are various ways of hiding it as it isn't all nicely presented within <script> tags. Take this one for example:

<BODY onload!#$%&()*~+-_.,:;?@[/|\]^`=alert("XSS")>

Check out the XSS Filter Evasion Cheat Sheet for some other examples.

If you are displaying the uploaded files in HTML, then you should set a Content Security Policy to prevent inline JavaScript and CSS from running. (Using CSS it is possible to embed JavaScript. Some versions of Internet Explorer support this)

The CSP can prevent inline script, but allow script to run in external js files from your server or from trusted sources (such as Google CDN).

You would need to make sure that you validate that the browser supports CSP first (you could implement a test for this in a page before displaying the content), and you could do this in combination with a HTML sanitizer that processes the page content only on output to the browser. This will be a belt and braces approach. You should use a well tested solution for the sanitization, such as OWASP AntiSamy. Vulnerabilities have been found in older versions of most sanitizers (even this one), but using it in combination with a CSP would prevent most attacks as this would give you the window of time to patch to the latest version.

The other option would be to add the Content Disposition header to any served content, so that it would be downloaded to the user's machine rather than displaying in a browser in the context of your domain. Any code running in the context of your domain can take advantage of the Same Origin Policy to attack users viewing the content. This would not be possible offline (Internet Explorer gives a warning message with confirmation prompt, and browsers such as Chrome and Firefox treat the file system as the null origin to stop the script accessing online content).