How man in the middle attack works on multiple targets? I'm finding hard to understand how it is possible to perform a man in the middle attack to a whole network.

It is easy for the attacker to tell every victim in the subnet that his mac address is the gateway's mac, so he can intercept all requests sent by the victims. But how can the attacker do the reverse: tell the gateway that he is all the victims simultaneously to get back their responses?

I know that it is possible since I've used some tools that implement that, but I can't figure out the internals details. Can someone enlight me, please?

Thanks in advance.

If due to ARP spoofing the gateway the clients send all traffic to the man in the middle, the MITM then forwards this traffic to the original gateway with its own MAC and IP as sender. This means that the responses to the traffic are send from the gateway to the MAC and IP of the MITM which then rewrites the response and forwards it to the original client. Thus there is no need to spoof the originals clients MAC at the gateway.


Why would hackers attack a DNS server with a DoS? I wake up this morning to a rebooted server. The DNS server was running at over 100%. After a little bit of work, I got fail2ban in place to block all of those requests.
The requests themselves are valid, jut repeated hundred of times per seconds. Once the block got many (hundred) of IPs, I can see that I am blocking 1 million UDP hits every few hours.
Is that just a [D]DoS attack? (probably considered dynamic since many computers are involved and once one was blocked long enough it looks like it stops the requests)
The one other possibility I can think of is that the attacker is trying to crash the DNS server and gain access when it restarts or crash the whole computer and attempt connections to other services. (i.e. in case you don't know how to get your firewall in place before you start your services)
Since my last firewall reset, here are my stats:

Hits: 2,346,742
Number of IPs: 473

It goes fast. Several hundred hits per second. The number of IPs doesn't grow much, however.

From @Schroeder's comment, I made some additional searches and found out much more about this type of attack.
I Was the Target!
First of all, it looks like I am the target of the attack. Why? Because the DNS Amplification attack means that the source port of the DNS requests is set to 53. This makes it possible to force my DNS to send a non-requested response to another server (see graph below). Since I have less than 0.1% of such hits using that port, I don't really think that my DNS was used for the Amplification, but rather that it was the target.
How does the attack work?
The attacker sets up a computer with software sending UDP packets to some random DNS servers asking for a domain name. The packet looks normal except for the fact that the attacker puts the IP address of a different computer instead of his IP address as the source. The result is that the DNS sends the reply to the wrong computer:
      10.0.0.1         10.0.0.2         10.0.0.3
+------------+   +------------+   +------------+
|            |   |            |   |            |
| Attacker   +-->| Some DNS   +-->| Me         |
|            |   |            |   |            |
+------------+   +--+---------+   +------------+
                    |      ^
                    v      |
                 +---------+--+   This step is not required, it happens if
                 |            |   your DNS is setup to accept recursive
                 | Master DNS |   requests (which is not a good idea)
                 |            |
                 +------------+
                       10.0.0.4

In the example above, the attacker's request should have 10.0.0.1 as the UDP origination address. But instead, the attacker changes his IP address with 10.0.0.3. As a result, when the DNS at 10.0.0.2 receives the UDP packet and is ready to send a reply, it sends the data to 10.0.0.3.
The Master DNS of the domain is going to be used if the DNS at 10.0.0.2 doesn't know anything about the domain name being queried.
IMPORTANT NOTE: This is true of all UDP services. DNS is probably the worst in terms of amplification, but NTP, for example, can also be targeted.
Why not just verify the Source Address?
It's not possible from 10.0.0.2 because UDP has no recollection of the real source of the packet.
What is possible is for ISPs to verify that all the packets going out of a given computer has the IP address of said computer. Some do, I think, but most probably do not. It has a small speed impact... which of course with a DNS amplification is to laugh at (the DNS amplification has a way worse effect on the Internet than a small check of UDP packet origins).
The other thing may be that a user may still be able to do by connecting to the Internet in such a way that he bypasses the ISP verification. I do not know whether and/or how that would be possible, but I wouldn't be surprised that it could be done.
Actually, this is very problematic since the origin of such attacks is difficult to track and that's certainly why they are still occurring en masse.
Why is that a DDoS?
The packet to request a DNS record, which is what happens here, is very small, maybe 300 bytes. So the attacker only has to send one small UDP packet.
The response, however, may be multiple kilobytes of data. Specifically, there is a domain which is being used in these attacks:
peacecorps.gov

and that domain returns over 3Kb of data! (It defines many 'TXT' fields which are used in that amplification attack.) This is why it is called Amplification Attack.
As a result, the requests from the Attacker are transformed in about 10× bigger responses and the computer being attacked (10.0.0.3) gets swamped with a large number of rather large UDP packets.
Here I show the resulting entry in iptables. The first number represents the number of hits to my DNS computer after about 40 minutes (so over 10,000 hits a minute...):
pkts     bytes target     prot opt in     out     source               destination
61637  4376227 DROP       udp  --  eno1   *       0.0.0.0/0            0.0.0.0/0            udp dpt:53 STRING match  "|0a7065616365636f72707303676f76|" ALGO name bm TO 65535

Notice also how the string was fully transformed to hexadecimal numbers.
Wouldn't an HTTP request be even worse?!
HTTP 0.9/1.0/1.1/2 use TCP which is a two way protocol and no amplification can be generated with TCP. i.e. TCP breaks if you did not first properly connect with the full handshake.
HTTP/3, though, introduce the QUIC protocol which is HTTP over UDP packets. So far I have not heard of major issues with QUIC, but the protocol has changed a lot within the last 6-7 years and it is not yet widely implemented. Here is an article about the fact that QUIC has built in features to prevent amplification attacks, so there is hope that it has been taken care of.
Some people are talking about using HTTP to query for domain names. (DoH — Domain over HTTP). Hopefully that would not be implemented using QUIC...
How can I Fix the Issue?
The amplification happens because your DNS is setup to accept requests of domain names that you do not control (i.e. domains you're not the owner of).
BIND has an option that allows it to do amplification, which is called recursion in DNS parlance. This feature is now off by default, but you may have turned it on (by mistake?).
Here is the wrong setting:
allow-recursion { any; };

What you want is to use the correct settings:
trusted-servers { 192.0.2.4; }  // list all your trusted servers

allow-recursion { trusted-servers; };

This will make your server automatically deny all such requests. So if you are not peacecorps.gov and you receive such a request, BIND will just stop right there and write a note in your DNS logs about the denied request.
Could I block the IP from those requests?
Yes. I started by doing that because my server was running well over 100% in CPU time. However, it may not be wise to do so. From the picture above, you can see that your DNS server sits between an attacker and a victim. If you block the source IP address, it's not the attacker's IP that you are blocking, it's the victim's. This means you're actually preventing the victim from seeing your domain names and do legitimate requests. That's probably not what you want!
At first, I generated a log message from my firewall. If I were to detect 5 or more requests to port 53 (UDP) from the same IP address in a short amount of time (5 sec.), then I would block the IP address. To do so, I used fail2ban.
First, I have a filter which detects the links with UDP and port 53 as the destination:
# Filter: /etc/fail2ban/filter.d/named-fast-requests.conf
[Definition]
failregex = \sIN=[a-z0-9]+ .* SRC=<HOST> .* PROTO=UDP .* DPT=53\s

Second, I have a jail which gives the other parameters:
# Jail: /etc/fail2ban/jail.d/named-fast-requests.conf
[named-fast-requests]
enabled  = true
filter   = named-fast-requests
action   = named-action[scheme=all,period=year,reason=named fast requests]
logpath  = /var/log/iptables/iptables.log
maxretry = 5
findtime = 5
bantime  = 1036800

The main point here is the maxretry and findtime which are set at 5 times in 5 seconds or less. When that happens, I block the <HOST>.
You will want to update the action with your own thing. I use my own iplock tool here. The command I use in the action:
actionban = /usr/sbin/iplock -s named -b <ip>

By default, fail2ban uses iptables directly. Search their actions to see how it's done.
Don't Be the Source of Amplification
You should block requests where the source UDP port is less than 1024. These are not valid. If you offer a server, port 53 will be a destination port, not source.
iptables -I INPUT 123 -i eth0 -p udp -m udp --sport 0:1023 -j DROP

WARNING: replace the position (123) with the correct number!
This rule says that for any incoming UDP packet on eth0, which has a source port between 0 and 1023, drop the packet. This prevents someone from using your DNS server for amplification. However, it does not protect your own server from amplification from others. Note that even with the proper allow-recursion setup, you would not prevent an amplification attack if the attacker properly selects one of your domain names for the attack.
Note: if you specify a different nameserver to resolve domain names on your computer, you may want to open those connections before you block everything from port 0 to 1023. This would be something like so:
iptables -I INPUT 123 -i eth0 -p udp -m udp --sport 53 -s 8.8.8.8 \
                    -d <your-static-ip> -j ACCEPT

This is the returned data from the nameserver 8.8.8.8, which you probably want to receive. It is not likely that your provider or some other official nameserver would be the direct source of a UDP attack. If you do not have a static IP address, you do not have to include the -d option.
However, it is probably a lot better to use the ESTABLISHED feature which is now available for UDP messages (it has been for a while now, but I remember a time when that was not available...):
iptables -I INPUT 123 -i eth0 -p udp -m state \
      --state ESTABLISHED,RELATED -m udp -d <your-static-ip> -j ACCEPT

This means you'll automatically accept answers from your domain nameserver provider since the firewall will know you sent a request and you want to accept replies. This rule has to appear before the DROP rule above.
Drop Unwanted Requests Early
Obviously, having BIND accept all those requests with peacecorps.gov is not something anyone wants. You can actually block those directly in your firewall. This works because UDP packets are not encrypted so the domain name is visible.
Here is a rule one can use to block those domain name requests:
sudo iptables -I INPUT 123 -i eno1 -p udp -m udp --dport 53 \
           -m string --hex-string "|0A|peacecorps|03|gov|" --algo bm -j DROP

Obviously, if your DNS has any domain or sub-domain including "peacecorps.gov" then it should not use that iptables rule. For most of us, though it should be rare.
The --hex-string option allows you to specify a string. The way it is defined in the UDP packet uses a form of P-string (size + data). Since "peacecorps" is 10 character long, we put 0x0A just before it. Again, the "gov" is three letters so we use 0x03. If we were to use the string "peacecorps.gov", it wouldn't work since the period would not match the 0x03 byte. The first size is optional, though (although you'd match anything that looks the same such as "bestpeacecorps").
Having such a rule will save your domain name service a ton of totally unwanted traffic.
Update: Although the attack stopped about two weeks after I posted my question, the "peacecorps.gov" issue still happens about 10 times a day.
Source: https://defragged.org/2020/05/20/tips-and-tricks-blocking-dns-requests-via-iptables/
Debugging You DNS
In order to see what query your DNS server receives and replies to, you can run the following commands in your console:
sudo rndc querylog

Now look at the logs, usually here:
less /var/log/named.log

Look at the bottom (hit G in less) and you should start seeing queries from remote IPs. The logs include the domain name being checked. It's very practical, especially if you missed entering some of your own domain names in your secondary or tertiary DNS.


Is the javascript browser tool at printfriendly.com secure for my browser I use Win Xp Home with SP3 and am pretty aware of security problems. I use Palemoon 12.1, a version of Firefox optimized for windows.

I usually use printfriendly bookmarklet to create printable pages or to print pdf from web:

I was curious to test it on the Anubis executable checker and I know of these possible problems:

Performs File Modification and Destruction: The executable modifies and destructs files which are not temporary.
Changes security settings of Internet Explorer: This system alteration could seriously affect safety surfing the World Wide Web.
Performs Registry Activities: The executable creates and/or modifies registry entries.

In any case my doubt is especially that this javascript command uses iexplorer to work and not my main browser with limitated account: is this possible?

May it be unsafe for base security of my account and for my PC?

Read this report carefully and don't worry about it. This isn't on what URL does with iexplore.exe (IE) but what IE does with typed URL. Every website will bring you similar report (e.g. report for google.com).

I recommend you to check that URL with services that analyze URLs, like VirusTotal.


Sending and blocking emails from hashed and salted address ? (for GDPR) I would like to send emails from a server, but also make them unable to trace for me as server owner or attackers. I want this for GDPR but also to protect people from abusers.
Short info about service: I am a provider of a service as a person, not a company. Emails will be tied to product (invites and service content) with 0 marketing, there will be rate limits preventing spamming, emails will be triggered only by real users, users won't be able to send just any email, but rather use specific templates, nonusers will be receiving emails too (invites).
Will it be enough to just store emails hashed and salted with one system-wide salt value? My main concern is mostly nonusers as I can't have their consent before emailing them. So I could provide them a way to block abusers or all emails from my server with just storing hashed and salted email and comparing every request to send email against it.
Another problem is, how can I prove that some user gave me consent to receive past emails? Is stored hashed and salted value enough?
Do you know how other big services like Gmail, Mailgun, etc solved this?

You might be over-thinking the problem. In order to record consent, you need to store the PII. It is ok that it is stored un-hashed.
You appear to want to hash the emails as a layer of protection, but I would rather that a service spends time protecting access to this stored information. If an attacker can get access to the hashed emails, then you can assume that they also have access to the salt, and access to other data that would allow the attacker to guess or test emails. So, I'm not sure if hashing will ultimately do any good.
What big services do is to restrict access to the data.


How can I find a web application's document root using a path traversal vulnerability? I am performing a penetration testing on an application hosted on an Ubuntu environment.

So using a path traversal vulnerability, I can download any file.

The API web application runs as root (shadow and brute-force are already my friends). Funny situation: I can not find the web root folder.

What I have tried:

Search for logs that can lead me to the path. nginx or apache2 is not there.
Search for nginx, apache2 or other configuration files
Search for common directories of web roots (https://serverfault.com/questions/144598/where-should-the-web-server-root-directory-go-in-linux)
Bash histories of all users

What else should I try?

Use the traversal vulnerability to read

/proc/self/environ

This prints out environment variables among other thread information.

Look for a environment variable called DOCUMENT_ROOT


Does it really matter if I use the same password for all my most important sites, including banking? I only need to store passwords for personal use, no classified info here. I have a poor memory and I prefer not to record numerous passwords as that creates a security risk itself. I don't want to use any kind of IT-based protection / storage system. If necessary, I could come up with say 4 passwords, assign one of those to each of my many logins, and create a document (in Evernote so I always have it with me) listing the logins and providing a clue (decipherable only by me) for which password applies to each. Anything wrong with doing that? I realise Evernote is probably not secure but no one will be able to decipher my clues.

If, as you say, your password storage is decipherable only by you, I'd say it is good-enough encryption for you - in the sense that it serves the purpose.

However, using unique, complex and long passwords is not only for the purpose of someone stealing them from you. Your strategy works only if you reconcile to risks like these:

One of the websites you use is hacked - and they crack your password due to a weakness in their password storage scheme (or a weakness in your password).
The hackers then attempt reusing this password on other sites and succeed.

I don't know the why you're reluctant to use Password Managers. As Troy Hunt says "Password Managers don't have to be perfect, they just have to be better than not having one".

As an Infosec pro, I'm paranoid and skeptic (don't easily trust even reputed vendors). I use an open source password manager KeePassX. I try to stay aware of the residual risks, compensate with "constant vigilance" (thank you, Alastor Moody) where possible and accept the risks where it's not.

Of course, your needs might be different.


Hide clear SNI when using https A server with IP address a.b.c.d is hosting multiple SSL websites.

When connecting to each website, a proper browser sends a request which include the destination host in an uncrypted clear text known as SNI, such as example.com.

After a successful SSL connection, the rest of the data are transmitted with the corresponding certificate.

The problem is, the SNI sent by the browser will be loggable by any MITM, and thus blockable by a government.

How do I prevent the other end from finding out which website I'm visitting on the server?

(Obviously not by using VPN or proxies.)

It is not possible to hide the SNI information if the server requires it to serve the proper certificate. There was discussion on encrypting this information in TLS 1.3. But this idea was abandoned since this would require establishing an additional encryption layer and thus adding additional overhead to the connection establishment. Apart from that this information might leak anyway due to DNS lookups and of course also through the certificate the server sends, which is in plain too. Thus if you want to better protect your privacy you need to use an additional encryption layer yourself, like a VPN.

For more details see also

Why do HTTPS requests include the host name in clear text?
How is HTTPS/SSL able to hide destination website that one is connecting to?


Can network admin detect an MiTM attack? I am on a WiFi network with a SonicWall router (I'm guessing this comes with an IDS).

If someone on my network launches an MiTM attack on me, will the network admin be able to detect this?

Does an IDS/IPS/Network Firewall prevent such an attack?

Or do I have to be vigilant and protect myself against such an attack?

It depends on the location of the IDS relative to the network and on the capabilities of the IDS.

ARP or DHCP spoofing attacks are done inside the local network. From outside this network they look no different than normal connections. Thus in order to detect such attacks the IDS has to be inside the same network where the attack occurs. ARP and DHCP spoofing attacks can also sometimes already be prevented by better routers without needing an IDS.

DNS spoofing can be done inside the network or from outside. If the attack happens inside, then an IDS outside of the network will not be able to detect this attack, although it might in theory detect some results of the attack. On such results might be the attempt to access a specific web server on the wrong IP address which could be detected by comparing the Host header in the request against the target IP address.

If DNS spoofing is done from outside the network protected by the IDS, then the IDS might be able to detect such attacks, depending on how sophisticated they are.

Apart from the theoretical limits which kinds of attacks can be attacked because of the location of the IDS in the network, there are practical limits on detection capabilities which depend on the type of IDS, the specific configuration etc. Thus even if attacks could in theory be detected, a specific IDS with a specific configuration might not be able to do it.


How to exploit binary with no syscall gadget or information leak in full ASLR and NX? So I have a Linux x86-64 binary linked with libc dynamically running on patched Ubuntu 16.04 remotely.

The executable makes one call to libc read and exits, allowing simple stack overflow into ROP. That's the only code compiled into the binary.

-----------------------
gdb-peda$ checksec
CANARY    : disabled
FORTIFY   : disabled
NX        : ENABLED
PIE       : disabled
RELRO     : Partial
-----------------------

Full ASLR is enabled.

I do not have access to a syscall gadget at any known address, and after hearing that vsyscall might work, I learned that vsyscall is now emulated in newer kernel versions and can't be used for a syscall gadget.

I've done manual searches for int 0x80, syscall, and sysenter hoping to find them in known executable section to no avail.

Does anyone know of any way to exploit this binary without any known address except for 'read'?

I suppose the only way that I can imagine is by somehow reading a libc address from the GOT and ROPing into dereferencing that address into a new function at a new offset... but I can't even find a gadget that will let me dereference.

I can't use partial address overwrite bruteforcing because x86-64 has too much entropy.

Is there some technique that I don't know of?

I'm banging my head against a wall on this one!

It ends up that the solution was using a partial address overwrite after all.
Using the address to read in the GOT, a two-byte partial address overwrite can be done to jump within a guarantee of 8192 bytes from read into libc, with a 1/16th brute-force chance to jump within 65535 bytes (due to 4 byte entropy of ASLR).
Within the two-byte overwrite, there is a magic gadget located 0x6109 bytes away from the address of read, which is within the two-byte distance.
From one_gadget, the magic gadget near read can be found:
0xf0567 execve("/bin/sh", rsp+0x70, environ)
constraints:
  [rsp+0x70] == NULL

Where in this version of libc (2.6.32), the call to read is found at:
0xF6670 read            proc near

Using a call to read, we can overwrite the GOT entry of read and ROP back to main to call our magic gadget.
Also, another solution was found by doing that same partial address overwrite to ROP into setting up a syscall then overwriting the last byte of read to jump 0x0E bytes forward and hit the natural syscall gadget found within read. This way you don't have to bruteforce ASLR and have a stable exploit!


Is having a unique key to encrypt data not sufficient Is it necessary to use a unique Initialization Vector (IV) with various cipher modes of operation (e.g. CBC) even though I use a unique key to encrypt plain text every time?

It is said that an IV needs to be unique but not secret, every time a new piece of information is encrypted.

Can't we just make IV as unique and secret?  Will it
mean that the IV can now be treated as a secret Key?

Can't we just make IV as unique and secret? Will it mean that the IV
  can now be treated as a secret Key?

Key distribution is hard, and there's no reason to make the IV a secret.

If Alice wants to send a message to Bob using symmetric encryption both must somehow already know the encryption key and the IV. To share the encryption key Alice and Bob could meet up in person once, then use the shared key to send many messages without having to physically meet again.

Now if the IV is a secret, and the IV must also change every message then Alice and Bob now have to meet to share the secret IV for every message. If they have to meet every time, then why not just share the message when they meet?

Seeing as the encryption is not weakened by an attacker knowing the IV, they might as well save themselves the hassle of meeting in person by just sending the IV with the encrypted message, rather than trying to securely exchange it. It saves them both a lot of hassle, and they're no less secure.


Combining User Context in Machine-2-Machine OAuth2 Client Credential Flow I have a REST API that is used by 2 separate application and is authenticating them by M2M OAuth2 Client Credential Flow.

One of the two application is an automation service without user context. The second one is a REST API where users authenticate with OAuth2 Implicit Flow.

Now I need to include the user context in my common REST API too, since some information should only be shared to certain users.

What is a secure strategy to implement that scenario with OAuth2? I thought I could just include the user (or a fixed string in case of the automation service) into the Access Token of the Client Credential Flow but that doesn't seem possible.

If you are able to authenticate the end-users as well as the client server, it's better to use the Authorization Code flow.

Alternatively, check the user context during the API call instead of in the authentication step.


Are "Security SMS codes" any safer than plain SMS? Many banking webapps send this kind of SMS, for their 2FA systems. These messages do not appear in the main "Messages" app, seemingly being destroyed after viewing - at least on iOS.

Firstly, are these messages any safer than classical SMS (like the one you send to your girlfriend, asking whether to buy bread) ?

Secondly, how are these messages sent ? Could it be done without using Twilio, or other SMS gateways ?

It's not actual 2FA but 2-step authentication. As linked by StackzOfZtuff they are in fact plain SMS messages.

The reason it's two step and not two factor is because two-factor would imply you need true possession of the object where the token is generated and there has to be assurance the token cannot be intercepted by another party. This is where SMS is flawed. SMS has a few known vulnerabilities which would technically make it possible for interception:

Compromised GSM network
False base station attacks on older versions of the GSM protocol
SIM card cloning

That said the messages are indeed sent using SMS gateways. So is it in fact fool proof way of authentication? Probably not. However each bank does a risk assessment on what the likelihood is of attackers performing the above attacks with the purpose of stealing bank accounts and probably realized the likelihood would be low. It would require a lot of effort in case of SIM cloning or false base station attacks versus the return on investment. If someone manages to compromise a complete cellphone carrier, then there would probably be more valuable attacks to perform than targetting individual bank customers.

Furthermore most banks often require additional confirmation mechanisms to perform actual transactions (but this depends on the bank of  course). For instance my bank will not allow me to send transactions to accounts to which I have not sent money before. Otherwise I need to sign the transaction with a card reader.


Secure symmetric encryption of passwords to allow admins to log in as users I am trying to work out the best way to secure user credentials for the login of the service i am working on.
The safest way would be to use hashing and salting however this project requires the password to be decrypted so that the support staff can login to the users account (if the user opens a support ticket if something is not working right).

I have done some research and it seems like AES is the best option.
My concern is how I should be storing the encryption keys on the server, because if the server is compromised then the hacker has access to the keys to decrypt all the credentials.

I would be grateful if someone could inform me on the best practices i should use for this project.

Side note: I am programming the backend in NodeJS and it will be running on a private linux VPS.

Generally, it's good practice to allow some other method of impersonation. By allowing administration staff access to passwords, you lose the ability to verify that a given user has performed a given action - it might be the original user, it might be your admin staff. It also compromises any users who happen to re-use passwords on your site and elsewhere - it would only take one admin user writing down the username and password, and finding the same details in use on another site.

If you must allow logging in as another user, the admin staff should reset the password to one they know, make whatever changes they need to do, then enforce a password reset when the user next logs in.

In short, don't do this - either implement another method of impersonation, which logs the admin user who accessed the account, the time, and what they did, or make use of your existing password reset processes, and ensure that the password known to the admin staff is changed the next time the user logs in.


Using RSA for Web Application Authentication I have a network that is only accessible through bastion servers over ssh. In it, I am developing web applications that are exposed to end users via GUI's served up over websockets. All traffic into and out of the network travels through an SSH tunnel from whatever service host, through the bastion, to the end user's machine.

So, traffic from the network's boundary to the end client is already encrypted. And, I already have a system setup for associating end users with SSH-RSA key pairs that are rotated periodically.

So, here's my crazy idea:

On any web app in the network that requires user authentication, have the end user provide their username on the network, as well as their public key. Have the web app's authentication service verify that this public key is indeed already associated with the end user in the network's keystore, then generate a challenge using the key. Client side, the end user decrypts the challenge using their private key and presents the end result to the server for authentication.

Here's the kicker: a load of services in this network are implemented in JavaScript on NodeJS. Because NodeJS can call system services, I can offload the work of generating challenges using public keys to a more traditional process. But, client side, I might need to decrypt the challenge in-browser using the private key.

Getting the private key into the browser is easy enough. Whether or not loading the private key into a browser is even a good idea is another question, entirely.

But, more specifically:

Are there any blazing "Oh, don't do that!" security flaws in the setup summarized above?
What are a few standard packages used in Linux for generating challenges using an RSA public key?
Does anyone know of a good package for decrypting messages using a public RSA key implemented in JavaScript?

Are there any blazing "Oh, don't do that!" security flaws in the setup summarized above?

The basic protocol you are proposing is: client identifies themselves, server checks the public key and verifies that they are a valid user, server encrypts a challenge using the public key, client decrypts to get the challenge back and returns it as proof of knowing the private key.

I don't see any blazing issues. It may be better to follow what SSH itself does for public key authentication as shown in RFC 4252, section 7. There, they have the client sign something with the private key that the server can then verify. Always good to stick with standards.

  What are a few standard packages used in Linux for generating challenges using an RSA public key?

If you have openssl, I'd do this openssl rand -base64 16 (or possibly 32 bytes).

Otherwise, you can use /dev/random (or possibly /dev/urandom if absolutely necessary). Something like this head -c 16 /dev/random | base64. See this for more details

  Does anyone know of a good package for decrypting messages using a public private RSA key implemented in JavaScript?

A number of these libraries support RSA. Not sure how good they are and how easy it will be to transform the SSH private key into something they can use.

This answer is a summary of our chat


Why does OWASP recommend security questions? I was reading the OWASP Forgot Password Cheat Sheet when I stumbled upon the recommendation to use security questions.

There is even a dedicated page about what information to gather.

Whenever I see such a "feature" on a web site it strikes me as unbelievably insecure for most users, because most users will choose a question an answer that is probably very easy to find out by anyone who knows the person a bit. So I was very surprised to see such a recommendation on OWASP.

Is it really a good idea to implement security questions?

Specifically with regards to security questions you are asking users to share potentially sensitive facts about themselves that are likely obtainable by social engineering of people near them.  That makes security questions at best inadvisable and at worst downright dangerous from a holisic security perspective.

OWASP, while good overall for explaining the problems, makes many inadvisable recommendations including security questions, input whitelisting, and XXE mitigations that prevent loading most XML documents.  Take the problem descriptions as they are, but really scrutinize their solutions before considering implementing them.


Best Practices for exporting sensitive database data and transforming it whilist keeping it encrypted? Say one has to export database data to use in another tool, such as an Excel Spreadsheet or for use as a csv file with some other program.

There's a problem with this, and that's that it ends up in an unencrypted form on the disk, where it can be forgotten about, snatched up by phishing, etc.

So when storing / using / getting rid of this sort of data, is there some sort of OS level solution (LUKS) for working with then getting rid of these sorts of temporary files?

I know when one uses SSH-agent the keys can be given a time limit to be stored in memory, before it's deleted, is it possible to do the same thing with an encrypted directory or encrypted volume, that can be "scrubbed" automatically, after a certain period of time?

Also, it would be crazy to do something like upload such a csv file stored encrypted on the disk to a web application that runs over HTTP and thus leave such things moving over a network encrypted when it moves across the network, that would destroy the point of encrypting it, even if it was encrypted on disk.

And also what about allowing an approved running process to access the information but not allowing it to be copied elsewhere?  That sort of thing sounds in the domain of permissions / ACLs to me.

Is it best to have a designated machine for doing this and limiting it's access to the outside world using a separate subnet / firewall / proxy or some combination there of?

I would generally be more concerned about the unencrypted Excel Spreadsheet with sensitive data 'getting lost' than the temporary file, which can be dealt with a good procedure stating that it must be deleted immediately after generation.

Anyway, here are a few ideas:

shred the temporary file after processed.
Store the file on a ramdisk. For instance /dev/shm is always in-memory only¹ (sometimes /tmp too). If the temporary file is written there and forgotten, it will be gone on reboot.
Boot a live disk to perform the transformation.
Do not use temporary files but pipes. Using some command-line trickery (and depending on your data!), you may be able to get the desired outcome avoiding temporary file, eg. mysqldump --skip-extended-insert sensitivedatabase | sed -n -e '/^INSERT.*VALUES(//p' -e 's/);//' }' | gpg -e -r recipient > encrypted-spreadsheet.csv.gpg
Use a service that already exports the data. If this data is already handled in a web application, add the ability to the application to export it in the final format (we are actually moving the problem, but if that application is already handling this sensitive data in a way deemed secure, it makes sense to incorporate this feature there, too).

¹ Nitpicker's note: it may get swapped.

As for the other questions:

  is it possible to do the same thing with an encrypted directory or encrypted volume, that can be "scrubbed" automatically, after a certain period of time

You could have a solution that automatically unmounts or deletes temporary files somewhere. But it seems overcomplicated. I would make a script that performs the full exporting process, and that scrubs the unencrypted temporary file as soon as it is no longer needed.

I have no idea what you mean with the encrypted csv moving over the network using HTTP part.

  And also what about allowing an approved running process to access the information but not allowing it to be copied elsewhere? That sort of thing sounds in the domain of permissions / ACLs to me.

You certainly can do that. Be that a script, embedded on the web application, etc. If you get fancy, all of this could use selinux labels to enforce the process.

  Is it best to have a designated machine for doing this and limiting it's access to the outside world using a separate subnet / firewall / proxy or some combination there of?

This probably will depend too much on your specific case and the way of transformation for giving a generic answer. Obviously, the most isolated, the better. But as mentioned, I would start by questioning why this highly sensitive data needs to be given to someone on an Excel Spreadsheet, to begin with.


Is there a way to sniff a "mousejack" attack? I'm sure a lot of you are aware about the vulnerability in unencrypted wireless mouses. If not, here's a website reporting on the vulnerability: https://www.mousejack.com/

It sounds like the best way to prevent an attack as a consumer is to either not use a wireless mouse or make sure that the mouse you're using/buying has had this vulnerability corrected. However, wireless mouses are so cheap that I'm sure many (including myself) have one from manufacturers that haven't really responded (yet?) to this vulnerability.

Since the communication is unencrypted, the attack I assume to be unencrypted as well, so I think that it would be pretty easy to "sniff" out this attack at a public space. As a consumer however, I don't have the skillz to write out my code and use my existing wireless mouse dongle to sniff out attacks.

So first question is, is there already a way to do this? (use your wireless dongle communication to alert of an attack)? I saw that there's a program to find vulnerable dongles with an antenna, but not to find an attacker actively trying to exploit it: https://github.com/rfstorm/mousejack

Second question, do you know other ways to prevent/discover a mousejack attack on your computer/somebody else's in a public space?

If I understand this issue correctly, mouse movements are sent unencrypted by design, and keystrokes are sent encrypted by design, however the receiver also accepts unencrypted (and unauthenticated) keystrokes as well, allowing an attacker to inject them?

Regarding detecting unencrypted mouse movements, I would say there is no easy way to detect such an attack. It simply appears to your computer as regular instructions for your mouse. Imagine you were blind, you could not move or speak, and the only sense you had was hearing. Now it gets worse. Not only are you only able to hear, but you aren't able to differentiate between differentiate between voices. That's what a wireless mouse receiver is like. It has no way of detecting such an attack because the fake signal is identical, for all intents and purposes, to the legitimate signal.

As for unencrypted keystrokes, that's a different story. Because keystrokes are not meant to be unencrypted, the receipt of an unencrypted keystroke is an anomaly. If you connect a Bluetooth device to your computer which can capture arbitrary Bluetooth packets, you should be able to detect when unencrypted keystrokes are transmitted. You would have to find out what the format is, how to detect when the packet is unencrypted, and then how to detect if the payload message contains a keystroke. If it is, then you should have your computer immediately disconnect the keyboard and mouse device and alert you so you can take action. I know of no way to already do this, but there are ample resources to learn how to.

Note that Bluetooth is historically not the most secure protocol, so even if this "vulnerability" is fixed, using a Bluetooth connected mouse and keyboard is rather foolish. PS/2 is low power, entirely interrupt-based so the CPU isn't being woken up thousands of times a second. USB is cheap, the cords reach as long as you want, and if you know how to manage your wires, they will never get in the way. Using wireless peripherals is asking for trouble.


How to keep website passwords safe among employees? We're a website developer agency, and most of our websites are made using in Wordpress, Joomla or Drupal as a CMS.

I'm having trouble to decide how we should manage our passwords, and I'm open for suggestions.

We could:

Use a default password for every website (big no)
Generate an unique password for every website and keep it in a database (make the database accessible from only our office's IP would be a big plus, but then in case of an emergency, it could make things worse if nobody is in the office.)

My main concern is not only keeping de passwords safe among the employees, but most importantly is keeping the passwords safe against the ex-employees.

I've had a case where an employee quit, and the next day he entered one of our websites and defaced the whole website.

Unlike servers, I can't use private-keys in this case, as we do to access our servers.

Any suggestions?

You cannot enforce forgetfulness; if an employee could, at some point, learn a password, and indeed typed it in the course of his job, then he may still remember it even when no longer an employee, and you cannot force him to "forget".

Therefore, the only way to "destroy" a password access is to invalidate that specific password. The "good" way is to make it so that each employee has his own administrative password. On the day the employee leaves, invalidate this password on all sites where it is active. If the employee leaves under some distrustful conditions, invalidate his password five minutes before you tell him that he is fired.

If you must share passwords because of some technical limitation (e.g. some site software which recognizes only a single "admin password"), then the only way to invalidate the password is to change it, and give the new password to all remaining employees.

Since each employee may end up with a lot of passwords to remember, they will need some kind of storage system. I have seen people use this product and be happy with it, but I don't believe this is the only one of its kind.


What are the most relevant security events/incidents any company should monitor? What are the most relevant security events/incidents any company should monitor using event management tools? Related to servers, antivirus, networks, etc.

You should log everything and analyze anything. People trying to brute your sites and servers. All of these are relevant incidents as they pose a threat.

It's not relevant to say to what is an incident, as every attempt might be a relevant event some time. What is important is to catalog them with different levels. Some threats pose greater danger than others. Some of them might contain a lot of "normal behavior".
The thing is that some things only become relevant after analyzing over a longer period of time. You might notice one attacker is doing his best extra hard to get in.
You might get a hacked server and then you'd have to be able to go back and look for related security events to learn from the incident.


What is the difference between ordinary XSS and Dom XSS vulnerabilities? I'm not a computer guy, but just out of curiosity I've recently started to learn PHP for programming my own website and I've read about some most common vulnerabilities that exist in web applications. I know the basics about Cross Site Scripting vulnerabilities. So, an XSS attack happens when the input from the user is not filtered properly and it is used in the body of the page, so attacker can inject things like javascript/vbscript codes into the browser and do stuff on the client side like hijacking their cookies and save them in a database.

But what is the difference between XSS and Dom XSS? Does there exist a fundamental difference between them? Do DOM XSS vulnerabilities pose a higher security risk? If yes, how? And how should I look for Dom XSS vulnerabilities in web applications? I find some stuff on the internet but some of them were advanced for my current level of understanding.

Thanks guys.

Ok so at a basic level there are three types of Cross-Site Scripting.

Reflected - You enter data to the application, which is then echoed back without escaping, sanitization or encoding and it's possible to include JavaScript code which is then executed in the context of the application

Stored - You enter data which is stored within the application and then returned later on in response to another request.  This data contains JavaScript code which is executed in the context of the application

DOM based - You enter data which modifies the DOM of the web page, this data contains JavaScript which is executed in the context of the application.  It's relatively similar to reflected XSS but the difference is that in modifying the DOM the data might not ever got to the server (which changes how it can/should be mitigated as server-side filters might not be effective).

An example would be web applications which make heavy use of client-side JavaScript and which take user data and update the DOM without ever actually sending the data to the server. If you see applications which make instant changes to the page you're viewing when you enter data, it's likely that they're using client-side JavaScript to update the DOM.

In terms of finding them, that's kind of tricky, as the tools to find DOM based XSS aren't as developed as the ones for Reflective or Stored XSS.  The only tool which specifically addresses it, which I'm aware of is Dominator.  There's also information on the DOM XSS Wiki which could be useful.


Is it possible to Exploit this Line Feed? During a security assessment on a website I found a a redirection link that was reflecting the values in the Location header. The first thing that came to my mind was CRLF injection so I tried a few variation of "%0a" and managed to include my payload in the response:

Request:

"https://ads.example.com/promoredir?redirect=http%3A%2F%2Fmain.example.com%2F%E5%98%8A%E5%98%8DSet-Cookie:%20test"

Response:

HTTP/1.1 302 Moved Temporarily
Date: Sun, 12 Jul 2015 14:18:41 GMT
Server: Apache
Set-Cookie: JSESSIONID=5C24F2C96CE37DAA026591F5CAD91900; Path=/; Secure; HttpOnly
Location: mail.example.com

Set-Cookie: test

Content-Length: 0
X-Content-Type-Options: nosniff
Strict-Transport-Security: max-age=31536000
Vary: User-Agent
Connection: close
Content-Type: text/plain; charset=UTF-8

However when I follow the redirection, my request changes to the following:

GET /Set-Cookie:%20test" HTTP/1.1
Host: main.example.com
Accept: */*
Accept-Language: en
User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Win64; x64; Trident/5.0)
Connection: close

Where am I going wrong here?

I would try to just do something like this:

http://www.example.com/somepage.php?page=%0d%0aContent-Type: text/html%0d%0aHTTP/1.1 200 OK%0d%0aContent-Type: text/html%0d%0a%0d%0a%3Chtml%3E  TEST TEXT %3C/html%3E

to start with. Then, if you see 2 different responses, and one of them is a 200, and if you see the text, you can then try whatever stuff with cookies.


Should smartphones with AI assistants be restricted for use in office? New Galaxy S8 & S8+ provide Samsung's new AI assistant called Bixby. This thing was claimed to upload photos and voice records to somewhere. Is it a risk for company confidential information leakage? What about others like Siri, Cortana, Google Assistant? Does it make sense to restrict all of them in offices? They can't be controlled in regard to what they send and how companies owning them use gathered information. I'm not law expert, but some user privacy agreements look very fuzzy and therefore suspicious (i.e. Samsung Bixby's)

Short answer is yes. Any recording device that you do not control should not be used where the information it records is sensitive.


SSD Encryption - Difference between models I understand that most SSDs which support encryption usually encrypt all data, but that only some actually encrypt the encryption key with a user BIOS password. This means that the models which don't do that can be circumvented by moving to a new computer.. is this correct??

To me this seems like little more than a deterrent - no different to setting a BIOS password on a non-encrypted drive. Is there something I am missing (apart from when disposing of old drives i.e. secure wipe)?

If my previous thoughts are correct, then my big question is -
Is there a list of which models/brands are actually secure (i.e. use the user password to encrypt the encryption key)??

I have heard that Intel is the only brand, but there seems to be very little information about this.

Thanks for any input!

The ones which are tied to a BIOS identifier are harder to move to another computer - the idea is that they will only work in the computer they are configured for, but this really only works if you have a Trusted Hardware Platform.

You can't circumvent the ones which don't have an encryption key tied to the BIOS password by moving them to another machine - you still need to use the encryption key to get anything off them.

SSD's which encrypt all the data are all secure (secure here can mean many things, but I mean - if you lose the key all the data is inaccessible to anyone. There are some variations, but for consumer grade SSD's this is broadly correct) - it doesn't matter which brand.

You should be able to find vast amounts of information online - both here (we have questions and blog posts on this subject) and all over the Internet, as security of data on SSD's is a hot topic.


How secure is commercial cloud storage compared to my private cloud storage? I've been wanting to determine the most secure way to store data via Cloud storage solutions. And although at first it may seem like a duplicate question, I have read all the similar ones and nothing seems to directly address this.

Here are my assumptions (correct anything if I've got it wrong):

Commercial solutions (ie. iCloud, Google Drive, Dropbox, etc)

Data is encrypted on a disk level by the cloud provider but not on a per-account level (meaning they can access your data unencrypted if they wish..)
Data is protected by your account password and two-factor
authentication (if enabled)
The only foreseeable way to truely protect your data is to have some kind of encrypted volume that is in the cloud drive (maybe like a TrueCrypt volume or encrypted sparsebundle)
If government agencies or the provider itself want to access your data they can
Commercial solutions would be more secure than privately hosted cloud storage solutions. Simply, because that have significantly more money to spend on InfoSec and Cyber Security for their systems.

Private cloud storage (ie. OwnCloud, Nextcloud, etc)

Probably aren't as secure as big applications like Google Drive or iCloud Drive
You control the encryption so it can't be read by the provider (ie. the files stored on your own data store)
All code is openly reviewed by experts & online communities rather than being closed source.
It is more likely that government agencies can find security vulnerabilities in a private system???
Data is presumably encrypted on two levels: your application encrypts the files on upload and all the storage is encrypted on disk level by your provider.

So here's the question...

What are the security benefits/concerns with using your own private cloud storage (where you control the encryption and security layers) vs entrusting companies (with millions of dollars expenditure on InfoSec) but are closed source and can access the data?

Ultimately - which is the better option and why.

Thanks in advance for your answers! :)

The question initially asks about security benefits, but then opens out to ask for "the better option" generally, and why.

In this answer I'm looking a the broader question, "which is best and why", for 2 reasons: first, being realistic, you won't decide on security alone (cost? knowhow? time? strength of personal feelings for/against privacy? enjoyment of challenge? other non-security pros/cons/motivations), and second, because security can be considered within the broader picture anyway.

Framing the question

Giving a definitive answer to the question as posed isn't always easy, because the decision almost always involves personal preferences (you might not be at much risk of intrusive data mining but still have very strong feelings about avoiding it if practical), and incomplete information (the actual risks which an infosec specialist could assess better than you, including your own ability to manage data if you keep it locally and the data safety of the proposed commercial host).

Fortunately I suspect that's largely a non-issue for your needs and this question, because the question suggests that you aren't already a data security specialist. So any answer has to fit an "informed enthusiast looking for their best options" or similar.  That means any answers where you magically make it all secure from scratch probably aren't suitable, which means we can look at the question from another angle that makes it much simpler.

Answer

From your question, the first definite point to discuss is the possible use of prebuilt systems (such as open source distributions that include cloud storage as part of an "all-in-one"/"out-of-the-box" general storage server). In these systems you don't just install a cloud storage package onto your OS. You install a complete customised and dedicated file server, which also provides cloud storage as a  feature.  You install from CD/USB, use the GUI/web to configure, and it's done.

This can be a very helpful source of assurance, because if its a well known project and has a reasonable security focus, then you can count on their own work (and their community of contributors) to cover a huge amount of the security unknown.

What I mean is, I wouldn't trust myself to set up FreeBSD, install NextCloud/OwnCloud/whatever, and trust it for security, but I probably would be comfortable downloading the FreeBSD-based installer for FreeNAS,  configuring it in the GUI for those tasks, enabling the NextCloud/OwnCloud plugin, and doing it that way, because the scope for my own ignorance and knowhow to wreak havoc is quite constrained, purely because the options I configure are preset in the GUI, they are functional, the important settings are already done, and it's mostly very clear to understand (or ask about if unsure). I'd still need to know some things, but much less and much more obvious. In fact that's exactly what I have done, minus cloud storage which I don't use.

So I'm going to assume that this is how you'd approach it if you did it yourself.

That means the real comparison isn't Google Cloud/Dropbox vs. do-it-yourself NextCloud/OwnCloud. It's Google Cloud/Dropbox vs. FreeNAS/NAS4free/Openfiler/Nextenta/OpenMediaVault etc that also supports NextCloud/OwnCloud - which has been properly configured and secured within a jail - and that's a very different proposition. It's much easier to be sure of the situation and security you'll end up with (the 'downside risk').

If the underlying system is good, the team building it is security aware, and you are comfortable with the basic config likely to be needed, then you will probably get a good level of security by any usual standard and can choose the final decision based on personal antipathy to commercial cloud storage and cost, which sounds like what you're after.

As an example, if you look at a similar decision for routers and firewalls, I switched many years ago from a commercial router to pfSense. Factors relevant to the decision included:

Cost (couldn't afford a router with the connections handling capacity I would need),
Physical access security (short of Secret Service/NSA breaking into my home, nobody is in a position to access or interfere with my computers through physical access except me. If you have hacky kids, housemates or visitors, consider the security of your data disks, backup disks (including where you keep them), physical keyloggers, FireWire/other accessible DMA devices and ports, or whatever)
Non-physical security (based on FreeBSD which is probably a lot more secure than most Linux or Windows setups, compared to most domestic routers which stop getting updates after a while or are just insecure by design or (very often) have huge security holes due to bad OS config, very popular/large community/serious use, speedy responses to security issues, team  appears to have a firm eye on security, has published security audits of the software by third party assessors which showed few issues, suggesting the team knows its job, enterprise security features if I ever want to get heavier on security),
Future-proofing (likely to have ongoing updates and active development for many years, or be forked if not, large user base, extensible and adaptable for almost anything I might need/want from it in future, even on my old Pentium 4 boards from 2003 which I have no other use for, I can get performance of hundreds of thousands of connections, caching, filtering, etc.),
Certainty of code/privacy (well-known open source, backdoors unlikely and guaranteed killed if ever found (including backdoors claimed to be "for support, honestly, bad guys won't find them!" as some products secretly have) runs on own hardware),
Easy to manage (very straightforward GUI after a bit of learning curve, any issues likely to be able to be addressed without buying another router),
and for the file server I later added, guaranteed data storing (some cloud providers suddenly failed or lost data when hacked, while not common you can't know what is happening "behind the scenes" and how dependable their setup and disaster recovery will be in reality, as a security specialist would assess it, because you just don't get access to that sort of knowledge).

I've used this platform for my router for many years now, and rarely need to do anything to maintain it apart from click when there's an update, and runs for months without issues. I've been using FreeNAS for data storage for a few months so far and it's been rock solid, any issues have been down to properly configuring the shares, so I expect much the same benefits from it.  It just keeps going, and once I get to a config I like, then I suspect I will barely need to know its there after that point.

The main downside in that equation is cost. Say you have 4TB of data (or whatever)... you might be committing yourself to a surprising hardware cost if you went that route:

Up to 4-6 x the amount of storage capacity compared to the actual data you expect to store, meaning for 4TB data you'd want up to 16-24TB of disks (file systems and especially the very reliable ones like ZFS used in many storage setups, might not want to be over about 60-80% full, disks fail so you need redundancy, and you do plan to keep a backup as well don't you?),
A good quality motherboard probably with ECC/RDIMM memory (platform stability, probably means using a server not consumer board for the ECC),
Lots of RAM (ZFS is hungry that way!),
At least one, maybe more decent SSDs (mainly for write cache but if affordable and needed then also for read cache),
Good quality power supply and probably a good brand secondhand UPS (so that power glitches don't trash the data),
Probably at minimum a second smaller, less well specified box for nightly backups/clones (replication, in case the entire primary box goes bang, so you don't lose it all).
Periodic replacement costs as equipment fails (each disk might need replacing every few years, so 4-6 disks might average 2+ a year if they're cheapest commodity type or about 1 a year for 5 year enterprise types)
Power (in some countries and locations, electricity is expensive and running a server with multiple drives will definitely add to the bill)

So cost is definitely an issue to consider when it comes to storage locally or in the cloud, because with a commercial provider you don't have to worry so much (usually!) whether their hardware and backup approaches are data-safe. Less so when it comes to say, a router where you can just restore config onto new hardware and carry on as normal if it all goes bang.

There may be workarounds if decent hardware cost is an issue. For example you might decide to run it on commodity or consumer hardware, with no backup or redundancy other than using two disks configured as a mirror. That would slash the cost a lot. You'd then set a task to copy your modified or important data into an encrypted disk system locally (truecrypt/veracrypt etc) and upload that to Google/Dropbox instead, or keep a copy on a third disk which you put safely on a shelf and update weekly. That might be a possible solution to the cost issue, but as with all compromises you'll have to consider if it works and the pros and cons for you.

As a guide and if it helps, I'm doing it all locally and its a lot of hardware - the primary server and UPS probably contains 8 x £250 ($300+?) of enterprise data HDDs, another £500 SSDs, and about 96GB of ECC ram for another £500 - and that's before the CPU, motherboard, PSU, UPS, and backup server. I've saved a bit with two 'tricks':  using a decent commodity brand for the backup (less worried about it once there's multiple copies of the data), and by having it do "double duty" (it acts as a router and service host when I'm awake during daytime, and when  asleep and the network is quiet at night it grabs a copy of the current file server contents acting as a backup server)  But that's my situation, yours may differ.


Feeding /dev/random entropy pool? Which way of additionally feeding /dev/random entropy pool would you suggest for producing random passwords? Or, is there maybe a better way to locally create fully random passwords?

You should use /dev/urandom, not /dev/random. The two differences between /dev/random and /dev/urandom are (I am talking about Linux here):

/dev/random might be theoretically better in the context of an information-theoretically secure algorithm. This is the kind of algorithm which is secure against today's technology, and also tomorrow's technology, and technology used by aliens, and God's own iPad as well. Information-theoretically secure algorithm are secure against infinite computing power. Needless to say, such algorithms are pretty rare and if you were using one, you would know it. Also, this is a "might": internally, /dev/random uses conventional hash functions, so chances are that it would have weaknesses anyway if attacked with infinite power (nothing to worry about for Earth-based attackers, though).
/dev/urandom will not block, while /dev/random may do so. /dev/random maintains a counter of "how much entropy it still has" under the assumption that any bits it has produced is a lost entropy bit. Blocking induces very real issues, e.g. a server which fails to boot after an automated install because it is stalling on its SSH server key creation (yes, I have seen that). /dev/urandom uses a cryptographically strong pseudo-random number generator so it will not block, ever.

So you want to use /dev/urandom and stop to worry about this entropy business.

Now you may want to worry about entropy if you are writing the Linux installer. The trick is that /dev/urandom never blocks, ever, even when it should: /dev/urandom is secure as long as it has received enough bytes of "initial entropy" since the last boot (32 random bytes are enough). A normal Linux installation will create a random seed (from /dev/random) upon installation, and save it on the disk. Upon each reboot, the seed will be read, fed into /dev/urandom, and a new seed immediately generated (from /dev/urandom) to replace it. Thus, this guarantees that /dev/urandom will always have enough initial entropy to produce cryptographically strong alea, perfectly sufficient for any mundane cryptographic job, including password generation. The only critical point is during installation: the installer must get some entropy from /dev/random, which may block. This issue also occurs with live CD and other variants with no read-write permanent storage area. In these situations, you may want to find some source of entropy to ensure that /dev/random will be well-fed, and will not block.

The operating system itself, and more precisely the kernel, is at the right place to gather entropy from hardware event, since it handles the hardware. So there is relatively little that you can use for entropy that the kernel does not already use. One of those remaining sources is Webcam data: a webcam, even facing a blank wall, will output data with thermal noise, and since it outputs lots of data, it is a good entropy gatherer. So just grab a few frames from the webcam, hash them with a secure hash function (SHA-256), and write that into /dev/urandom. This is still big overkill.


Is it possible to keep several files on windows partition after re-installing windows? Is it possible to keep several files and programs on windows OS after re-installing it? because of someone can steal the device that uses windows OS, then just removes the windows and install another one (cracked or even bought). So you just lose your device and all of the security applications! I think it is really secure-less and there is several methods that may security applications uses it for the times like this. So, Is it really possible to have and keep several applications on the windows even after re-installation?

Although windows can be installed on top of another copy, this is not required to break into it. A simple password removal tool, like "Offline NT Password and Registry editor" will do the job.

I'm not sure I understand what you mean by "security applications". Lojack for laptops? That can reinstall itself even if you format, on supported laptop models.


Is this generation of project URL safe? In proceedings of my question's answer I found a way of generating a project URL regardless of project's path in Document root.

If in a file in the root of the project I define the following:

define(PROJECT_URL,
       '//'.$_SERVER['HTTP_HOST'].
       substr(__DIR__, strlen($_SERVER['DOCUMENT_ROOT']))
);

I get a static URL that is useful and will work if I move the project elsewhere.

However because I am fairly novice with web servers and applications I wonder if this approach has any exploitable security vulnerabilities.

Per the documentation, $_SERVER['HTTP_HOST'] is populated based on the Host header sent by the client.  You should probably start worrying at this point, as it's never a great idea to trust data that may be provided by a malicious client.

There is also SERVER_NAME, which provides a similar but slightly different result.  You can find some good discussion about these two and the appropriate ways to use them in this Stack Overflow question.

Now, what attacks could someone perform via manipulation of your hostname?  That depends on your app.  Perhaps you're caching a rendered page and serving it up to another user, and that page was generated with Host: mymaliciousserver.com, so now your second user is clicking links to the attacker's server, where they phish out account details.  There are any number of possible vulnerabilities, and rather than trying to patch them all, I'd rather just avoid the problem entirely.

It's also somewhat questionable code anyways - it adds a lot of complexity for something that doesn't change often.  If it's your own custom app, just set the hostname once and be done with it; if it's a distributable app, have the customer set the hostname once and be done with it.  You're adding moving parts and magic, and those tend to have security issues as well as performing poorly in environments you haven't explicitly designed for (e.g. sites behind a reverse proxy, sites hosted on a non-standard port, sites run in a multi-site environment, etc.).


How to implement "Enter Password to Continue" for a page in a stateless application which uses JWTs? In my application I want a page to be accessible to the user only after s/he has entered the password again to prevent someone else opening that page in case the user has left a logged-in device open. The application uses JWT tokens for authentication.

How should I implement a re-verification of the password and preserve that until the task on the next page is complete?

The easiest way is to perform both the task in question and the authentication in one go. Let the user fill out any forms and enter the password at the same page. When the user submit, you check that both the JWT and the password are correct server side, and if so execute the task.

This works well for tasks that don't require a lot of user input, like changing your email. For something bigger, like giving access to a full admin area, you need something else.

Lets look at the admin area as an example. One solution is to have a separate login endpoint for the admin area, and after authenticating the user at that endpoint you just add a flag to the JWT payload, e.g. isLoggedInAsAdmin: true. If you want to have the logg in for the admin area expire quicker than the regular logg in you can just use a separate JWT instead.


Creating secure PIN in Java I need to create cryptographically secure PIN number in Java using secret id for each user.

This simple approach comes to mind:

int getPin(String secretUserIdStr)
{
    String pepper = "randomPepper...";
    String seed = secretUserIdStr + pepper;

    SecureRandom sr = SecureRandom.getInstance("SHA1PRNG", "SUN");
    sr.setSeed(seed.getBytes());

    return sr.nextInt(10000);
}

Is this good idea? Is it secure? If not, what is the best practice?

Don't do it. Reason: general principles. ("Don't roll your own crypto.")

You're deriving a key (something secret) from something else secret. You don't need to invent a procedure for yourself. Go with an established key derivation function. Something like PBKDF2.


Would it be secure to procedurally generate passwords based on a master key and an account label? So the idea would be to have one master password that is combined with a label (key+hotmail) to create a password specific to that account.

Now I can have one complex password I can remember but different passwords for each account, meaning stolen data cannot compromise multiple accounts.

I suppose if someone knew what my accounts were, had multiple generated passwords and knew the labels for each and knew the method of generation they could reverse engineer my master key?

I am not an expert in this field and this is quite possibly a duplicate. But any other questions I could find that I suspected were asking similar things were using terminology I just wasn't sure about. Apologies in advance :)

I suppose if someone knew what my accounts were, had multiple generated passwords and knew the labels for each and knew the method of generation they could reverse engineer my master key?

If the password generation function is CRC32(master_passphrase||label), it could probably be reverse-engineered.

Now if you use a cryptographically secure hash function, like SHA256, it will be computationally unfeasible to determine the master password.

Security can be further improved by using PBKDF, which uses a cryptographically secure hash function in multiple rounds to slow brute-force attacks (and, possibly, also worsen the complexity of future attacks based on as-yet undiscovered cryptographic weaknesses).


What's the alternative of content security policy (CSP) header in Internet Explorer IE? As mentioned in the Content Security Policy documentation & from the "supported browsers" page on the CSP site, CSP is not supported in Internet Explorer.

So, if we want to support CSP in our application with all the supported browsers which includes IE, what is the approach that one should follow? Is there any alternative to CSP (X-Content-Security-Policy is already deprecated) for IE?

I observed that Facebook.com uses CSP headers with Chrome but doesn't use any alternative of CSP with IE. I came across the iframe header which can be used in place of CSP but couldn't work out how.

MDN - Content Security Policy (CSP) - Browser Compatibility shows what is supported in MSIE and what not. To summarize:

MSIE only supports the sandbox flag
MSIE only supports the old X-Content-Security-Policy header. It does not matter if it is deprecated -  MSIE by itself is deprecated too.

  Is there any alternative to CSP

There is no general alternative to CSP. Some parts of what is know in CSP can be achieved with X-Frame-Options but most of the functionality of CSP is simply not available in MSIE.


Is it possible to track someone by sharing a youtube link with timestamp? Is it possible to track someone by sharing a youtube link with them with a timestamp such as &t=1s if they click on it on their end?

I've recently have been told not to use timestamps because of this...

Yes, you give someone a unique timestamp, and then that URL can be tracked with the timestamp like an ID value.

But you would have to be able to track URLs, too.


Might Google log every website I visit using Chrome? I wanted to start using Chrome for quite some time now, but I really don't want to tell Google about whatever sites I visit.
So I was wondering if Chrome sends any data to Google telling them what I'm doing etc.
I'm specifically wondering if the latest versions of Chrome are using Google's DNS servers to resolve any domain I'm visiting because then they would know about every single site that I'm visiting on the Internet, even if I blocked Google Analytics etc.

I'm also thinking about the navigation bar which might send every character I type in real time to Google to give me search suggestions before even knowing that I was just going to enter a URL instead of a search.

In case you confirm my suspicion, could I do something about all this?

Read Google Chrome's privacy policy here

Navigational errors caused are resolved automatically through Google servers. You can disable them in the Settings > Advanced Settings > Privacy. In fact, you may disable all checkboxes in Privacy section in order to not allow contacting Google every time you search something.

You can also block cookies from Google in the Content Settings, disable auto-translation, Auto-fill and password manager. And of course, disable all extensions, themes and apps.


Iot devices and wan security How does my router know what requests to accept and which to ignore, when treating connection requests to the smart devices that originate from outside my network?

I have no port forwarding configured, yet when i use the Nest app, or the Wink app, for instance, I am able to send commands from outside my LAN.

I get that the app needs to authenticate when connecting to my device, but how can it even establish a connection from outside my network, knowing that all routers act as natural firewalls, and that no ports have been opened.

Generally, these smart devices have a third component - the service provider.

Using Nest as an example, when you connect it to your wifi network, it connects back to Nest servers so that the Nest iOS or Android app can make changes to your home device remotely. You don't need to connect directly to your home network.

The most direct security benefit being that you don't need to expose your home network to the internet, which is unsafe.


What can a hacker do with a card number without CVV? From my experience a card can not be used for online payment without a valid CVV, then why is it still always emphasized that card number must be protected?
What can a hacker do, assume that he has all the information on the front side of the card (card number, holder name, expiry date) but not the CVV?

The only fields required to charge a credit card are the number (also called a PAN or personal account number), the expiration date, and an amount. Without the CVV it is still very possible to charge the card.

Many merchants will require the CVV and/or postal code as basic anti-fraud mechanisms. There is also an incentive for many merchants as providing these can qualify them for lower interchange rates, the service fee paid by merchants to the card network.


How do I brute-force a password when I know the format? The format is 1 letter + 5 numbers (e.g. A12345).
Any method and software?

You can generate a list of passwords matching the format you want using hashcat:
hashcat --stdout -a 3 ?u?d?d?d?d?dhd

will generate Udddddhd where U is upper, d is digit, but the last 'hd' is just 'hd'.  More information about hashcat mask mode is here.
Once you've generated your wordlist, you can use a tool like Hydra or Hatch to feed those passwords into the web form.


JS frontend for RESTful backend vs authorization I'm building web-app. And i have concern about security issues.

In world a lot of similar questions, but many of them does not have really clear answer(advise,suggestion). This is the reason why i made this question.

I have login page with username and password. User will POST them by HTTPS.
Server will compare it with username and password from DB and if everything is ok, will generate access_token using some specific approach like: sha1(username + password + expiration_date), btw, expiration_date it's Unix time of expiration that token. In result expiration_date and access_token will be written into DB to related User record, access_token will be sent to client JavaScript app by HTTPS certainly.

Then all requests from client app will be made by HTTPS and access_token will be included into each header request. Server side should authorize this request, and detect User ID who had sent this request. Server will make search in DB by access_token and will check expiration_date). After access_token will be founded  need to check next condition: current_date <= expiration_date, if true then ok, if false then we need to regenerate access_token using new expiration_date and save them to DB and send new access_token to client?

if Yes then what do you think about that approach form security side ?

Supposing you at least hash+salt the password of your users in the database, your workflow seems correct.

But I'd like to add two things :

DON'T PUT THE USER'S PASSWORD IN THE TOKEN, it's not secure at all and totally useless. The expiration_date is also useless here. Just put a sha1(username + 'randomly generated string') and you will be good.
If the expiration_date has expired, you shouldn't auto reauth the user, but ask her to log in again. Because if you do as you say, why use a expiration_date then?

For my #1, since your token is sha1(username+password+expiration_date), as an attacker, I can know the username, the expiration_date and the access_token, results of the sha1.

I only have to build a local brute force system that will try all the combination of the password possible, and compare sha1(username + "current testing password" + expiration_date) to the given access_token, and once I find it, I'll know the password :/


Is it safe to download internet files over TOR This question is related to downloaded files from the internet  (i.e. chrome.exe from google site or an attachment in email), excluding torrents which TOR expressly recommends against.

I have started to increase my use of TOR (after reading Little Brother and Homeland) and noticed that in the recommended steps, they express that you should not open files downloaded over TOR while connected to TOR; but not about the download itself.

Wouldn't the download itself also identify you. I am very hesitant to download anything over TOR for this reason and would like to better understand how it works and if it is safe to do so without exposing myself or my details to the end servers.

Usually, the reason to be cautious about downloading executables over Tor is that the Tor network creates a man-in-the-middle scenario: the exit node can alter the download, or replace it entirely, so that you receive some sort of malware instead. The way to solve this is to use https, in which case your connection to the website has an additional layer of encryption that will produce a large, obvious warning if someone tries to tamper with the response.

Now, as far as anonymity goes, the warning you get from the tor browser bundle is essentially saying that by opening files outside of the browser, they can't protect you any more - they can't make any guarantees about what those files will do, and it's possible they will send your real IP back to some server somewhere.

Downloading chrome over Tor, for instance, does not mean that chrome will be automatically protected through tor; it will make requests to websites directly (unless you configure it differently), and the install process might even send a note to Google saying "hey, this person installed chrome".  Other times, opening a file can break anonymity in less obvious ways; for instance, opening a video file in your media player may trigger a fetch of a remote media file, giving that server your IP. I suspect this doesn't matter to you, since you are installing another browser; it's mostly a matter for people with high anonymity needs.  If it does, however, that's the purpose of systems like Whonix.


What is an actual hardware firewall? I have to do a piece of college homework about firewalls and I came across this question, What makes a hardware firewall different from a software one?

If I have a corporate network, and I configure some arbitrary software firewall on one computer, and make all the PCs from this network pass thru this one computer, is this computer considered a hardware firewall?

I understand that some routers have a builtin firewall, and this is considered a hardware firewall. But it's actually a device with a piece of firewall software installed where all computers have to pass thru to get to the internet, right?

Thanks in advance.

A hardware firewall is a network appliance dedicated strictly to performing the functions of a firewall.  They may have hardware built specifically to speed the routing of packets, such as large RAM buffers on the network adapters.  And they are running an operating system designed to run a network appliance.

A software firewall is an OS-level application performing firewall-like functions running on an ordinary computer that may also be running other applications.  Networking functions are accomplished through software running in the OS.

The idea is that a hardware firewall will be "more secure" than a software firewall because it has a reduced attack surface.  With no other applications running on the hardware firewall, there are no additional memory buffers to exploit, no extra protocols with vulnerabilities, no unknown programs, no stray ports left open.

If you take an ordinary PC running a desktop OS and run a software firewall on it, it's still a software firewall.  The desktop OS may be running dozens of services that don't perform firewall functions, and they increase your attack surface.  You may not even be able to disable some ports on a consumer version of Windows 10, for example.  However, if you install an OS intended to be a dedicated networking appliance, such as OpenWRT, and if you run only a firewall on it, you'll have security much closer to a hardware firewall; although you probably won't get the same performance as you might with a commercial firewall appliance.

Keep in mind that having a dedicated firewall is one thing; but having a secure configuration on it is another.  The best hardware firewall on the market is useless if you don't change the default password, if you don't set deny all by default, or if you fail to disable UPnP support.  This is true regardless of if it's a hardware or software firewall.

Also note that if an attacker passes their attack through a firewall to a web application and manages to violate the web server's integrity, the end result is the same.  Firewalls stop traffic on unknown ports; they do nothing for malicious traffic on known ports.


Is it safe to deploy software without memory protections such as DEP/ASLR? Several software packages on our SOE Windows machines come with DEP and ASLR memory protections off by default. More troubling is that these applications run as SYSTEM and load their own drivers.

Given that these are expected memory protections and these are modern applications, is there any reason why they would/should be disabled for applications?

Am I, as an administrator, able to forcibly enable them without?
What is the risk of having such applications in the environment?

DEP/ASLR is just layer of security. It doesn't secure you against everything, and against any sophisticated attack in particular. Unlike @ConsideredHarmful thinks, DEP/ASLR being disabled alone isn't a vulnerability, as DEP/ASLR just adds a layer of complexity to an attack by randomising memory locations.

The security paradigm in general should be not to try to come up with a single "indestructible" layer of security, but instead apply as many of layers of security as possible, so if one layer is compromised, the other layers would protect. Just like democracy! With ASLR/DEP disabled, you make it easier for a potential attacker to attack, but this isn't the end of the world


What is the best practice to save encrypted data to a database? If I am to store different types of sensitive data, let's say: field1 in Table A and field2 in Table B and the data is unrelated, should I use a different key for each field?

Add information 1:
The encryption/decryption is going to take place in the application code and the database will only store the encrypted data.

Add information 2:
Right now, I'm planning to use AES-256-CBC and the database is MongoDB.

Based on the information provided in the update, I would suggest to use the same key for each field. Different keys would be an overkill as you might want to encrypt other fields as well and also it would be cumbersome to map the fields with different keys.

Following points can be considered with this approach :

The encryption key is not stored at the same db where encrypted data is being stored. (consider to store in configuration file)
The encryption key should be sufficiently long to prevent any brute-force attack on the key itself.
Based on the frequency of usage and sensitivity, decide whether to use Symmetric or Asymmetric. (refs : comments)

Edit 1 :

If using the config file to store encryption key, make sure to not commit the key accidentally. Instead, commit a dummy key and replace it during deployment.


Does ISO 27000 certification increase the risk of being attacked? I asked my client (a bank) why they don't certify themselves against ISO 27000 standards. The answer was that if they certified, it would increase the risk of being attacked.

Does that make any sense? Can hackers be aware that some particular company is certified? To my mind, if the company does not state it publicly, no one would ever know... Also, since this is a bank, it already inherently possesses some level of risk.

I do not see an increased risk of attack due to being an ISO certified organization.

ANSI (The governing standards body) does not release ISO certified organizations business names.  There are organizations that offer you the ability to look up an ISO 27001 certified organization but those organizations have elected to register voluntarily.

I would suspect that another driver (high cost, large time footprint, operational workflow implementation) is behind a lack of desire to become certified.


How can I protect me from... myself? So, here is the deal:
I have a file I cannot change. I cannot, but I know in the future I will want to change it. It's of my understanding that it's not possible to make this file unchangeable while being root. But I want to make it so that the only way to change it is by building my whole OS from ground up.

So, here is my ideas so far:
Use entr to make a script that verifies if
the file has changed and does something very harmful if so: like rm -rf /bin /sbin /usr /boot /etc... you get the idea (only mantaining /home). But in order for this to
work, this script must be running on a infinte loop since boot and I must not be able to stop it or to prevent it from starting in the first place.
So, to achieve this, I can only think of: obuscation. Maybe create the script, naming it and placing it in such a way that I would not be able to easily guess which one of the running processes is it without risking kernel panic or something similar, nor could I tweak my HD with a live boot USB searching for which file to delete.

Is this possible to implement? Would this work? If not, how could I achieve my goal?

If you set up a system to block you, you would ultimately also be able to bypass it.
The best way would be for you to place it outside of your reach. An option would be to burn a CD/DVD-R, so that you cannot physically change it. I don't know if whatever will make you want to change it could also make you want to destroy it. You could provide the disk with the never-to-be-changed file to a third party, such as a lawyer. You could also publish the file on the internet, in such a way that it would be very hard for it to disappear. You could share it on keybase, IPFS, a Dropbox account you purposefully lose the password to, get it uploaded to archive.org...
You can also use a similar approach to restrict you from modifying it, such as tweeting today the full hash of the file, so you couldn't change it into the future without revealing that it is not the original file.
Note that you could publish an encrypted version encrypted, so it doesn't show its contents to the world.
Finally, if you really know so well that in the future you will want to change it, I wonder if it's really a good idea that you make your future self unable to do that.


Webgoat missing function level access control lesson I've recently installed WebGoat 8.00M12 on my computer and I tried to solve the "Access Control" section for a demonstration in my class.

Everything was simple and smooth till I got stuck at the last point in "Missing function level access control lesson" subsection, and I got stuck there for 2 full days now. Can anyone help me, please?

The question is this:

Note: the previous page contained 2 CSS hidden hyperlinks (/users and /config) that doesn't work when i click on them or try them (These are supposedly only available for the admin)

What I have done:

Using burp suite to spider the web application the intended directories where not found.
Tried many combinations like (show-users users list-users display-users) on many directories but just couldn't find the "users page" they talked about in the hints, but I did found some useless page at /WebGoat/users which contained only "su 1" just FYI.

Helpful notes:

The WebApplication runs a RESTful structure.
It runs on Java (I tried reading the source code on github but didn't understand anything).
Download WebGoat v8.0 (if you care) Here

Well its kind of a hard challenge.

First when you click on the human symbol to logout you should notice there is a role: User.

Also when you visit /users you should notice in the source code a hidden button like "admin" or something like that. So you need to try somehow change the role.

Logout and try to register again. Notice the post parameters.

So do the same on /users but with:

parameters in a list --

  {"parameter1":"value", etc }

add a new parameter role

  ( value: Admin wont work so try WebGoat_Admin)

and also change

  content-type:application/json


How does the Emotet malware work? I was searching this site for questions on Emotet and I was kind of baffled when nothing was on here. So the title is kind of self explanatory. See this question as an attempt to "complete" the content of this page.

How does the current (and/or former) iteration(s) of Emotet work?

This one is pretty advanced.

Infection method: malicious script, macro-enabled document files, or malicious link.

Cloaking: familiar branding designed to look like a legitimate e-mail.

Persuade mode: key words like “Your Invoice”, “Payment Details”, "Upcoming Shipment".

Versioning: Early versions arrived as a malicious JavaScript file (original one was used to steal bank account details by intercepting internet traffic). Later versions evolved to use macro-enabled documents to retrieve the virus payload from command and control (C&C) servers run by the attackers.

Anti-countermeasures: detects if it’s running inside a virtual machine (VM)  - will standby if it detects a sandbox environment. Emotet is a polymorphic Trojan that can evade typical signature-based detection. It has several methods for maintaining persistence, including auto-start registry keys and services. It uses modular Dynamic Link Libraries (DLLs) to continuously evolve and update its capabilities.

Update method: C&C servers, just like "Windows update" works. Updates allow additional malware installation (this is the current version) usually focused on banking Trojans, or convert itself into a system that assimilates stolen information such as financial credentials, usernames/passwords, and e-mail addresses, after which it sends them away.

Other capabilities: it can take advantage of the Windows Eternal Blue Vulnerability, starts various keep-alive services named as random numbers.

If you suspect an infection, isolate the system from the network immediately and run a deep scan with advanced anti-malware tools like Kaspersky or Malwarebytes. An alternate method is to restart in safe mode and perform a complete system restore and then scan your system with Reimage.

Other things that should be done is secure the e-mail part better and train users not to click/open everything they receive and report suspicious e-mails.


My site just got hit with a SQL injection attack, how can I tell what they were doing? So a page on my site (it's a PHP page that displays newsletter articles) was vulnerable to SQL injection and got hit. I discovered it because it was doing enough database queries to cause the CPU load to spike on the database server. I found at least some of the IP addresses that were doing it from my Apache logs and blocked them in the firewall, and I at least kind of hardened the page. What I'm trying to determine is exactly what queries they were running, they obscured/escaped stuff pretty well. This is the query string I'm finding in the Apache logs:

  ?id=742%2F--%2F%2F!30000and(select%2F--%2F1%2F--%2Ffrom(select%2F--%2Fcount()%2Cconcat((select%2F--%2F(select%2F--%2F(select%2F--%2Fconcat(0x27%2C0x7e%2Cunhex(Hex( cast(sbamail.email%2F--%2Fas%2F--%2Fchar)))%2C0x27%2C0x7e)%2F--%2Ffrom%2F--%2Femail.sbamail%2F--%2Flimit%2F--%2F284079%2C1)%2F--%2F)%2F--%2Ffrom%2F--%2Finformation_schema.tables%2F--%2Flimit%2F--%2F0%2C1)%2Cfloor(rand( 0)2))x%2F--%2Ffrom%2F--%2Finformation_schema.tables%2F--%2Fgroup%2F--%2Fby%2F--%2Fx)a)%2F--%2Fand%2F--%2F1%3D1%2F

This is what the database was showing for running queries:

  select title from newsletter_articles where article_id = 2010/--//!30000and(select/--/1/--/from(select/--/count(),concat((select/--/(select/--/(select/--/concat(0x27,0x7e,unhex(Hex(cast(sbamail.email/--/as/--/char))),0x27,0x7e)/--/from/--/email.sbamail/--/limit/--/119488,1)/--/)/--/from/--/information_schema.tables/--/limit/--/0,1),floor(rand(0)2))x/--/from/--/information_schema.tables/--/group/--/by/--/x)a)/--/and/--/1=1/

(both of those repeated Ad nauseam with minor parameter changes) When I tried pasting that query back into MySQL I get a syntax error.

When I actually paste that query string on the affected page (before I patched it), it simply displays this:

  Duplicate entry ''~redacted@redacted.com'~1' for key 'group_key'

The "Duplicate entry" bit is what really has me worried. I initially thought they were just after email addresses(there are no passwords stored in this database, fortunately); it's happened before on other pages and all I can see are "select" statements, but if they were able to insert/update/delete that's much worse news.
How can I unescape/decode what they were doing? Should I be panicking and looking at restoring the database from backup?

The page in question is on a pretty basic LAMP stack, if that's relevant at all. No framework or anything.

First you should check if your mysql user has the FILE privilege. If so, consider the system completely compromised.

The injections you provided along with the error 'duplicate entry' are a clear example of error based SQL injection.

PHP's mysql_query doesn't support more than one query, so it's not possible to insert/update/delete from a select query in your application. But if your application has an insert/update/delete query that is vulnerable to sql injection (which is very likely) then it is possible to tamper with the database.