Are there any examples of network-based application firewalls? As we know, network based firewalls will filter based on information such as source IP, source port, etc. But they won't know what program on the source machine is generating that traffic.

Complementing a network based firewall there may be a host-based firewall installed on local machines, responsible for controlling which programs are allowed network access. In some cases, these firewalls are configured to block by default.

Issues:

Malware/rogue software could disable the firewall rules on the local machine and exfiltrate data. Since the network firewall is running on a remote device, it couldn't disable that. But it wouldn't need to, as the network firewall would not know what binary on the source IP has generated that traffic.

Solution:

From a high-level view, it would seem the easiest solution to this problem would be to have a remote network based firewall capable of application level filtering (e.g. block all outgoing traffic from test.exe). But on a lower-level view it's not too easy.

I've done a few searches around network-based application firewalls but I can't seem to find much on it.

So I'm wondering if anyone's come across this before and has seen any examples?

The possible solutions I can think of would involve potentially writing a network procotol based driver to add the source application into the packet header (or a hash of the source application for added security), and then writing a sort of advanced iptables to inspect the packets for the source program field. There would be a lot of factors to consider, such as integrity checking of the source path (i.e. to check whether the malware has not simply altered the packet to make it appear to originate from a trusted application).

Any insight would be appreciated, thanks.

... have a remote network based firewall capable of application level filtering (e.g. block all outgoing traffic from test.exe).

The phrase "application level filtering" is usually used with a different meaning than yours, i.e. filtering on the application protocol level (web, mail, ...) and not on the level of a specific application program.

The exact program which caused the traffic is only known on the host where it is running but is not known inside the network. So no direct filtering can be done based on this inside the network. One could maybe associate the data seen in the network with a specific application instance running on the host if one would have some agent on the host though. But in this case one would run into the same problem as with having a host base firewall, i.e. that one cannot fully trust the host because it might be compromised.


Is it necessary to encrypt nonce in cryptographic communication? Server and client have a pre-shared key K. Assuming that that pre-shared key is safe, so nobody other than server/client can encrypt/decrypt.

Client sends a nonce N to server, server then sends back message M in encrypted form AES_CBC_Encrypt(M+N, K, IV) to client. So client can verify if there is a replay attack trying to send an expired message.

Is it necessary for client to send N in encrypted form AES_CBC_Encrypt(N, K, IV) or it is OK to just send N in clear.

(M and N are 16-byte array, M+N means concatenation of M and N).

Nonces do not necessarily have to be secret, their only use is to ensure that a message is unique.

Basically, the fact that a plaintext includes a nonce means that the ciphertext is of no use to an attacker.

An attacker being able to influence the nonce is a different issue and will likely cause some problems in protocols that depend on it.


If a message can be repeated, is that confidentiality, availability, or integrity? In my security course at university, we covered three components of network security, which were confidentiality, availability, or integrity. I'm putting together a presentation based around these 3 components, but my recollection of the course is quite fuzzy.

If a third party is able to sniff and replay a message, and thus they can perform an action like turning off a pump or sensor a second time, which component would that fall under? Suppose they can't actually understand the message and don't have direct visibility of the effect, but they have some context like time of day and frequency.

I think it was integrity, because the recipient can't trust the information, but it also affects the availability of that sensor and could perhaps be confidentiality if they know when particular messages are sent.

Which section would you put this type of attack under?

Replay attack is a standard term for this type of attack. You are right that the security property that is broken by this attack falls under the umbrella of integrity. It is an attack on the integrity of the communication as a whole. The integrity of each individual message is preserved, but not the integrity of the whole sequence of transmissions.

The attack may cause some effect in the recipient that in turn compromises the availability or confidentiality of other assets, but these compromises are not caused by the replay attack in itself. For example, if an actuator overflows a limit because it accepts too many messages to move in the same direction, and the reason the attacker could send these messages is that the communication protocol is not protected against replay, then an attack compromises the availability of the actuator, by compromising the integrity of the communication as a step in the overall attack.


How are PGP/GPG private keys stored? The default format in which SSH private keys are stored is notably fairly insecure against bruteforce attacks. This question provides a very nice solution to the problem by encrypting SSH private keys in the PKCS#8 format with 1m rounds of PBKDF2 to make computing passwords really expensive, computationally speaking.

How are PGP/GPG private keys stored? Are they computationally cheap to brute-force or do they fare a bit better than the default format used by OpenSSH? If they're cheap to brute-force, are there ways of improving the security on them?

The OpenPGP standard includes format for encoding private keys as sequences of bytes, and for symmetrically encrypting sequences of bytes with a password-derived key. While any PGP implementation is free to store private keys in any way that it sees fit, most (if not all) will use the OpenPGP formats.

The critical point in password-based encryption is how the password is derived into a key. This is all the password hashing theory: the derivation process shall be made slow (e.g. with a configurable number of iterations) and should use a variation parameter called a salt. OpenPGP includes a "decent" password-based key derivation, called Iterated and Salted S2K. If you take care to specify a high enough count (that is, as high as is tolerable on your machine), then you can get good security. With the GnuPG implementation of OpenPGP, the iteration count is specified with the --s2k-count command-line option.

Of course, the primary defence against brute force on a password is to choose a strong password, that is a password with a lot of randomness in it. Iterations and salts are just methods to cope with the not-so-strong passwords that erupt within the tangled mess of human neurons.


Can you track someone using a proxy? This is past experience, and is not the case anymore since I am a programmer now and have nothing to do with security anymore. However in the past I had a person constantly trying to make comments of spam type (definitely a troll and not a bot). So I have blocked his ip. Lo and behold he comes back saying that I can't block him because he has more proxies he can use.  I didn't know about existence of IT Security then, and now that I have found it, I can ask you guys here. This has happened in the past, and is not true anymore. But I am wondering is there a way I might have been able to track the person hiding behind proxies.

A proxy is a system which relays communications. From your server, you see the communication as coming from the proxy machine. The IP address you see is the one of the proxy. If you block that IP, you block all traffic coming from that proxy, but no traffic from elsewhere. If the villain switches to another proxy, he will pass.

Some HTTP proxy add a header line X-Forwarded-For which identifies the true source IP address. Note, though, that if the proxy system is under control of the perpetrator, then he can configure it not to include that header, or, worse, to include a fake X-Forwarded-For value.


PCI Compliance 8.2.1, 8.2.3, 8.2.4 - Which user passwords is this talking about? Say you have an e-commerce site that falls under the SAQ A-EP standard, due to extra content on the same page as the iframe that takes the credit card info.

This used to fall under the SAQ A standards, but apparently the standards have recently changed to include any site that hosts any other content on the same page as the iframe that takes the credit card info. So now most sites using Stripe/Paypal/Heartland with a JavaScript implementation all fall into this new category.

Which passwords fall under the requirements of 8.2.1, 8.2.3, and 8.2.4?

Is it

the website's user/customers
System Admins on the server hosting the site
the company that is receiving the payment info, or
all passwords connected in any way?

That document version 3 aka 3.0 (and the corresponding DSS) is 5 years old, the current version of both DSS and (all) SAQs is v3.2.1 from 2018 (although a draft of the next major version, 4.0, is scheduled for comment soon); see https://www.pcisecuritystandards.org/document_library and select Filterby=SAQS. The actual standard which the various SAQs (and audits/ROCs) are designed to implement, PCI DSS (Data Security Standard), is under the default Filterby=PCIDSS, and it says at the beginning of section 8:

  Note: These requirements are applicable for all accounts, including point-of-sale accounts, with administrative capabilities and all accounts used
  to view or access cardholder data or to access systems with cardholder data. This includes accounts used by vendors and other third parties (for
  example, for support or maintenance). These requirements do not apply to accounts used by consumers (e.g., cardholders).

  However, Requirements 8.1.1, 8.2, 8.5, 8.2.3 through 8.2.5, and 8.1.6 through 8.1.8 are not intended to apply to user accounts within a point-ofsale payment application that only have access to one card number at a time in order to facilitate a single transaction (such as cashier accounts).

See particularly the last sentence of the first paragraph. Since A and A-EP systems don't actually receive or store cardholder data, it's their ability to control access to it that matters.

SAQA-EP 3.2.1 reflects this, briefly, in item 8.1, emphasis added:

  Are policies and procedures for user identification
  management controls defined and in place for nonconsumer users and administrators on all system
  components, as follows:

but 3.0 did not (according to my saved copy), and even now it's not made obvious this applies throughout all of 8.1.* 8.2.* and 8.3.*. Also note A-EP 3.0 did not include DSS 8.2.2 but 3.2.1 does.

Neardupe To whom do the PCI DSS password requirements apply? which answers essentially the same for 3.2 in 2017, before 3.2.1 was published.


How does SSL/TLS work? How does SSL work? I just realised we don't actually have a definitive answer here, and it's something worth covering.

I'd like to see details in terms of:

A high level description of the protocol.
How the key exchange works.
How authenticity, integrity and confidentiality are enforced.
What the purpose of having CAs is, and how they issue certificates.
Details of any important technologies and standards (e.g. PKCS) that are involved.

General
SSL (and its successor, TLS) is a protocol that operates directly on top of TCP (although there are also implementations for datagram based protocols such as UDP). This way, protocols on higher layers (such as HTTP) can be left unchanged while still providing a secure connection. Underneath the SSL layer, HTTP is identical to HTTPS.
When using SSL/TLS correctly, all an attacker can see on the cable is which IP and port you are connected to, roughly how much data you are sending, and what encryption and compression are used. He can also terminate the connection, but both sides will know that the connection has been interrupted by a third party.
In typical use, the attacker will also be able to figure out which hostname you're connecting to (but not the rest of the URL): although HTTPS itself does not expose the hostname, your browser will usually need to make a DNS request first to find out what IP address to send the request to.
High-level description of the protocol
After building a TCP connection, the SSL handshake is started by the client. The client (which can be a browser as well as any other program such as Windows Update or PuTTY) sends a number of specifications:

which version of SSL/TLS it is running,
what ciphersuites it wants to use, and
what compression methods it wants to use.

The server identifies the highest SSL/TLS version supported by both it and the client, picks a ciphersuite from one of the client's options (if it supports one), and optionally picks a compression method.
After this the basic setup is done, the server sends its certificate. This certificate must be trusted by either the client itself or a party that the client trusts. For example, if the client trusts GeoTrust, then the client can trust the certificate from Google.com because GeoTrust cryptographically signed Google's certificate.
Having verified the certificate and being certain this server really is who he claims to be (and not a man in the middle), a key is exchanged. This can be a public key, a "PreMasterSecret" or simply nothing, depending on the chosen ciphersuite. Both the server and the client can now compute the key for the symmetric encryption whynot PKE?. The client tells the server that from now on, all communication will be encrypted, and sends an encrypted and authenticated message to the server.
The server verifies that the MAC (used for authentication) is correct and that the message can be correctly decrypted. It then returns a message, which the client verifies as well.
The handshake is now finished, and the two hosts can communicate securely. For more info, see technet.microsoft.com/en-us/library/cc785811 and en.wikipedia.org/wiki/Secure_Sockets_Layer.
To close the connection, a close_notify 'alert' is used. If an attacker tries to terminate the connection by finishing the TCP connection (injecting a FIN packet), both sides will know the connection was improperly terminated. The connection cannot be compromised by this though, merely interrupted.
Some more details
Why can you trust Google.com by trusting GeoTrust?
A website wants to communicate with you securely. In order to prove its identity and make sure that it is not an attacker, you must have the server's public key. However, you can hardly store all keys from all websites on earth, the database would be huge and updates would have to run every hour!
The solution to this is Certificate Authorities, or CA for short. When you installed your operating system or browser, a list of trusted CAs probably came with it. This list can be modified at will; you can remove whom you don't trust, add others, or even make your own CA (though you will be the only one trusting this CA, so it's not much use for public website). In this CA list, the CA's public key is also stored.
When Google's server sends you its certificate, it also mentions it is signed by GeoTrust. If you trust GeoTrust, you can verify (using GeoTrust's public key) that GeoTrust really did sign the server's certificate. To sign a certificate yourself, you need the private key, which is only known to GeoTrust. This way an attacker cannot sign a certificate himself and incorrectly claim to be Google.com. When the certificate has been modified by even one bit, the sign will be incorrect and the client will reject it.
So if I know the public key, the server can prove its identity?
Yes. Typically, the public key encrypts and the private key decrypts. Encrypt a message with the server's public key, send it, and if the server can repeat back the original message, it just proved that it got the private key without revealing the key.
This is why it is so important to be able to trust the public key: anyone can generate a private/public key pair, also an attacker. You don't want to end up using the public key of an attacker!
If one of the CAs that you trust is compromised, an attacker can use the stolen private key to sign a certificate for any website they like. When the attacker can send a forged certificate to your client, signed by himself with the private key from a CA that you trust, your client doesn't know that the public key is a forged one, signed with a stolen private key.
But a CA can make me trust any server they want!
Yes, and that is where the trust comes in. You have to trust the CA not to make certificates as they please. When organizations like Microsoft, Apple, and Mozilla trust a CA though, the CA must have audits; another organization checks on them periodically to make sure everything is still running according to the rules.
Issuing a certificate is done if, and only if, the registrant can prove they own the domain that the certificate is issued for.
What is this MAC for message authentication?
Every message is signed with a so-called Message Authentication Code, or MAC for short. If we agree on a key and hashing cipher, you can verify that my message comes from me, and I can verify that your message comes from you.
For example with the key "correct horse battery staple" and the message "example", I can compute the MAC "58393". When I send this message with the MAC to you (you already know the key), you can perform the same computation and match up the computed MAC with the MAC that I sent.
An attacker can modify the message but does not know the key. He cannot compute the correct MAC, and you will know the message is not authentic.
By including a sequence number when computing the MAC, you can eliminate replay attacks. SSL does this.
You said the client sends a key, which is then used to setup symmetric encryption. What prevents an attacker from using it?
The server's public key does. Since we have verified that the public key really belongs to the server and no one else, we can encrypt the key using the public key. When the server receives this, he can decrypt it with the private key. When anyone else receives it, they cannot decrypt it.
This is also why key size matters: The larger the public and private key, the harder it is to crack the key that the client sends to the server.
How to crack SSL
In summary:

Try if the user ignores certificate warnings;
The application may load data from an unencrypted channel (e.g. HTTP), which can be tampered with;
An unprotected login page that submits to HTTPS may be modified so that it submits to HTTP;
Unpatched applications may be vulnerable for exploits like BEAST and CRIME;
Resort to other methods such as a physical attack;
Exploit side channels like message length and the time taken to form the message;
Wait for quantum attacks.

See also: A scheme with many attack vectors against SSL by Ivan Ristic (png)
In detail:
There is no simple and straight-forward way; SSL is secure when done correctly. An attacker can try if the user ignores certificate warnings though, which would break the security instantly. When a user does this, the attacker doesn't need a private key from a CA to forge a certificate, he merely has to send a certificate of his own.
Another way would be by a flaw in the application (server- or client-side). An easy example is in websites: if one of the resources used by the website (such as an image or a script) is loaded over HTTP, the confidentiality cannot be guaranteed anymore. Even though browsers do not send the HTTP Referer header when requesting non-secure resources from a secure page (source), it is still possible for someone eavesdropping on traffic to guess where you're visiting from; for example, if they know images X, Y, and Z are used on one page, they can guess you are visiting that page when they see your browser request those three images at once. Additionally, when loading Javascript, the entire page can be compromised. An attacker can execute any script on the page, modifying for example to whom the bank transaction will go.
When this happens (a resource being loaded over HTTP), the browser gives a mixed-content warning: Chrome, Firefox, Internet Explorer 9
Another trick for HTTP is when the login page is not secured, and it submits to an https page. "Great," the developer probably thought, "now I save server load and the password is still sent encrypted!" The problem is sslstrip, a tool that modifies the insecure login page so that it submits somewhere so that the attacker can read it.
There have also been various attacks in the past few years, such as the TLS renegotiation vulnerability, sslsniff, BEAST, and very recently, CRIME. All common browsers are protected against all of these attacks though, so these vulnerabilities are no risk if you are running an up-to-date browser.
Last but not least, you can resort to other methods to obtain the info that SSL denies you to obtain. If you can already see and tamper with the user's connection, it might not be that hard to replace one of his/her .exe downloads with a keylogger, or simply to physically attack that person. Cryptography may be rather secure, but humans and human error is still a weak factor. According to this paper by Verizon, 10% of the data breaches involved physical attacks (see page 3), so it's certainly something to keep in mind.


Are there use cases where a mutually authenticated TLS is preferred? I am trying to see if there are use cases where a mutually authenticated TLS is preferred over a one way authentication.

Usually the authentication of the client/user is done over the TLS tunnel.

Is a p2p connection a good use case for mutual authentication? In such a scenario each end will somehow prove that they are part of the network. However, in this case the authentication/verification can happen over a one-way authenticated tunnel. What would mandate a mutual auth here?

Are there any other cases I am missing?

There's a lot of potential benefits to mutual authentication using TLS. Here's a few.

You don't need to keep a username/pass database
No worry about password reuse.
Credentials can be stored on physical tokens/devices.
Credentials aren't going to be written on post-it.
Users don't have to enter credentials when visiting your resource (certs are automatically used).
Less concern about TLS interception proxies as you're authenticating the client.
There's little chance of brute forcing a certificate (no worry of password dictionary/brute force)

You would then be reliant on a centralized authority to issue or sign your certificates.


How to find the OS of the target machine I am new to this field. Currently I am using Linux OS and
my question is how can I find the OS of another machine to
which I do not have a direct access.

I am interested in any possibilities which can lead me to
this information.

Also I need to detect the OS precisely.

I used nmap -O  before.

If you can send packets to the target machine, use nmap -O, which provides OS fingerprinting.

If you can eavesdrop/intercept network traffic with the target machine, use pof, a tool for passive OS fingerprinting.

You didn't provide much information about what are your constraints or why the standard tools (like nmap or pof) didn't work for you.  Therefore, I'll provide some generic advice: I suggest you look into OS fingerprinting.


Is Visa PayWave secure? Recently (well, a few months or even a year ago), my bank, here in Australia, introduced this PayWave technology to their Visa debit cards. They claim it's secure, but all they talk about is their policies, which requires you to notice that there's a problem (From the website):

  Visa payWave-enabled cards are backed by Visa's Zero Liability Policy1
  and are as secure as any other Visa chip card. They carry the same
  multiple layers of security, which ensures that you are not
  responsible for fraudulent or unauthorised transactions.

It obviously doesn't carry the same layers of security, as you don't have to use your PIN or signature. Well, for purchases under $100, but even that is a lot of money to some people (any amount is a lot to me), so it's a bit weird to say that anything under that is a "low-value" purchase for anyone. If a homeless person found/stole my card, I don't see them buying anything above that.

Someone I was talking to even suggested that a person could walk down the street with one of these PayWave machines in their bag, and bump it against other people's bags in the hopes of finding a compatible card.

It sounds like they're sacrificing security for convenience, and underestimating the ingenuity of thieves. I can't seem to find any information that isn't advertising or otherwise biased, and I don't have any expertise in this area at all to tell the difference between a legitimate concern and paranoia. Is Visa PayWave as secure as it claims?

The secret of credit card transaction security is that by law (in many jurisdictions) the card issuer is responsible for fraudulent transactions after a certain limit, not the card holder. Since they already assume the bulk of the liability, most (all?) simply make the jump to say that the card holder is not responsible under any circumstances for fraud.

This means that card security is simply a cost-benefit trade-off for the issuer. It's worth the cost for the issuer of writing off a certain amount of fraud if in exchange they get a reasonable return for the policy. Hence the $100 limit; Visa is confident that within that range, they can detect fraud reliably enough to make it worth their while to remove certain security measures.

Therefore, whether or not it's secure is their problem, not yours. Obviously you have to actually report suspicious transactions. This makes it a bad idea to keep an account open that you don't monitor. But that has always been the case.

This is really the way it should be. If the party liable for security is the one in the best position to implement it, then the level of security you get tends to be appropriate for the value of the thing that's being secured.


What are the security considerations for setting up a basic security testing environment? I only have 2 spare machines and both are very old, but I really want to set up an environment so that I can start practicing security. One of these machines has a Pentium 4 processor and 512 MB RAM; the other is older (and I don't know the specs off the top of my head).

I want to set up a VM environment on these machines, if possible.
What would be the best way to do it, to keep the rest of my network secure? Would a better option be to leave them as they are (Windows XP box and a Ubuntu 10.04 server) and change some settings/install vulnerable software (burp suite, web goat, etc)?

What other issues should I consider, from a security point of view?

You should still be able to run VM's quite successfully on these boxes. The key issue is that they will be slow, so if you want to run brute forcing on them (eg with John the Ripper) they will take a lot longer than recent machines.

I have a couple of VM's running quite successfully on kit much older than that - for basic victim VM's (one has an app similar to webgoat) they work perfectly well.


Should we use API keys for authorization on a public API (instead of JWT)? We are building a public-facing API. The consumers of this API will be business partners of ours - we will probably have a personal relationship with each one, and I suspect there will only ever be 10-40 of these customers. So we're not dealing with the same problems that e.g. Spotify/Facebook might with their public APIs.

We want to give each user different API read permissions, so some level of authorization is required.

We were going to issue each customer with an API key, and store the related permissions in a table. But now we're wondering if we should implement JWTs to handle this (or even another option like OAuth).

Pros of API keys:

Simple. Easy for the clients to implement (fairly important for us)
Makes rate limiting simpler, as there will be fewer auth calls.
Cheaper (we are using AWS lambda so paying per request)
Quicker for us to build too

Pros of JWTs

More established way of handling authorization. No need to reinvent the wheel.
Avoid any unforeseen pitfalls around implementing your own access mechanism
Fewer database look-ups on our end (but presumably slower flow overall)
...any others I've missed here?

We're leaning towards API keys for simplicity, but want to ensure we're not missing something important.

I believe API keys are actually an older convention for auth than JWTs (which were only standardized in 2010); you certainly aren't re-inventing any wheels to use them. This scenario definitely seems ripe for using them, whereas I think JWTs would actually add quite a bit of needless complexity.

JWTs are especially valuable for certain authentication and/or authorization scenarios, but I don't think any of them apply here. You're presumably not separating authentication from authorization (JWTs allow one party to authenticate a user and then another party to provide - or deny - service based on that). You're keeping a small pool or users and presumably will have few active connections and request rates, so cutting one database (or, realistically, data cache) lookup per request isn't likely to be urgent. It doesn't sound like your partners will likely expect multiple tokens each with a minimal set of permissions - it's more secure in some scenarios, but definitely more complicated to implement - and you could do that with API keys anyhow.

On the other hand, JWTs introduce significant risks. JWT-parsing libraries need to be secure against a number of attacks that just don't come up with API keys (those have their own risks, but they're mostly well-known things like SQL Injection, brute-forcing, or timing attacks). JWTs require generating, storing, using, and hopefully rotating cryptographic keys; they may also require distributing them if you're separating the authentication provider from the service provider. JWTs are hard to revoke, so they're usually made short-lived, meaning you need to support some other authentication token (like a DB-stored refresh token) or low-friction auth method, at which point you might as well use the API key anyhow.

Another option (which may or may not be suitable for your customers or API) would be to use TLS client certificates. These can be more secure than either of the approaches discussed (the control and simple processing of individually issued and revoked keys, the built-in expiry and impossibility to brute-force of JWTs, and unlike either you avoid ever transmitting the actual secret over the wire even within a TLS tunnel and "accidentally send the token via plain text" isn't even a meaningful risk). They also have drawbacks, though (requires handling key material, makes enrollment more complicated, makes it harder for an individual user to enable access for multiple devices, rotation can be more complicated in the event of a compromise, users are likely just less familiar with them). I wouldn't even mention it except that you're expecting a very small number of users and a working relationship with them.


oauth and passing received toking to other application An external software provider is hosting a website for us. We supply the login functionality via a restfull api(login/password over ssl). But the website is also the client for OAuth resource that it can access. This would imply that the user of the website who is already logging in with it's website credentials should login again to the OAuth resource.

We would like to make it easy for the user. So what are the option to not make a double login?

As i understand OAuth correct, it is not a good idea to store the token. So is it possible with every restfull api login to also do a OAuth login and pass the token to the website?

A user can login once and use OAuth, and this token can be used to access a 3rd party resource.  This design is common, especially for webapps that want to tap into the Facebook API.

As a pentester I find nearly every OAuth implantation is in violation of at least one of OAuth's RFC security  requirements.  It is as if developers aren't even reading RFCs anymore * gasp *.

RFC-5839 section 4.5 states it acceptable to store the shared secret and OAuth token,  as long as they not stored in plain-text.  Hashing the credentials is preferred, however this token can be encrypted and retrieved without introducing a significant security threat or violating an RFC security requirement.


Encryption and Signature with same RSA private key Talking about cipher-suites that use RSA both for sign of DH parameters and for digital signature. The keys used for these process are the same, or is there a public and private key for each one?

If the keys are different, explain a scenario that could be dangerous to use the same key for encryption and signature.

What advantages a hacker could gain if the keys was the same?

Is there some situation where using the same key is good?

Why are they different?

I suppose that you are talking about SSL. A SSL server supports several combinations of cryptographic algorithms called cipher suites. In particular, a given SSL server may support either or both of these key exchange mechanisms:

RSA: the server has a RSA private key; the client generates a random value (the "pre-master secret") and encrypts it with the server's RSA public key; the server decrypts the received value to get the pre-master secret.
DHE_RSA: the server has a RSA private key; the server generates a new random Diffie-Hellman key pair, signs the public part with its RSA private key, and sends that to the client; the client verifies the RSA signature (using the server's RSA public key) and completes the DH key exchange. (Variant: ECDHE_RSA: same principle, but the DH is done over an elliptic curve.)

So indeed, a given SSL server may potentially use the same RSA private key for both encryption (when using a RSA key exchange) and signatures (when doing a DHE_RSA key exchange). Note that the server may perfectly well have two RSA keys, one dedicated to encryption (RSA key exchange) and one for signatures (DHE_RSA key exchange); people just do not do that on a regular basis because having two distinct keys implies buying two distinct certificates.

In all generality, the same key should not be used for both encryption and signatures. Note that a big source of confusion is that RSA encryption and RSA signatures are two distinct algorithms, that just happen to use private keys that have the same format, and people found it smart to call both of them "RSA".

The two main reasons for not using the same private key for both encryption and signatures are the following:

Interactions between the encryption and the signature algorithm are not well-studied. There could exist some method that leverages encryption to forge signatures, or signatures to perform an underhanded decryption. In the specific case of a SSL server, it is usually believed that there is no such interaction, notably because when the server decrypts, it does not advertise the result back to the client, and also because when the server signs, it does so over a DH public key that the server generated itself, not over data sent by the client.
Normally, encryption keys and signature keys call for distinct life cycles; encryption keys should be escrowed, and signature keys must not be escrowed (see this for some discussion on the subject). Again, in the specific case of a SSL server, you do not need to escrow the RSA private key because the encryption is transient (data is decrypted when it arrives, not stored), so it is possible to manage the server's private key in a way which is fine both for an encryption key and a signature key.


How do the processes for digital certificates, signatures and ssl work? I have been trying to understand how ssl works. Instead of Alice and Bob, lets consider client and server communication.
Server has a digital certificate acquired from a CA. It also has public and private keys.
Server wants to send a message to Client. Server's public key is already available to client.

Assuming that ssl handshake is completed.

Server to Client:

Server attaches its public key to the message.
Runs hash function on (message+public key). Results is known as HMAC.
Encrypt HMAC using it's private key. Result is called digital
signature.
Send it to Client along with the digital certificate.
Client checks the certificate and finds that it's from the expected
Server.
Decrypts HMAC using Server's public key.
Runs the hash function on (message+public key) to obtain the original
message.

Client to Server

Client runs hash function on (message+public key) and then encrypts
using the same public key.
Server decrypts using private key, runs the hash function on the
resultant data to obtain the message.

Please let me know if my understanding is correct.

There are a few confusions in your post. First of all, HMAC is not a hash function. More about HMAC later on.

Hash Functions

A hash function is a completely public algorithm (no key in that) which mashes bit together in a way which is truly infeasible to untangle: anybody can run the hash function on any data, but finding the data back from the hash output appears to be much beyond our wit. The hash output has a fixed size, typically 256 bits (with SHA-256) or 512 bits (with SHA-512). The SHA-* function which outputs 160 bits is called SHA-1, not SHA-160, because cryptographers left to their own devices can remain reasonable for only that long, and certainly not beyond the fifth pint.

Signature Algorithms

A signature algorithm uses a pair of keys, which are mathematically linked together, the private key and the public key (recomputing the private key from the public key is theoretically feasible but too hard to do in practice, even with Really Big Computers, which is why the public key and be made public while the private key remains private). Using the mathematical structure of the keys, the signature algorithm allows:

to generate a signature on some input data, using the private key (the signature is a mathematical object which is reasonably compact, e.g. a few hundred bytes for a typical RSA signature);
to verify a signature on some input data, using the public key. Verification takes as parameters the signature, the input data, and the public key, and returns either "perfect, man !" or "dude, these don't match".

For a secure signature algorithm, it is supposedly unfeasible to produce a signature value and input data such that the verification algorithm with a given public key says "good", unless you know the corresponding private key, in which case it is easy and efficient. Note the fine print: without the private key, you cannot conjure some data and a signature value which work with the public key, even if you can choose the data and the signature as you wish.

"Supposedly unfeasible" means that all the smart cryptographers in the world worked on it for several years and yet did not find a way to do it, even after the fifth pint.

Most (actually, all) signature algorithms begin by processing the input data with a hash function, and then work on the hash value alone. This is because the signature algorithm needs mathematical objects in some given sets which are limited in size, so they need to work on values which are "not too big", such as the output of a hash function. Due to the nature of the hash function, things work out just well (signing the hash output is as good as signing the hash input).

Key Exchange and Asymmetric Encryption

A key exchange protocol is a protocol in which both parties throw mathematical objects at each other, each object being possibly linked with some secret values that they keep for them, in a way much similar to public/private key pairs. At the end of the key exchange, both parties can compute a common "value" (yet another mathematical object) which totally evades the grasp of whoever observed the bits which were exchanged on the wire. One common type of key exchange algorithm is asymmetric encryption. Asymmetric encryption uses a public/private key pair (not necessarily the same kind than for a signature algorithm):

With the public key you can encrypt a piece of data. That data is usually constrained in size (e.g. no more than 117 bytes for RSA with a 1024-bit public key). Encryption result is, guess what, a mathematical object which can be encoded into a sequence of bytes.
With the private key, you can decrypt, i.e. do the reverse operation and recover the initial input data. It is assumed that without the private key, tough luck.

Then the key exchange protocol runs thus: one party chooses a random value (a sequence of random bytes), encrypts that with the peer's public key, and sends him that. The peer uses his private key to decrypt, and recovers the random value, which is the shared secret.

An historical explanation of signatures is: "encryption with the private key, decryption with the public key". Forget that explanation. It is wrong. It may be true only for a specific algorithm (RSA), and, then again, only for a bastardized-down version of RSA which actually fails to have any decent security. So no, digital signatures are not asymmetric encryption "in reverse".

Symmetric Cryptography

Once two parties have a shared secret value, they can use symmetric cryptography to exchange further data in a confidential way. It is called symmetric because both parties have the same key, i.e. the same knowledge, i.e. the same power. No more private/public dichotomy. Two primitives are used:

Symmetric encryption: how to mangle data and unmangle it later on.
Message Authentication Codes: a "keyed checksum": only people knowing the secret key can compute the MAC on some data (it is like a signature algorithm in which the private and the public key are identical -- so the "public" key had better be not public !).

HMAC is a kind of MAC which is built over hash functions in a smart way, because there are many non-smart ways to do it, and which fail due to subtle details on what a hash function provides and does NOT provide.

Certificates

A certificate is a container for a public key. With the tools explained above, one can begin to envision that the server will have a public key, which the client will use to make a key exchange with the server. But how does the client make sure that he is using the right server's public key, and not that of a devious attacker, a villain who cunningly impersonates the server ? That's where certificates come into play. A certificate is signed by someone who is specialized in verifying physical identities; it is called a Certificate Authority. The CA meets the server "in real life" (e.g. in a bar), verifies the server identity, gets the server public key from the server himself, and signs the whole lot (server identity and public key). This results in a nifty bundle which is called a certificate. The certificate represents the guarantee by the CA that the name and public key match each other (hopefully, the CA is not too gullible, so the guarantee is reliable -- preferably, the CA does not sign certificates after its fifth pint).

The client, upon seeing the certificate, can verify the signature on the certificate relatively to the CA public key, and thus gain confidence in that the server public key really belongs to the intended server.

But, you would tell me, what have we gained ? We must still know a public key, namely the CA public key. How do we verify that one ? Well, we can use another CA. This just moves the issue around, but it can end up with the problem of knowing a priori a unique or a handful of public keys from über-CAs which are not signed by anybody else. Thoughtfully, Microsoft embedded about a hundred of such "root public keys" (also called "trust anchors") deep within Internet Explorer itself. This is where trust originates (precisely, you forfeited the basis of your trust to the Redmond firm -- now you understand how Bill Gates became the richest guy in the world ?).

SSL

Now let's put it all together, in the SSL protocol, which is now known as TLS ("SSL" was the protocol name when it was a property of Netscape Corporation).

The client wishes to talk to the server. It sends a message ("ClientHello") which contains a bunch of administrative data, such as the list of encryption algorithms that the client supports.

The server responds ("ServerHello") by telling which algorithms will be used; then the server sends his certificate ("Certificate"), possibly with a few CA certificates in case the client may need them (not root certificates, but intermediate, underling-CA certificates).

The client verifies the server certificate and extracts the server public key from it. The client generates a random value ("pre-master secret"), encrypts it with the server public key, and sends that to the server ("ClientKeyExchange").

The server decrypts the message, obtains the pre-master, and derive from it secret keys for symmetric encryption and MAC. The client performs the same computation.

Client sends a verification message ("Finished") which is encrypted and MACed with the derived keys. The server verifies that the Finished message is proper, and sends its own "Finished" message in response. At that point, both client and server have all the symmetric keys they need, and know that the "handshake" has succeeded. Application data (e.g. an HTTP request) is then exchanged, using the symmetric encryption and MAC.

There is no public key or certificate involved in the process beyond the handshake. Just symmetric encryption (e.g. 3DES, AES or RC4) and MAC (normally HMAC with SHA-1 or SHA-256).


how to debug for DNS changing Maleware on (linux/android)? So by opening an application/browser on my Android device, I found that it pops open a set of ads. I channeled a proxy through my laptop trying to track the first web site it connects to. Later I found out that my router DNS is the same as that server that pops open the ads. Also I found that my /etc/resolv.conf contains the same DNS but it was printed by the network manager. So scenarios are:

My mobile device was infected and attacker accessed my router since it has different kinds of vulnerabilities and changed the DNS there. Then network manager prints it to my /etc/resolv.conf
My laptop was infected (highly unlikely since its a new device nothing untrusted installed)
Attacker accessed the router first then some how figured to infect apps on my android device and this DNS change in /etc/resolv.conf also happened by network manager just copying router's dns to my system

So the question is how to debug this situation? How to figure out the core problem here, also many apps are infected on my android device? I tried different anti-virus but no luck. Also I would appreciate a word of advice on reporting the attacker here since I believe this could be a sort of mass attacks on the routers within my IP address range.

It is most likely that your router was infected and android takes its dns server from dhcp. There are a few driveby attacks that work on routers like this.


Is it possible to convert a gpg key to a different encryption algorithm Maybe the answer to this question is a simple and unequivocal "No", followed by "How can you possibly think that this is possible?". And if I think about it rationally this seems to be the only answer. But programming and math have taught me that a lot of things one intuitively judges as impossible or pointless may have a surprising answer. So here comes one of those questions.

Is it possible to convert a gpg RSA key to an ECC key? Or, to generalize, is it possible to switch between different cryptography algorithms for one and the same key such that one retains the key's functionality; signing and encrypting etc. If so how does this work and if not, why not?

No, not really.  You can't switch because ECC and RSA are totally different algorithms that use totally different keys.  The best you can do is generate a new ECC subkey which will be signed by your old RSA master key, then you can use that subkey for signing and/or encryption.


How to actually calculate Pairwise Transient Key? I have following information:

PTK-Pairwise Transient Key: A value derived from PMK, Authenicator nonce(Anonce), Supplicant nonce(Snonce), Authenticator Address, Supplicant Address. This is used to encrypt all unicast transmission between client & an AP. PTK consist of 5 different keys. They are:

1. KCK - Key Confirmation Key used to provide data integrity during 4 -Way Handshake & Group Key Handshake.

2. KEK – Key Encryption Key used by EAPOL-Key frames to provide data privacy during 4-Way Handshake & Group Key Handshake.

3. Temporal Key – used to encrypt & decrypt MSDU of 802.11 data frames between supplicant & authenticator

4. Temporal MIC-1

5. Temporal MIC-2

And also, my lecture slides says:

  PTK = HMAC_SHA_1(PMK ||
  AP_nonce || STAnonce ||
  AP_MACaddr || STA_MACaddr)

Now my question is, how can I actually calculate PTK if following keys are given?

KCK = "aaaaaaaaaaaaaaaa"

KEK = "bbbbbbbbbbbbbbbb"

TK = "cccccccccccccccc"

The PTK is split on the 5 keys you mentioned.

16 bytes for KCK
16 bytes for KEK
16 bytes for TK
8 bytes for MIC-1 key for transmitting
8 bytes for MIC-2 key for receiving

To recreate the PTK you need to have all the above values. I believe you are missing the latter 2 keys.


Email Address Spoofed Starting this morning at 9:30 am I started receiving numerous emails in my AOL account thanking me for registering at their website and asking me to confirm my registration.  I am not opening them at all and are just moving to SPAM. They are coming from inside the US as well as worldwide and now total many many hundreds and are coming every few minutes.  There is nothing suspicious in my sent items folder. What can I do other than shutting down this AOL email account and start using gmail?

Again this is an AOL account but I am providing an alternative gmail account for your requirements.

Shutting down AOL account is not going to do any good for you. It could happen for your gmail account too. You should find out root cause.

If you are suddenly getting "spam bombed", it has been suggested that you check any accounts such as PayPal, Amazon, eBay etc. where you may have money.  Sometimes spam bombers do this in the hope that you will miss emails from these types of accounts informing you of a withdrawal/theft of funds.
You should report the emails as "SPAM" to AOL
It Might be a good time to check your account security.
Finally, account security is your responsibility; so you should make sure that you do not provide your email address in random places.


Spam email "via" my domain, but SPF record exists I just got an email from some random person's name "via" info@mydomain.com, although info@mydomain.com is just a distribution group within G Suite.

We have an up-to-date SPF record added from Google, and I'm not quite sure what or how another person is able to send an email via my domain.

Here are some references from the message source, without giving any specific information to my domain or the receipients:

Date: Mon, 01 Apr 2019 23:41:44 -0500
Subject: Mass Shootings orchestrated to pass gun control for the Federal
 Reserve Shareholders planned U.S. Holocaust- How can your industry help?
From: 'Random Person' via Info <info@mydomain.com>
<snipped>
Message-ID: <186271992.14957742.1554180104822@mail.yahoo.com>
Thread-Topic: Mass Shootings orchestrated to pass gun control for the Federal
 Reserve Shareholders planned U.S. Holocaust- How can your industry help?
Thread-Index: AWY0NTc5UrmPA22gl2edULFwYvLC7TIwMTU5
References: <186271992.14957742.1554180104822.ref@mail.yahoo.com>
Mime-version: 1.0
Content-type: multipart/alternative;
    boundary="B_3637013229_1574776269"

All of our users have 2FA enabled, although I don't think that's relevant here. This is clearly a spoofed emailed as info@mydomain.com is not a registered account within the domain (just verified it).

Any ideas how this may have happened and how to prevent it?

Also, this message doesn't seem to contain any valuable information other than it was potentially leveraging Yahoo to send email on "behalf" of my domain, which I'm not quite sure how that worked.

Having an SPF record in your DNS records helps the recipient know which email server is legtimiate for your domain. The recipient looks up the sending domain for the valid server IPs and then decides what to do with the email.

If the sending IP is on the list, then the email is likely OK.
If the sending IP is not on the list, then it should be treated suspiciously.

This checking logic requires that the receiving email server is configured to check for SPF records. If you are not checking for SPF records, then the entire SPF checking process is not done.

If your email headers do not include the SPF fields, then your email server is not set up to check for SPF and it is not protecting your company in this way.

You need to look up your email service documentation to figure out how to turn on SPF checking.


How can I control all my website's requests? I need to manage my website's requests. Because of both I need to detect suspect requests and block that IP and calculate the number of users which visit my website.

So I've created a table like this:

// requests
+----+----------------+-------------+
| id |  UserId_or_IP  |  unix_time  |
+----+----------------+-------------+
| 1  | 12353          | 1339412843  |
| 2  | 23353          | 1339412864  |
| 3  | 178.253.29.175 | 1339412894  |
| 4  | 3422           | 1339412899  |
| 5  | 342232         | 1339412906  |
| 6  | 2353           | 1339412906  |
| 7  | 7785           | 1339412951  |
| 8  | 148.23.29.109  | 1339413640  |
| 9  | 178.253.29.175 | 1339413621  |
| 10 | 5462           | 1339414490  |
| 11 | 342232         | 1339414923  |
| 12 | 178.253.29.175 | 1339419901  |
| 13 | 8007           | 1339424860  |
| 14 | 8007           | 1339424822  |
| 15 | 2353           | 1339424902  |
+----+----------------+-------------+

And I insert a new row into that table for every request (opening every page of my website means one request).

And here is my query in the index.php: (index.php is a page which all pages pass from it)

INSERT INTO requests (id, UserId_or_IP, unix_time)
            VALUES   (NULL, :id_or_ip, UNIX_TIME());

All I need to know: What I'm doing is correct or there is a better approach to control requests? As you know That query makes my webpage-loading slow (a bit), because there is an INSERT query before loading every page. Well has it worth? In fact I'm trying to understand what's the standard way for doing that.

In fact I'm trying to understand what's the standard way for doing that.

On a properly configured web server, the web server will log all received requests in a log file (the exact content of this log file is usually more or less configurable, depending on the server software used). There are also standard means to centralize the logs if you want to like syslog which will allow you to store and analyze the logs on a different machine.

The content of this log can then be analyzed by external software in order to either:

Generate statistics, there are plenty of monitoring software depending on your exact software and needs,
Detect suspicious behavior, here you will search for IDS / IPS software, depending if you need Detection only (IDS), or Protection too (IPS). Fail2Ban for instance is a classical example of such software for Linux/Unix platforms: it will monitor your server log files, detect suspicious behavior according to some set of rules and optionally trigger an action (send an alert email or update firewall rules to ban the offender for instance).

You did not asked for it in your question, but speaking of standard practice, generally the logs are also analyzed in order to analyze the health of the server software and detect possible issues and outage. As for IDS/IPS, you have the option to have a simple detection & alert system or automatic actions.

With this, usage stats + health stats + security issues, you get a pretty complete view of your server activity.


Data Protection act, who is liable? Recently, a large UK shopping chain had the staff payroll database leaked (including bank details, all unencrypted)

We've been told that we cannot take legal action and have so far been denied compensation as although our data was stolen, it wasn't an external breach, it was someone internal leaking the data with privileged access.

In this situation, who takes the blame?

Whether a data leak is because of internal or external factors does not matter. In the situation you describe the UK company is the data controller and solely responsible for taking the appropriate measures to avoid data leaks. Even if they used a processor this processor may be liable, but the UK company is still responsible.

See the basis for all European Data Protection law, the 95/46 directive.

You can file a complaint with the ICO, the UK data protection authority, their conclusions will greatly help you in any legal follow-ups.

Keep in mind that for compensation you will likely have to illustrate that you suffered damages and need to show how you quantify these damages.


When I connect my PC to a network, what (meta) information can the router/admin see that identifies my PC? If I connect my Windows 7 PC to a third-party (wireless) network what information might the network admin be able to obtain about me?

I know they can (sometimes) see traffic but that's not an issue as I always run a VPN.

I'm talking about other identifying information. Presumably they can see my MAC address and timing and volume of usage. What about the assigned name of the PC? Anything else that identifies my PC?

Windows systems tend to be chatty. For instance, at many times they will try to discover other Windows machines on the same LAN, or printers; and they will announce themselves similarly. This behaviour is largely muted if you inform Windows that you are on a "public network" (see this page). If you let your Windows behave as if it was at home, then it will begin to shout information across the LAN, including its name.

Even if you configure Windows to be discreet, the sysadmin may still, through passive eavesdropping, notice the presence of your machine, see its MAC address, observe the connection from your machine to your VPN server (and thus know not only that you use a VPN, but also the IP address of the VPN server). Your machine may also issue some out-of-VPN network accesses (e.g. to Windows Update), and the sysadmin will see those. If the sysadmin is a bit more nosy, he may begin to actively scan your machine for open network services, and analyse emitted packets to infer your operating system brand and version (e.g. with nmap).

The MAC address alone is (normally) already sufficient to track you, i.e. notice that it is still you if you disconnect and reconnect later on. The MAC address can be changed in software, but the other elements (IP address of the VPN server, OS type and version...) will most probably allow reliable tracking nonetheless.

If you use a VPN and the VPN is any good, then the nosy sysadmin will not be able to see the contents of your traffic; however, he may still infer things because he will know when you exchange data and how much data you exchange. E.g. he will be able to distinguish between Web browsing and online gaming. He will also be able to distinguish between accesses to Youtube and to Stack Exchange, if he already knows somehow that you are connecting to one of these two sites.

If your VPN is poorly configured (i.e. if your DNS requests are not sent through the VPN), then the sysadmin will know the names of all the sites you browse.


How often do security patches break business applications Are there any statistics on how reliable security patches are? Such as the fraction recalled or corrected?

Part of keeping a computer secure is applying security patches to it. The period between a patch being made available and installing the patch has a heightened risk of a compromise, since hackers have been tipped off about a vulnerability. If your only concern is security, you should therefore install all security patches, and install them as soon as possible.

Yet I know of professional (in the sense that they are paid to do the job) system administrators who do not install security patches because, they say, they are concerned that installing the patches will "break" their system in some way.

It easy to decry them as foolish. But a more nuanced analysis notes that their job is not simply to keep a computer system secure. The system has a business task to do, and a security breach is only one of several failures they must worry about. A rational approach takes into account the cost of each failure mode, the cost of protections against the failure, and its likelihood of occurring. Not installing patches can be a rational decision, at least theoretically, in some circumstances. More reasonably, delaying installation of a patch while waiting to see if it has problems could be rational in more circumstances.

However, for such decisions to be rational, the probability of a security patch breaking a business application must be known, and moderately high. Otherwise the given reason is more an excuse for laziness.

Just how likely is a security patch for an operating system component or framework (such as a web server) to break a business application running on that platform. Are there any statistics at all on how likely a patch is to break something?

Now, nobody really does a mathematical calculation of expected-gain, but rather operates on some intuitions about relative risk. I suspect the system administrators have a flawed intuition about the likelihood of a patch breaking their system. As a programmer of business applications, I find it hard to believe a patch to an operating system component or framework that had been reasonably tested by the vendor could break the application, unless the application was badly written and riddled with errors that posed other business risks anyway. But how can we correct such faulty intuitions without some kind of statistics on faulty patches? Such as the fraction of patches recalled or corrected?

A more rational approach taken in same places, is not to avoid updates altogether, but rather to delay and test before applying. Big organizations like Microsoft, Apple, and Mozilla have put out bad updates in the past rendering devices or software unusable, or unusable behind a proxy/firewall, etc. It can be a good calculated risk to delay an update for a while to see if there are any problems that pop up soon after it is released, and then to apply the update to a few non-critical systems first to check for problems in the local environment, before a wider roll-out.


Is it possible to use obtained wildcard certificate on more than one server? We are using two Debian servers in our company (with apache 2 as httpd) and our services are split between those two of them. One server has the website, the other has some minor sites under subdomains.

The idea was two use the obtained wild card certificate on both server to secure all our websites. I this possible, even though I only used one server to create the private/public keys, from which the public key was submitted to rapidssl to get the necessary certificates?

So can I just copy the private/public key to other server and install the rapidssl certificate there without problem as well?

Yes, this is possible and it is often used in practice.

The certificate is only bound to the private key and not to a machine. And the private key is not bound to machine too so you could take it to more machines. But you should be aware that the wider you propagate the private key the harder it will be to protect it. And without proper protection it might not be private for long.


Using IPsec through NAT I have read that it is recommended to encapsulate IPsec packets into UDP (port 4500) packets in order to circumvent NAT. Could anyone please provide a detailed explanation of the reasons behind this solution?

The problem is IPsec tunnel mode, which uses the ESP protocol.
ESP doesn't work with NAT for two reasons:

ESP creates a checksum covering the whole packet, including the addresses. If the NAT changes the addresses, the integrity check will fail and the packet will be discarded.
ESP also doesn't use ports. Therefore, NAT will not be able to add them to its translation database. (This doesn't prevent communication, but it does complicate it)

The solution is encapsulate the ESP traffic in UDP.
The NAT can change the UDP addresses because the checksum of UDP is not relied on. The use of udp/4500 also means that the ESP traffic will be limited to a single port, allowing the NAT to add it to the translation dictionary.

For more details, check RFC 3947 and RFC 3948


Adding conditions after LIMIT in MySQL Is there any way to add any condition after LIMIT for example the OR or AND?

I am wondering if we can add any query after LIMIT which returns nothing.

for example the query is:

SELECT * FROM table WHERE email = '$email' AND pwd = MD5('$pwd');

and after sql injection the query would be:

SELECT * FROM table WHERE email = '$email' AND pwd = MD5('') or 1 limit 1;

This question has been answered in the comment Section, But this is a explanatory Answer for anyone who wants to look at. So, Looking at Code snippet we could actually see

$login = "SELECT fName,lName FROM great WHERE email= '$email' AND pwd = MD5('$pwd')";

As you can see on the pwd = MD5('$pwd') at the end, the Code actually takes a User Inputted Password converts it into MD5 and than compares into the Database. Now, the Plan is to breakout of the pwd in the MYSQL Query.
For that, the ') could be supplied so it would look like pwd = MD5('')') in this case the last ') is of no Use and it would give SQL Syntax Error as the Query won't identify two ')')  So our next Plan is to comment the last ') out of the Query, for that i had used the Typical Two Hyphens -- - . Now the Magic here is if you supply only -- than the Query at end won't get commented out because it would result into --') Giving you a Syntax Error ! So for that reason, i supplied Intentional space after -- and added another hyphen for demonstration. Because in Many Blackbox SQL Injection cases, you couldn't predict and Exact Syntax error is the only hope, so in this -- - comment the last Hyphen has really no significance but the SPACE between them holds the key to comment ') out from the back .

Now, The query would work as usual and Due to lack of Input Sanitisation we can Always TRUE the query by or '1'='1' . But, that alone won't solve issue as the or '1'='1' would select Every ROW from the database and the code snippet given below does a Additional Check here

            if($exe){
            $num = mysqli_num_rows($exe);
            if($num == 1){
                  //You are in
                }

Aha, Now mysqli_num_rows > 1 Which would not let you go to the if part and You would not bypass. So in the Initial Payload, that is the sole reason we are Adding LIMIT 1 to avoiding our rows > 1 .The LIMIT 1 would select first row of their Back-End Database and VOILA you are through.

So Joining all these small pieces, we could construct our Final Payload as ') or '1'='1' LIMIT 1-- -  Hence Final Query would be

 $login = "SELECT fName,lName FROM great WHERE email= '$email' AND pwd = MD5('') or '1'='1' LIMIT 1-- -')";

That's it ..! :)


Best practice for encrypting data on Mifare I am currently playing around with Mifare Classic 1k. And everything I have read about it's security is true. So it's not exactly the best thing to use to avoid cloning. But I was wondering if someone manages to hack one of these cards following online tutorial could we stop them from knowing what's on the card itself?
Maybe using some sort of Crytography tool to "hide" the data that's written to the block? For instance, If I want the card to expire on the 10/25/2020 11:15:20 I could convert that value to a decimal i.e 1603710920 epoch time and then convert it to a hex value 5F96AFC8. And then take it a step forward and have the first row in the block be encrypted using maybe AES? so even if the card is cloned the clone would still expire unless the hacker could figure out how to change the expiration date.

If you are reading and writing raw data, you may assign whatever meaning you want.
In fact, an epoch of 1603710920 would probably be stored as bytes "5F 96 AF C8" (or perhaps C8 AF 96 5F, if using a different endianness). However, you could equally store the expiration as the number of seconds since birth of your first child, or as days since start of validity. You get to choose the meaning.
Using odd values would probably difficult understanding what is there, but probably not solve the original problem. Let's say you encrypt 5F96AFC8 resulting to 55F1A991, someone fiddles with the contents of an expired card and, trying to enlarge it blindly decides to write 55FF0000. You read the contents and after decryption that decodes to 11/04/2035. So it returns "not expired"
What you probably want instead is authentication, the assurance that the contents of the card have not been tampered with. A really simple way to ensure that would be to include a hmac of the card contents with a secret value. A more advanced solution could instead use an elliptic curve signature over the contents. The point is that you verify that the signature is valid before trusting any data in the card.
This could be done in addition to encryption or not use encryption at all (perhaps the contents of the card aren't secret at all, e.g. "Card #1234 belonging to John Smith, expires Oct 26 2020 10:15 UTC").
In any case, you should note that none of those approaches prevent cloning. If you have a card storing "credit", a user could copy its contents, spend some money, then write back the old contents. They don't need to know which field stores the amount or if it is encrypted. They just know that it reverts back to a card with X credit.
Similarly, John Smith could clone the card of Paula Bean which won't expire for another six months. Being restricted to cloning a full card means that it would be linked to Paula (in case someone would check the name, perhaps John doesn't look like her ☺), but it doesn't stop the potential fraudulent use of a clone.
In case, you should (in the case of card holding money, ideally they would verify the credit online -so John cannot change the amount stored in the card, and cloning Paula's card would still empty her credit-, or at least it should be reconciled so that a card with more credit than inserted into it after deducting the purchases would get automatically banned)


Restricted Access through Box and log commands I want to solve following problem. Please share your ideas or directly product name that capable to solve this.

Our system administrators need to connect customer's servers but I don't want to share SSH, VPN etc information with sys admins. Also I need to see log of commands that executed by sys admins.

to make the long story short, our system admin connect to the "box", then admin will be able to access customer servers. I deploy access credentials to the box, thus I don't need to share this sensitive info with system admin. One more thing, I want to see which commands executed by system admin for customer servers.

My plan is like following;

We have one linux machine. I will deploy customer credentials on it.
System admins will be able to connect this linux machine via VNC.
Then admin can connect customer with one click.
Record whole VNC session.

Thank you

1) What would be the purpose of deploying customer credentials? Is there any mechanism to reduce this portion. E.g., using ssh keys where client could revoke the key afterwards. If so, you've alleviated this issue.

2) VNC connection from your staff, is this local, remote. Have you created appropriate rules to enable trusted sources to connect while dropping others?

3) when you state "connect to customers with one click" I am under the impression you could create one independent program without having to use a central machine. You could write a simple shell script to record locally (on your colleagues' machines) and have it emailed to you.

4) vnc2flv

From the "just woke up" category using vnc2flv

# Example script maybe /usr/bin/watchdog.sh
# can be entered in user’s .bashrc or .bash_profile or other

self=`whoami`
dir=/tmp/`mktemp -d /tmp/recordingXXXXX`
file=`$self`-XXXXX

printf “Recording session for $self on `date`” > $dir/README
recordwin.sh -all /tmp/$file

As simple as you can get.

1) get ID of the user
2) make a temporary directory
3) specify a temporary file
4) enter that information into a "README" file in temp directory
5) Begin recording

There are plenty of mechanisms to do this, that is a generic baseline. You could have parameters set for your colleagues the moment they log in, and when they exit (e.g. before closing session, using ssh keys, have their recordings placed in a tar and scp'd over, along with an email sent to you stating they've exited and recording is complete.)


What is the difference of TrustManager PKIX and SunX509? Java's JSSE provides two different TrustManagers:

PKIX and
SunX509
I tried to find out what the differences between both are, but couldn't find any in the documentation. Can you tell me the difference between both TrustManagers?

From a basic usage point of view, difference is how resulting TrustManagers are initialised, as per Java Cryptography Architecture Oracle Providers Documentation for JDK 8

SunX509: A factory for X509ExtendedTrustManager instances that validate certificate chains according to the rules defined by the IETF PKIX working group in RFC 3280 or its successor. This TrustManagerFactory supports initialization using a Keystore object, but does not currently support initialization using the class javax.net.ssl.ManagerFactoryParameters.
PKIX: A factory for X509ExtendedTrustManager instances that validate certificate chains according to the rules defined by the IETF PKIX working group in RFC 3280 or its successor. This TrustManagerFactory currently supports initialization using a KeyStore object or javax.net.ssl.CertPathTrustManagerParameters.

One thing to note is, Java Cryptography Architecture Standard Algorithm Name Documentation for JDK 8, only lists PKIX as a TrustManagerFactory algorithm. SunX509 is left to provider's documentation because it is a vendor provided implementation, whereas PKIX is provided by all vendors. For example if you're running on IBM JRE, there is no SunX509, but IbmX509. Consecutively, if we hardcode "SunX509" in our application, we will receive a NoSuchAlgorithmException. Hence, for portability, it is best to use platform default algorithm as below, as both will work for keystone files (currently both Sun and IBM JREs default to PKIX).
TrustManagerFactory trustManagerFactory=
    TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());

While both factories can be initialised with a KeyStore parameter, using PKIX allows for alternatives, which can be configured using initialisation parameters. An interesting example is using LDAPCertStoreParameters for using an LDAP certificate store instead of a keystore file (an example here).


toy OpenPGP encryption with manually generated keys I've been playing around with OpenPGP encryption for a little while.  Here are the steps I've followed:

Generate an unencrypted public/private key pair in my keyring with gpg --gen-key.
Dump the public and private keys to files with gpg --export-secret-key -a > private.key and gpg --export -a > public.key.
Inspect the keys with pgpdump -i public.key and pgpdump -i private.key.
Use Mathematica to verify that p q == n, PowerMod[e, -1, LCM[(p - 1), (q - 1)]] == d, and PowerMod[p, -1, q] == u.
Encrypt a plaintext file plain.txt with gpg -e -r <my email> -a -o crypt.gpg plain.txt.
Decrypt the plaintext file with gpg -d crypt.gpg.

This all works fine.  However, I'd like to be able to try encrypting/decrypting with keys that I generate myself: is this possible?  Specifically, I'd like to know:

Is there a tool I can use to generate an RSA keypair from a given p, q, and e?
If not, and I need to implement RFC 4880 myself, what OpenPGP packets should I include?  Does GPG require that I include a UserID, Signature, and Subkey packets, or could I get away with just a Public Key or Secret Key packet in each file?

Related: does the line, "The checksum [...] MAY appear on the first line
after the base64 encoded data" mean that the base64 CRC is optional?

Can I get GPG to encrypt and decrypt using these keys without adding them to my keyring (as, being homebrew, they are likely to be unsecure)?
Finally, GPG will not let me generate keys shorter than 1024 bits.  If I try to encrypt/decrypt using a custom-generated 128-bit key, will GPG allow it?  What about with the 12-bit key p = 53, q = 61, e = 17 from Wikipedia's worked example?

Did you look at RSAKeyPairGenerator(doc)(source) from BouncyCastle? It generates an RSA keypair and encodes it. I bet you can create a keypair object, fill it with your values and call the encoding function to get a file gpg will accept.

EDIT:

So I've used openssl to generate an RSA keypair and export the values of its components:

openssl genrsa -out rsa1024.pem 1024
openssl rsa -noout -text -in rsa1024.pem

I've removed the colon byte separators and newlines from the big int values and padded the exponent to be an even number of nibbles in hex.

If you apply the following patch to this file:

--- bcpg-jdk15on-152/org/bouncycastle/openpgp/examples/RSAKeyPairGenerator.java 2015-03-01 12:03:48.000000000 +0200
+++ src/foo/RSAKeyPairGenerator.java    2015-05-19 23:07:00.754583435 +0300
@@ -1,11 +1,13 @@
-package org.bouncycastle.openpgp.examples;
+package foo;

 import java.io.FileOutputStream;
 import java.io.IOException;
 import java.io.OutputStream;
+import java.lang.reflect.Constructor;
+import java.math.BigInteger;
 import java.security.InvalidKeyException;
 import java.security.KeyPair;
-import java.security.KeyPairGenerator;
+//import java.security.KeyPairGenerator;
 import java.security.NoSuchProviderException;
 import java.security.Security;
 import java.security.SignatureException;
@@ -13,6 +15,10 @@

 import org.bouncycastle.bcpg.ArmoredOutputStream;
 import org.bouncycastle.bcpg.HashAlgorithmTags;
+import org.bouncycastle.crypto.params.RSAKeyParameters;
+import org.bouncycastle.crypto.params.RSAPrivateCrtKeyParameters;
+import org.bouncycastle.jcajce.provider.asymmetric.rsa.BCRSAPrivateCrtKey;
+import org.bouncycastle.jcajce.provider.asymmetric.rsa.BCRSAPublicKey;
 import org.bouncycastle.jce.provider.BouncyCastleProvider;
 import org.bouncycastle.openpgp.PGPEncryptedData;
 import org.bouncycastle.openpgp.PGPException;
@@ -76,11 +82,34 @@
     {
         Security.addProvider(new BouncyCastleProvider());

-        KeyPairGenerator    kpg = KeyPairGenerator.getInstance("RSA", "BC");
+        //KeyPairGenerator    kpg = KeyPairGenerator.getInstance("RSA", "BC");

-        kpg.initialize(1024);
+        //kpg.initialize(1024);

-        KeyPair                    kp = kpg.generateKeyPair();
+        //KeyPair                    kp = kpg.generateKeyPair();
+
+        BigInteger modulus = new BigInteger("00f4035469c2adf91d69052f94a8d466cae6ea485ecfd0dc6b1d408e8ef36ef2a60d907234c4e9e8e1261e42008a41140134fc62840338433ab5948b96ceb91b65b0aacf8b54deca21a90386ebe17df16ebd1bef137ba8a43e9e2671ecd0bc5cd052321d50e61eac532cf718309f76760b9e68690897d76a5f9955541f3bd87a89", 16);
+        BigInteger publicExponent = new BigInteger("010001", 16);
+        BigInteger privateExponent = new BigInteger("00abfc6cd2de54a0f109c48df4c9ca6b6937b889a9c9effc6bc3026d78743c0eeadb44a43a6d5030c40089f31b4e56f032a995fa19f1eb05f7ab6437bee395b4a867af07b71b8515728b44fed0f5c85367e182a00a8df3df3098b82e74f224b294c90e261869c81fb9b89824bec026b7189bad0c11d4366636e2d357ca06e591c1", 16);
+        BigInteger p = new BigInteger("00fdd4b067e9361abe40231470f8249e68d64aeade1828dcd8b60cb33942df55bb6806fa81784b5fc24f73d8313baac2c0150307e56df621c77cc650b2b6996193", 16);
+        BigInteger q = new BigInteger("00f619297aa83e72f5a0c9b0df87c1bb0927c17d70cf99c6a9f6c58ddbb848d40abdd0c853baf7de55ac8dfe84ba38d12e86703e4b82273066c252369a5f3534f3", 16);
+        BigInteger dP = new BigInteger("4dbd9e69b4db85454f8f6eeb4a94ac8f9f5242acd2e970fa4e87853cbc667a737360efc847778e548cd1061dce1076a52dca47d8d4dcd56baba37183cab91f51", 16);
+        BigInteger dQ = new BigInteger("64232af0a103002e1865d955ab5cd6294c86fbeeea5a6d2efd9db7325f932accd01de355c6af5345d337d807d3ea889b80d2ad56763852068e2d7bd066cb34a7", 16);
+        BigInteger qInv = new BigInteger("00c9cbb6242b3e748d557e12cbacaa4b3e5a7fd0573fcc31b77268a492e270055628a02a46ff4cefbdff28aaea289faca9826d8e22cf5b0f4de2e15bdd4b3d3d14", 16);
+
+        RSAKeyParameters pubkey_params = new RSAKeyParameters(false, modulus, publicExponent);
+        RSAPrivateCrtKeyParameters privkey_params = new RSAPrivateCrtKeyParameters(
+               modulus, publicExponent, privateExponent, p, q, dP, dQ, qInv);
+
+        Constructor<BCRSAPublicKey> pubkeyctor = BCRSAPublicKey.class.getDeclaredConstructor(RSAKeyParameters.class);
+        pubkeyctor.setAccessible(true);
+        BCRSAPublicKey pubkey = pubkeyctor.newInstance(pubkey_params);
+
+        Constructor<BCRSAPrivateCrtKey> privkeyctor = BCRSAPrivateCrtKey.class.getDeclaredConstructor(RSAPrivateCrtKeyParameters.class);
+        privkeyctor.setAccessible(true);
+        BCRSAPrivateCrtKey privkey = privkeyctor.newInstance(privkey_params);
+
+        KeyPair kp = new KeyPair(pubkey, privkey);

         if (args.length < 2)
         {

You'll get a program which when run with arguments "-a foo password" produces pub.asc and secret.asc files which are a pgp keypair.

EDIT 2:

$ pgpdump secret.asc
Old: Secret Key Packet(tag 5)(510 bytes)
    Ver 4 - new
    Public key creation time - Tue May 19 22:56:20 IDT 2015
    Pub alg - RSA Encrypt or Sign(pub 1)
    RSA n(1024 bits) - ...
    RSA e(17 bits) - ...
    Sym alg - CAST5(sym 3)
    Iterated and salted string-to-key(s2k 3):
        Hash alg - SHA1(hash 2)
        Salt - 3b b9 cf f9 02 cf 90 9e
        Count - 65536(coded count 96)
    IV - d8 ac aa 96 61 7a 03 34
    Encrypted RSA d
    Encrypted RSA p
    Encrypted RSA q
    Encrypted RSA u
    Encrypted SHA1 hash
Old: User ID Packet(tag 13)(2 bytes)
    User ID - id
Old: Signature Packet(tag 2)(156 bytes)
    Ver 4 - new
    Sig type - Generic certification of a User ID and Public Key packet(0x10).
    Pub alg - RSA Encrypt or Sign(pub 1)
    Hash alg - SHA1(hash 2)
    Hashed Sub: signature creation time(sub 2)(4 bytes)
        Time - Tue May 19 22:56:20 IDT 2015
    Sub: issuer key ID(sub 16)(8 bytes)
        Key ID - 0xB01AC717560C7F68
    Hash left 2 bytes - ee a9
    RSA m^d mod n(1024 bits) - ...
        -> PKCS-1


Why would a stranger keep signing up to trading sites with my email address? I have read the answer below and understand that there is not much I can do about the basic problem of somebody signing up for newsletters and online services using my email address:

Someone used my email address in order to access an online account

However, my question is more nuanced. Here's the story...

About one month ago, somebody started signing up for crypto/trading sites using my gmail address. They keep using the same name and assuming it is actually their name, the person appears to be a 27 year old male from Seaford, Delaware and has had a few minor convictions for possession of marijuana and minor traffic offences (if one can believe the free background checking sites). I am based in Sydney, Australia.

So far this person has signed up for:

Coinigy
Coinone
Robinhood
Stash
Marketminer
Tradecoins
Plus a few other newsletters and investment dashboards.

They have even transferred funds ($10 and $25) to a couple of the sites and made some small investments in ETFs.

Some of these sites require email address verification, so I don't click on the verification links for those sites, thereby limiting the problem. Somewhat surprisingly though, many of these sites do not require email address verification, so for these sites I have been logging in and changing the password.

The "attacker" (I use the term loosely, because I'm not 100% sure what is actually going on here) has not been able to access my gmail account, as it has a super-strong password (which I have recently changed, just for good measure), 2FA using Google Authenticator, plus I have been checking the access logs regularly and have seen no logins other than my own IP addresses. Nobody else has the password to the two computers I use to access my gmail account and I have scanned both of them with two different anti-virus products to ensure that I haven't been hacked. I have checked on https://haveibeenpwned.com and the address wasn't listed.

The email address itself a fairly obscure, non-obvious, non-dictionary word. It wasn't cool 10 years ago but since the rise of crypto trading it might now be considered cool by some. I do use the address as my contact email on my domain name registrations, so this could be where they found it.

I am trying to determine if this individual is:

Completely ignorant of what they are doing.
Deliberately using an email address that isn't theirs for some obscure reason that isn't nefarious.
Deliberately using an email address that isn't theirs because they are attempting some kind of illegal activity.

I realise this isn't a simple question with a single right answer, but if someone can provide a very compelling theory as to what might be going on and/or some strategy I should take to deal with the problem, I will happily accept that answer. I am mostly interested in the "why would they do this?" aspect though.

I have 4 different people who regularly use my email address to sign up for things. I get incorrectly addressed emails daily. After a few years of getting their emails, I know just about everything about them and most of their immediate family (I know one 10 year old's class schedule).

I try to get them to be more aware of the fact that they are not using the email account they think they are using, but they simply will not be careful enough to check. Sometimes things die down for a bit, but they always start up again.

My vote is that this other person is just not careful or deeply confused.


Is it safe/correct to grant full access to Everyone on a Windows symlink? Does it really matter what the permissions are on a symlink as long as the smymlink target is configured correctly?

I'm going to say no, with a symbolic link its only linking to a file on the system like a shortcut and all its designed to do is act as a bookmark for easy access from whatever location you put its shortcut at. In this case the only thing that COULD happen to your symbolic link is that if you don't put permissions other people on the system could potentially copy and or delete it but the original documents would be fine.

I did this once on Linux with a Minecraft server at one time and a rogue admin with FTP access thought he was going to "kill" the server by deleting it but unaware to him it was only a symbolic link he had killed and not the entire directory.


How to appropriately secure a database using different db users I was thinking recently of the steps to secure a database server. Most of us are aware how to handle the security on the application side, although a malicious user might find his way to bypass the application security and exploit the database.

Scenario

Consider having a database with the following table prefixes blog_ and website_ (for examples and simplicity sake, K.I.S.S.)

I am thinking of something along the lines of the following, but am unsure which is appropriate and which is an overkill, hence the purpose of the question.

Solution 1: One writes, many read

Have only 1 user in the application with write access on the database. On all tables
Have more than one user with read access on to the database, but separated on a per table set(for a specific prefix) basis. Meaning that each RO db user respectively read access only to their set of tables.

  global_write@localhost: writes to all tables in the database

  blog_read@localhost: Reads from tables with prefix blog_

  website_read@localhost: Reads from tables with prefix website_

Solution 2: One read and one write on a per table basis

Have a pair of db users per application component, one to read and one to write

  blog_read@localhost: Reads from tables with prefix blog_

  blog_write@localhost: Writes to tables with prefix blog_

  website_read@localhost: Reads from tables with prefix website_

  website_write@localhost: Writes to tables with prefix website_

Solution 3: Only one pair of accounts is sufficient

Only one pair of accounts is responsible for the entire database

  global_read@localhost: Reads from all tables

  global_write@localhost: Writes to all tables

TL;DR

Solution 1: One writes, many read

Solution 2: One read and one write on a per table basis

Solution 3: Only one pair of accounts is sufficient

Are all of these appropriate?
Which ones are an overhead?

PS: Sorry if I did not tag appropriately with access-control

Separating by Tables

Separating by tables seems like a good idea, especially if you can estimate the importance of tables to your security. For example a table with usernames and passwords is pretty important, while a table with blog comments is less important. If you have a different database user for those important tables, an injection into eg a query that selects comments might not gain an attacker any real advantage as they cannot access relevant data (the most they could do is reflected XSS).

Separating the blog and website also looks reasonable (at least from a system design point of view), but I would give each of them a separate database (they don't really seem to belong to each other, so it makes logical sense, and would also be easier to handle).

Separating by Read/Write

I'm not sure that separating by write and read would be worth it (what would be the actual thread it defends against? I think mainly an attacker injecting a blog post), and it will probably be hard to enforce. For example if you have an admin user which can edit the database via a web interface:

attacker can read: attacker can read out admin password, and thus write (via web interface)
attacker can write: attacker can add admin password, and thus read (via web interface)

Managing the privileges by table prefix also does not seem like a good idea. Consider this (this might be a bit far-fetched, but it's just one example of how to bypass your rules):

website_read has select rights on website_*
global_write has write rights on website_* and blog_*
blog_read has select rights on blog_*
attacker has access to global_write and blog_read (eg via injection) and wants to read out website_users (to which they don't have the rights)
attacker renames table website_users to blog_users with global_write [*], and reads it out with blog_read.

[*] If this is actually allowed would depend on your concrete system, but generally rename is a write activity. Eg MySQL requires ALTER, DROP, CREATE, and INSERT which I would call write rights.


Obscure compression before encrypting Assuming whatever encryption algorithm used was designed to support compression without any information leakage, would there be any reason not to use some custom compression algorithm to add obscurity to security?
Instead of a compression algorithm, what if it were just a simple custom algorithm that mixed the bits or bytes of the input? Would that impact the security at all?
(This is assuming that the implementation of said algorithm is secure against side-channel attacks.)

Obscurity does not give you additional security, hiding a key to the door does not make your lock stronger especially if you're dealing with someone trying to pick it.
Compression prior to the encryption is only to make the input smaller which has a significant impact on the performance of the asymmetric crypto it is also handy when you need to transfer the files on the wire as they're smaller. If you mix your input or output then you need to provide the user with the way of undoing the "mixing" which could be obtained by the adversary rather easily. If you consider making this operation secret, then the question would be, why not use a one time pad instead?


How to generate an unpredictable but short string? I want to built some short urls for uploaded files and links on my own server. Simple enough, I upload files and my script returns me a link. On access, the link is translated to the file and download starts. Because it's modern, I want my links short and not so UUID4-like, because they are very long.

I do not want that someone tries a few URLs to found a valid link. Maybe I should sent only 404's if an IP already had 10 wrong urls called?

To built short random strings I thinking about the following:

Prefill a SQL-Table (key, target) where key is case sensitive with all possible values in a defined range like [a-zA-Z0-9_-~=]{12}. Then select with WHERE target IS NULL ORDER BY RANDOM() an empty key and use it. This will guarantee I never generate the same key twice but needs a lot of storage and may be too much.
I generate an UUID4-string and throw an hash-algorithm like SHA3 over it and use the first 12 characters of the result. Then I try to insert this in my table. If this fails, I need to generate an new UUID4 and so on...

Of course, these are very theoretical thoughts, because I think I will be fine with 1000 keys per year. But how shortening url services do this? Or they don't care about "unpredictableness"?

If you want unpredictable then you need unpredictable. If you start with an UUID and then play games with hash functions, you don't get unpredictable values; you just get values that "look randomish".

So just use a cryptographically secure PRNG. This means /dev/urandom (or anything similar, e.g. java.security.SecureRandom if you happen to be developing in Java).

This simple command (works on any Linux system) generates random, unpredictable 12-character values:

dd if=/dev/urandom bs=9 count=1 2>/dev/null | base64 | tr '+/' '_-'

The used characters consist in letters (uppercase and lowercase), digits, '_' and '-'. Thus, they should fit nicely in URL (raw Base64 produces '+' and '/' signs, which can be problematic, but I fix that with the tr call). Entropy is 72 bits, meaning that the strings are part of a space of size 272. If you generate 1000 strings, then probability of hitting one of the strings is 1000/272, i.e. about 1 in 4.7 billions of billions (so the attacker had better be super-lucky).

Translation to any particular programming framework is left as an exercise. The really important point is to use a strong PRNG, not things which are merely meant to be "statistically random" like SQL's RANDOM() or UUID. This is security: we want cryptographic randomness, not statistical randomness, because we need to defeat an intelligent attacker.