What can I do to increase the security of my app based on a shared secret?

This is a problem that has been solved by OAuth 2 and Open ID Connect (OIDC). I would strongly suggest building on it rather than inventing your own protocol. Take a look at the OAuth 2 device workflow (Google has an excellent description).

First off, it helps to split authentication from authorization. Authorization is what you want to achieve (i.e. authorizing the TV). Whether the authentication workflow requires login or not, or involves multiple factors is something that should be considered separate from the authorization workflow and may be something you want to be configurable.

Diagram below outlines the OAuth 2 device workflow.

TV app                                    Server
   |                                        |
   |----------- request code -------------->|
   |<------- url & auth code (QR) ----------|
   |                                        |
   |   code             mobile     user     |
   |-- & url --> user --> app --> login? -->|
   |                                        |
   |---------------- poll ----------------->|
   |<-------------- token ------------------|
   |------- use token to register --------->|

The auth code can be in QR form that the user scans with the mobile app, or it can simply be a code that the user will have to manually type into their mobile app.

In the aforementioned Google link, what they describe as device ID may be the UUID you mentioned, but it could also be a securely generated random string associated with your app.

There are guidelines in OAuth 2 and OIDC for utilizing shared secrets, which are only necessary for protecting credentials. In the workflow above since the mobile app is responsible for authentication, it has the option of using a shared secret with the Server (also known as a confidential client) or not (also known as a public client).

The question of whether you need authentication depends on where your Server resides. If it is on the internet, authentication is recommended - many OAuth 2 server libraries/applications let you integrate with various social media accounts so this may not be as painful as you'd think. On the other hand if the Server resides in the user's local home network then perhaps authentication is not necessary.

There plenty of client and server OAuth 2/OIDC libraries that will make your life easier. For the server side you can even make use of server applications (e.g. Keycloak) sporting features such as TOTP to improve security.


How to decrypt email with revoked key in enigmail?

Yes, after you revoke your key you can still decrypt messages sent to you encrypted with that key.

Note that you are also able to decrypt messages sent to you with the revoked key after the revocation.  In fact, even if people aren't supposed to use that key (GPG/Enigmail will not allow to use a revoked public key to encrypt a message to someone), many of them could still have your public key when it was still valid and haven't refreshed it since then.

You should send to these people your new public key and your revoked old public key so they can update their keyring; afterwards, they'll be able to use only your new key. Alternatively, if you've uploaded your keys to a keyserver, tell them to refresh their keyring from Enigmail Key Management: they have to select your public key and choose Keyserver -> Refresh selected public keys (or Refresh all public keys to refresh all keys in the keyring).

Enigmail should show you revoked keys as well. If it doesn't, open Enigmail Key Management, go to View and make sure Display invalid keys is ticked.


PHP shellcode inside /tmp. Is the server compromised?

Your assessment sounds correct.  It's impossible, of course, to say that your server has not been compromised, but by default, PHP receives all the data before executing the script, so it writes file uploads to /tmp and provides that filename to the running script.

If your /tmp is mounted with atime enabled, the fact that atime==mtime is reassuring.  If the attacker found a way to execute those scripts after their upload (LFI vulnerability, for example), that would change the atime, so the fact that it wasn't updated indicated they weren't executed (again, if atime support is enabled on /tmp).


Alice has the private key and the boss the public. The boss can use the public key to read things Alice makes, but Alice is the only one that can encrypt her stuff with the private key. Even if the boss edits a file he will not be able to save it (re-encrypt it) because he does not hold the private key. Alice can read and write because she holds both the encryption and decryption key.


why does ssh prompt with the ECDSA key instead of RSA

ssh-keygen -l seems to default to RSA and it knows the default path.

ssh-keygen -l defaults to use a RSA key within the users home directory, i.e. ~/.ssh/id_rsa. Keys in the home directory are used to authenticate the client to the server. What you want to verify instead is the host key which is used to authenticate the server against the client. To get this fingerprint you need to specify the key with the -f argument no matter if RSA or ECDSA is used since it is not located in the users home directory but in a system directory, typically /etc/ssh.

  If ssh would prompt with an RSA key then ssh-keygen can be used with the shorter command. Unfortunately, ssh prompts with the ECDSA key.

SSH prompts with whatever key the server is presenting to the client, i.e. it fully depends on the server setup and not on some SSH builtin defaults.


How should I store and secure self-signed certificates?

This answer applies to SSL or PGP type digital certificates, which bind an identity to a public key.  It has been pointed out to me that the question as originally asked did not specify what kind of certificate, so my answer may not fit the question.

Digital certificates which bind an identity to a public key do not need special security because they contain only the public key.  There is no reason not to store a copy in your source code control system.

The corresponding private key does need to be kept secure, but will need to be installed in the server(s) or email client(s) which are identified in the certificate.  So, the private key is necessarily exposed to anyone who has administrative access to those machines.  Also to anyone who has access to backup tapes, etc.

You absolutely do need a backup of the private key; if it is lost, as through a disk failure, you'll need to replace the certificate with the corresponding public key.  I keep mine on an encrypted volume for which a very few people have the key.  (More than one person needs to know that decryption key.  People really do get into accidents, quit their jobs, etc.)


Deletion of trusted root certificate store folders in a product

Yes, you can.
If your appliance is not designed to communicate with any external hosts, then creating your own certificate store, which only includes CA's you manage, is a reasonable thing to do.
It doesn't really add that much security, given that breaches of any of those CA's are very unlikely, but it doesn't have any negative effects either.

While it is not 100% directly your question, there is a design pattern many developers fall into that can lead to vulnerabilities.
Many developers use client certificates signed by a CA they control in order to provide a mechanism for authentication. However, they only check if the certificate is valid - not if it is actually signed by their own CA. This means an attacker can generate a valid client certificate through a public, trusted CA and use that to authenticate with the service.
While deleting all external CA's would solve this problem too, the inclusion of a check whether or not the presented client certificate is signed by the correct CA is a much better solution.


How is cryptoshredding a "sole pragmatic option for data disposal in the cloud"?

Without the additional context of the rest of your training materials, it's hard to say for sure -- I don't think there's a textbook definition of "cryptoshredding".  However, if we use the Wikipedia definition, we see:

  Crypto-shredding is the practice of 'deleting' data by deliberately deleting or overwriting the encryption keys.[1] This requires that the data have been encrypted.

Consequently, yes, this only works for data that is encrypted.  Cryptoshredding would not work for unencrypted data as there's no key to destroy.

  Is the assumption that the data would have always been encrypted at rest from the very beginning?

Yes.  This is a best practice for any sensitive data in a cloud service anyway.  Your cloud provider can replace hard drives, have machines go down, maintain their own backup lifecycle, etc., all outside of your control.

  If not, how would cryptoshredding help to remove remnants of data that was previously stored unencrypted?

Data is not "previously stored" in my mind.  It is stored, and you might make an encrypted copy of the data, but then you need to destroy the original -- and don't have cryptoshredding as an option, which leaves you with 0 options for cloud storage effective data destruction.  This is back to the idea of storing it encrypted from the get go.


FreeNet is a distributed p2p network that relies upon a Distributed Hash Table to store and distribute data. One of the hallmarks of this network is that once a page is pushed to FreeNet,  it can't removed (as long as people are viewing the content).

However,  it sounds like a Tor mash-up is what your looking for.


Does a password-derived public key authentication improve security over pure password-based authentication?

That would have been too easy, wouldn't it? While obtaining the private key from the public key itself is still nigh impossible, the method to obtain it from the too-simple password is trivial, thus all this method does is replacing the purely server-side hashing by a hash-like algorithm that puts more of the workload to client-side. The only potential benefit might thus be slowing down online attacks, but existing methods such as limit-rating are probably still better-suited and more notably already well-established...


You can spot processes that:

have a similar name with an officially named process - example: svch0st instead of svchost, expiorer explorer
have randomly generated names - examples : fh74w7ha, xf8f4a34, m599j42k.
have insufficient/inaccurate general description - example: file

You can detect P.I. malware by studying:

Parent-Child process relationship (example: processes named accordingly but not started by winlogon.exe or wininit.exe but by another process)
Usage of suspicious memory protection (PAGE_EXECUTE_READWRITE - sometimes at a round address like 0x1000000).
Comparison between PEB and the VAD structure (process memory vs kernel memory)


Will blocking outgoing traffic from port 80 and only allowing port 443 from my home network lead to more security?

Yes, it will add security, but not as much as one could expect.

Many web sites have HTTP port open, but redirect your requests to HTTPS port. -> Blocking HTTP will not give more security. Your first request sent via HTTP will be visible to possible interceptors. After web site redirected you to HTTPS port, all further requests will be not visible. So you more security during the 1st request only.

Some sites support HTTPS only. -> Blocking HTTP will give more security when accessing sch sites.

Some sites support both HTTP and HTTPS. -> In such cases you will get more security because the traffic will be protected by TLS/SSL.

Some sites still support HTTP only. -> Blocking HTTP means such sites will be not accessible from your home network. But yes, information that is potentially available to any interceptor in case of HTTP will be no more available (because you just don't access such sites).

There are also sites that combine HTTP and HTTPS. They provide some resources like images and fonts via HTTP and other resources via HTTPS.

Only you can decide if blocking HTTP is reasonable to you.


Is there a difference between "Maximum Tolerable Downtime" and "Maximum Allowed Downtime"?

They are not the same thing. MTD is the time allowed before the entire business becomes at risk (e.g. the company may go out of business), whereas MAD is the time at which an actual business impact occurs (which may or may not be crippling - it could just be painful). MTD is "how long before it puts us of business," MAD is "this did hurt us directly or indirectly an undetermined amount."


Is my Windows 10 machine experiencing DNS poisoning? I keep getting Chinese IP addresses when connecting to a U.S. government domain

Content Delivery Network

This is probably part of a Content Delivery Network with a lot of political issues to consider.

If you try dig www.whitehouse.gov a, underneath the answer section you'll see the following:

www.whitehouse.gov. 131 IN  CNAME   wildcard.whitehouse.gov.edgekey.net.
wildcard.whitehouse.gov.edgekey.net. 731 IN CNAME e4036.dscb.akamaiedge.net.
e4036.dscb.akamaiedge.net. 20   IN  A   23.73.28.110

See the CNAME addresses? Try host -t A www.whitehouse.gov for a better explanation:

www.whitehouse.gov is an alias for wildcard.whitehouse.gov.edgekey.net.
wildcard.whitehouse.gov.edgekey.net is an alias for e4036.dscb.akamaiedge.net.
e4036.dscb.akamaiedge.net has address 23.73.28.110

Do you notice that I'm getting a different IP address than you? Note the wildcard*edge* portion? What is that? It's an edge server which is supposed to be closest to you.

Are you using a VPN on your Linux machine, or Windows Machine? Maybe one that's in Hong Kong, Hangzhou, or somewhere else in East Asia? Maybe your router is configured to use a VPN, or go through TOR?

The IP address you received belongs to Aliyun Computing Co. Ltd, which is part of the Alibaba Cloud/CDN suite.

But wait, how did we we get an Aliyun Cloud/CDN (Alibaba) IP address from Akamai? Aren't they competitors?

Again, are you using a VPN on your Linux machine, or Windows Machine? Maybe one that's in Hong Kong, Hangzhou, or somewhere else in East Asia? Maybe your router is configured to use a VPN, or go through TOR?

Akamai does operate in China, but...

Want to make money in China? You have to follow Beijing's rules. I think we just found something embarrassing for Akamai: in order to operate in China, they were likely forced into a partnership with them.

To do business in China, almost all foreign companies were previously required to hand over control of their intellectual property to a joint Chinese partner to be allowed to operate in the country.

Let's look at the IP you gave us: whois 139.129.57.70 | grep -i 'Ali\|Hangzhou':

netname:        ALISOFT
descr:          Aliyun Computing Co., LTD
descr:          No.391 Wen'er Road, Hangzhou, Zhejiang, China, 310099
address:        NO.969 West Wen Yi Road, Yu Hang District, Hangzhou
e-mail:         jiali.jl@alibaba-inc.com
address:        No.391 Wen'er Road, Hangzhou City
e-mail:         anti-spam@list.alibaba-inc.com
e-mail:         cloud-cc-sqcloud@list.alibaba-inc.com
address:        Hangzhou, Zhejiang, China
address:        No.391 Wen'er Road, Hangzhou City
e-mail:         guowei.pangw@alibaba-inc.com

In this case, the Akamai partner is likely Alibaba/Aliyun. This allows the Chinese government, if they so desire, to serve malicious content to visitors by way of the CDN.

Every single CDN is, in my opinion, MITM as a service.

Wireshark? You might be doing it wrong.

What if you did have a DNS hijacking/MITM issue of some sort? If you want to use Wireshark, you probably cannot do a packet capture between your router and computer unless the problem exists primarily on your Windows 10 machine. You'd simply be receiving whatever your router provides you with.

If there is no problem with your Windows 10 machine, then using Wireshark on your computer will likely not provide you any meaningful information. What if your router has been compromised?

What you could do is put a switch with port mirroring capabilities between your gateway and your router, and use the port mirror to see what's going on. Or a LAN throwing star. This way, you can see what your router is sending and receiving, and compare that to what Wireshark sees.


Is storing encrypted and hashed versions of the same data less secure?

So, you want to be able to quickly test if some data is stored in the encrypted database, without having to decrypt the database.

Now, if you can search this database without decrypting (without the key) then so can an attacker search the database without the key. So, yes in general this is less secure.


Distinct error messages for wrong email / password

Essentially, this comes down to a balance between usability and security.

In the case of Google and Facebook, there is an assumption that pretty much "everyone" has an account, therefore the risk of giving away what email address has been used is minimal - it makes more sense from a customer support purpose to minimise calls from people who have mistyped their email address. They also both use fairly advanced methods of detecting fraudulent logins, with the ability to show a list of all known sessions, and detection for multiple logins from geographically distinct locations (e.g. if you log in from the UK and the US within an hour, something suspicious is going on).

For most smaller sites though, there isn't that assumption. Most people don't have an account with any given website, therefore the information that they do becomes valuable. In that case, you can reasonably hide whether someone is a member or not by giving a generic rejection method. You can also give a standard message reading something like "We have sent a password reset link to the address provided - if you don't receive it within a few minutes, check your spam folder, or double check the entered address" - you don't need to expose your user list through forgotten password screens.

If the smaller sites could either make the assumption that "everyone" has an account, or run a full suite of fraud detection methods, like Facebook or Google, it wouldn't be an issue, but the cost of running that type of protection tends to be prohibitive until you reach huge scale. Generic error messages, on the other hand, are cheap, and effective.


Try the hardware based approach, such as this one. Those devices are not that expensive but solve the issue of reusing old HDDs and SSDs fast.


How to know if there is an audio hidden message in an audio file? Can we hear it?

An MP3 file consists of many frames. A frame is a piece of data of about 1000 bytes long, with a little sound data in it. Each frame consists of a header and some data. The header contains the bitrate for the frame, whether the sound is in stereo, and so on. A player plays a file by looking for these headers, decoding the frames and playing the sound.

You can put any data that does not look like a frame header in between the frames and the file will still play correctly. The player simply ignores your data and searches for the next frame when playing. By hiding data in between the frames, the sound is not modified at all.

Another way is to use some bits in the header to hide the message. The "private" bit, for example, is unused by most applications and can be used to store a bit of data. This would go unnoticed when playing the file.

If you want to hide data in the sound data itself, you have to change the sound data a bit to contain your message. Whether this is audible depends on how much the original data is changed, and whether it is done with knowledge of the MP3 specification.


How do Common Vulnerabilities and Exposures (CVEs) work?

As the name suggests it, CVE are divided to two categories: Vulnerabilities and Exposures both are mainly errors in a given software code or its configuration that respectively may allow an attacker to -for example- gain a privilege escalation (running a code as root on a user level environment) or to gain an indirect access to a system/network.

Before trying (at least) to adresse your points, its important to keep in mind that the very first stage of creating a CVE is discovering and disclosing a vulnerability. And for the sake of simplicity :

How do CVEs work?: Once a vulnerability is discovered and disclosed, CVE Numbering Authorities (CNA) assigns this vulnerability a CVE ID then post the CVE on the CVE List
How do you find a CVE related to a particular product?: There are many repositories around, like the one suggested by @bonsaiviking, you can use https://cve.mitre.org/, https://www.cvedetails.com/ or https://nvd.nist.gov/...
Where can I find a hotfix for a CVE?: Generally hot fixes are updates pushed by affected software vendors, this why its important to keep its Os, Softwares, Web browsers updated.
Do all listed CVEs have hotfixes? To make it simple, the answer is NO, researchers and bug hunters usually respect what is called  Vulnerability Disclosure Policy, to make it simple, a typical disclose starts by contacting the affected vendor and making it aware of the found vulnerability, by providing it a Proof of Concept for example and technical details, the vendor should correct the concerned vulnerability within 90 days  passing this deadline, the vulnerability could be disclosed, IF t this happens, then the "bug" is assigned a CVE.
What do I do if I can't find a hotfix for a listed CVE? In general, nothing except waiting an update from the vendor or avoiding the use of the affected software.

The answer was not exhaustive but hope it helped !


Why install a certificate in the Trusted Root CA of the server?

You are correct, the Root CA is required when clients connect to the service with client authentication certificates.
Your server may be a client of another service.  In that case, it also needs the Root CA in its own trust-anchor store (Trusted Root CA as you called it).  Without this, the server couldn't authenticate the signatures on any certificates presented to it when it connects via TLS to the other service.


How to build a secure stateless authentication system for a client-side JavaScript application

Any mechanism you come up with, that does not involve the user entering a secret for     every request will be vulnerable to XSS. The solution is to try and eliminate XSS from your domain, not to have a solution that is secure in the presence of XSS vulnerabilities. A strong CSP policy will help you in browsers that support it.

Javascript can access cookies, as long as you don't mark them HttpOnly, but you don't need it to as the cookies will be sent automatically with every request. So cookies are going to be more secure then LocalStorage as long as you mark them Secure and HttpOnly.

So I would go for Scenario 2, using cookies. If it needs to be stateless the cookie should be a signed and HMAC'ed token with a time expiry in addition to the cookie timeout, such that the servers stop accepting these tokens after the timeout. It will be hard to securely implement the logout functionality without storing state on the server, you can instruct the client to throw away the token, but without a blacklist or whitelist maintained on the server-side you will be vulnerable to the token being used after the user has logged out.

Response to Comments

  CSP, is it something I need to implement on the load balancer in front of A? A is just static html and javascript files.

CSP will need to be served on the top level page (The url the user sees in the address bar), this is A in your case. You should be able to configure you static web server to add headers to every response. Doco. You could also do it at the load balancer but that seems more complex then is required.

  If I add CSP on requests going between A and B, ...

CSP is added between A and the browser, not between A and B. In the CSP policy you can configure the browser to be able to communicate with B.

  somebody could easily modify the javascript to send different headers.

They could only do this if your html/javascript is being sent over HTTP, not over a secure link e.g. HTTPS. If your top level URL is HTTP all scenarios will be susceptible to man-in-the-middle attacks. The communication between A and the browser and B and the browser need to be over HTTPS to have any security.

  A and B are running on different domains, B adds a CORS filter to only allow JSON POST requests from a specific domain for A, should the CSP also be added  alongside the CORS headers?

You will need both CORS headers, sent from B to allow communication from A. And CSP headers sent from A to restrict javascript execution as a mitigation strategy for any XSS vulnerabilities in your site.

  Since the authentication token is generated, validated and invalidated in the Authentication Service, would HMAC really be needed

In most situations a HMAC makes tokens more secure. If you need to remain stateless you will need a HMAC. If you Authentication Service is state-full, that is it has a list of valid, current tokens, then HMACs are less important as long as your token id's are not guessable. If this is the case consider GUID's for your tokens.

  The cookie should obviously be created at A as that is what the browser is interacting with.

My understanding is the browser is also interacting with B, via AJAX, hence the need for the CORS policy. As it is B that is managing the login state and requiring the authentication token the cookie should be set by B. Also as A is just serving static content it would be more elegant if A did not require to set cookies, a dynamic process.

  Seeing as it's completely HTML and JavaScript, I'm guessing it's impossible to access data inside the cookie if HttpOnly and Secure is turned on?

Yes with HttpOnly turned on javascript on the browser will be unable to access the cookie. In your case it would be best if the cookie was set by B, with HttpOnly and Secure set, this should happen on a successful login call. Then all subsequent calls to B will automatically have the cookie added to them, and the javascript will not be able to access the cookies.

Timeline - As I would see it.

User navigates to https://a/index.html
A responds with a HTML page
including CSP headers, the HTML includes a <script
src="https://a/code.js"> tag.
The browser fetches https://a/code.js
code.js puts up a login box, user enters username and password and instructs the browser to send an AJAX POST request to https://b/login
The browser sends a HEAD request to https://b/login to ensure it can make posts.
B responds with the CORS headers allowing POSTs from https://a
The browser sends a POST request to https://b/login containing the username and password
B checks the password and if successful returns a success message and sets a cookie which is HttpOnly and Secure. This cookie contains a token such as a random GUID. This token is stored in B's state.
The user clicks a button
A request is sent to https://b/customer/5/update containing the cookie
B validates the cookie value and proceeds to do the action requested e.g. updates the customer.
The user click logout
The browser sends an AJAX request to https://b/logout
B invalidates the token contained in the cookie in it's state, and send a message back to the client clearing the cookie there also.


You're missing the same-origin policy. In order for your scenario to work, the attackers needs JavaScript code from his page to be able to access the iFrame. That's not possible.

Remember this rule: A window/frame can not talk to another window/frame without the target window/frame allowing it (by the use of messaging, for example).

In your case, the target frame won't allow it. So the attacker won't be able be able to access the iFrame's DOM to grab the token. He can't programmatically submit forms or send requests through the iFrame either.


Do 64-bit applications have any security merits over 32-bit applications?

ASLR is a hide-and-seek game: in case the attacker succeeds in overflowing a buffer and overwriting pointers, the OS loads the application code (the main executable and its DLL) in randomized locations, so as to make it harder for the attacker to actually hit a meaningful location. By construction, it works better when the playground is larger. The extended address space in 64-bit mode is a larger playground.

To quantify things: in 32-bit mode, the address space is, at most, 32-bit wide (actually 31-bit when the kernel is not aware of 64-bit mode). The DLL loader must respect page alignment, and a page is 4 kB on x86. This gives, at most, 20 bits of entropy in the randomization of ASLR, but in practice lower than that because spreading the DLL throughout the whole address space can induce higher fragmentation (preventing the application from allocating, e.g., a 1 GB continuous buffer, something which is commonplace when editing pictures or movies). This is what the text you quote alludes to. Let's assume, for instance, that the DLL loading will occur only within one eighth of the address space, i.e. 29 bits; with alignment, this means 17 bits of entropy, aka 131072 possible locations for any given DLL. In any case, an industrious attacker could rely on luck, by simply repeating the attack (with a script) a few dozen thousand times until it works.

With 64-bit mode, the address space is extended to 48 bits, lowered to 44 bits on Windows, because they did not bother to go beyond. This adds 12 bits of entropy; in other words, it multiplies by 4096 the attacker's effort (in case the attackers relies on luck).

Therefore, it is true that, in some way, the extended address space of 64-bit mode increases security. Of course, ASLR has any effect only when a buffer overflow has occurred, in which case you are already in big trouble.


How can I determine the domain controller when it's not in DNS?

Jumping on the NMAP bandwagon: also look for machines that have TCP port 389 (LDAP), 636 (LDAPS), 3268 (LDAP Global Catalog), or 3269 (LDAPS Global Catalog). The last two are particularly juicy, since only a DC can be a Global Catalog server.


Is using full TLS suite in all communication scenarios a defacto standard?

Strictly speaking, TLS already supports what you're asking about. There are "null cipher suites" - that is, TLS cipher suites that have no ("null") encryption - that would do what you want. Using one will allow you to use existing TLS libraries and take advantage of all the work and hard thought and lessons learned that has gone into the TLS standard and implementations in major libraries. You can also just use TLS with an AEAD (Authenticated Encryption with Associated Data) cipher - most often AES-GCM (the AES block cipher in Galois/Counter Mode of operation) - and send your data as the "Associated Data" rather than the message contents; the associated data is part of the integrity check but is not encrypted.
However, as several commenters have pointed out, there's almost certainly no good reason to do this. Encryption and decryption with modern algorithms and hardware is very fast - potentially much faster than the network connection the traffic arrives over - and once you have the ability to establish a secure channel, there's no reason to make it less secure than you could. If you're running so extremely close to the performance threshold that the time spent on encryption/decryption is meaningful, some very slightly better hardware is probably a better solution. Also, make sure you're using a TLS library that is built for performance, and that supports AES-NI (AES native instructions, which all modern CPUs support).


PCI Compliant Secure Vault Design and Implementation

"Secure Vault" is being used here in the context of tokenization, which almost always means "the Vault that someone else (e.g., your payment processor) keeps the cards in, and you only ever deal with the tokens that correspond to items in the vault."

You can, of course, implement a "vault" yourself - that means a data store for PAN data which meets all the criterion of the PCI DSS such as encryption, authentication, access control, logging, etc. etc.  Read the DSS, implement what it says, and do it well enough to pass an audit - voila, you're done!

But, again, "vault" is used to identify the data store backing tokens.  If you're storing cards in your vault right next to the system where you're storing and using the tokens, you haven't gained anything; you still have to do all the work of protecting the PAN data.  If you're not using tokens, it's not a vault, it's just a database with PAN data in PCI scope and subject to the DSS.

The purpose of tokenization is to limit the burden of the DSS.  If your processor stores the PAN data, and only gives you the tokens, then you do not need to store and maintain them to the same level of compliance that you would have to if you stored the PAN data.  "Vault" is just a term that describes this separation between the sensitive storage tier and the end-user (you, the merchant) tier (that is, the tokens).


Could letters alone in a longer password be just as strong as one with digits and specials?

The Basic idea

Yes, the concept you're looping around is "how many possible passwords exist under these rules?", which is formally called the "Password Space".

The formula to figure out how big the password space is for a given set of rules is quite simple:

(num_allowed_symbols) ^ (password_length)

Your examples:

16 character password consisting only of lower case English letters: there are 26 lowers, so:

26 ^ 16 = 4.4x10^22 possible passwords
8 character password consisting of English letters of both cases, numeric digits, and special characters: depends on how many special characters you allow, but lets say 26 lowers + 26 uppers + 10 digits + 16 specials = 78 characters, so

78 ^ 8 = 1.4x10^15 possible passwords
Unicode: let's say there's around 100,000 valid unicode characters. Then you only need a 5 or 3 character password (respectively) to match the password spaces of lowers, and lowers, uppers, digits, specials.

but before we get too excited over these numbers ...

Where it gets real

The number of possible passwords above is only really useful as a measure of security if every possible password is equally likely to be chosen. In my experience this is not true since graycat123 is a far more common password than h&9R#d3*ng (or ǨΞᾃЯ ҉ℒⵞ௵ޘ↡╤ if using the whole unicode space, but don't ask me how to type that on an Android keyboard!)

This very quickly gets into psychology and user-habits and we often see stats like this:

  SplashData estimates almost 10% of people have used at least one of the 25 worst passwords on this year’s list, and nearly 3% of people have used the worst password, 123456.

That's why you see more and more websites giving "password strength meters" next to the Change Password box -- these are checking your password against common lists, looking for dictionary words, looking for keyboard patterns, etc. These meters are a far better measure of password strength than just the length because, well, people are bad at choosing passwords.

Far better to use a password manager and let a machine choose your password so that h&9R#d3*ng is just as likely as graycat123. If you want passwords that you can remember, then a good option is to come up with passwords using the diceware trick.


How does Google save authentication information in the browser?

Yes, you are correct that authentication tokens like session identifiers are often stored in cookies. Sometimes local storage is used instead of cookies, but that has basically the same security implications.

There are a few things that makes it more secure than it may sound at first:

At least some browsers on some systems encrypt the cookies. (This will not protect against an attacker that has access to your system in a logged in state, but it's something.)
Google probably implement  some kind of sanity check every time an authentication token is used, like checking some kind of browser fingerprint or the geolocation of the IP.
Finally, an attacker that has access to your computer so that she can read your cookies has pretty much won anyway. In such a situation, cookie theft is only one of many many problems you have.

In the end, picking an expiration time is a security versus usability trade off. It's a matter of opinion, but I don't find Googles choice unreasonable.


Any time you execute code acquired from someone that you haven't fully reviewed and it runs on an Internet connected system, there is a risk that the person who wrote or deployed that code could transmit data about your usage to another system.  That's true regardless of the OS. So yes it's possible.

The question then becomes "has this happened in the past", and "is it likely to happen in the future".  The only case that springs to mind where people may have been sending data form a linux distro to 3rd parties inadvertently was the ubuntu linux Shopping Lens which could be regarded as spyware and was by some.

Outside of that, I'm not aware of any instances of large-scale spying from the mentioned linux distros. As you say pentest distros like kali are obviously an inviting target, but then their users are more likely to notice an indiscriminate transmission of data from their systems.

Ultimately it comes down to trust.  By executing code belonging to someone else you are trusting them (think of trust in this context as "the power to betray") with any data you enter into that system.

How you establish trust in a system is a really good question and one which, as far as I can see, is far from answered.


How does one detect and deter golden SAML attacks?

Several methods could be used to detect'em:

Correlating service provider login events with corresponding authentication events in ADFS and Domain Controllers

Identifying certificate export events in Active Directory Federation Services

Customizing SAML response to identify irregular access

Detecting malicious Ative Directory Federation Services trust
modification

Those methods are well documented here
And for mitigation, the best you could do is following best practice guides and recommendations, Microsoft provides an excellent resource for doing this.
FireEye published a well-detailed paper on the attack and provides some extra guidance on ADFS attack mitigation: Remediation and Hardening Strategies for Microsoft 365 to Defend Against UNC2452
If not using ADFS this should help.


Did iOS 11.1 fix KRACK for only iPhone 7 and newer?

It seems, that you were spot on with your questions. Several news sites report the same, for instance ArsTechnica mentions:

  Update: As Ars readers have pointed out, Apple's support documentation states that the iOS KRACK fix is available for iPhone 7 and later, and for the early 2016, 9.7-inch iPad Pro and later. It is unclear whether this means previous devices are still vulnerable, or they had previously not been vulnerable and thus didn't need an update. Ars has reached out to Apple for comment.

EDIT:
I dug through the comments on ArsTechnica and found this post. It states the following:

  For iOS, the bug was fixed in iOS 10.3.3. I was able to establish a theory based on firmware analysis and tested the theory on a few devices running older and newer iOS firmware. The pattern was that devices with A8 processors were vulnerable (crashed the Wi-Fi chip) if running iOS 10.3.2 or earlier, but models with older processors were not vulnerable for any iOS version I could test. (I wasn't able to test any A9 or newer models running iOS 10.3.2 or earlier, but firmware analysis points to them being vulnerable as there were fixes in 10.3.3.)

In the macrumors forum I've found this post, which says something else, I wasn't able to find the part in the transcripts until now. Maybe someone else can shed some light on this.

  Yep apparently in the older devices iOS WPA2 implementation was not correctly done and as a side benefit are not vulnerable to KRACK. According to Steve Gibson of Security Now, iOS was not vulnerable to KRACK because they misimplemented the WPA2 specification. My guess is iP7 and up is when WPA2 was fixed and therefore now those devices need the patch! Perhaps Apple will relay this information soon to clear it up for all. That could mean the non iOS11 compatible devices are all fine as well.

So that might be the reason why there was only a fix for the newer devices.
In the KRACK paper itself, the iOS 10.3.1 was marked as not vulnerable to the retransmission of the 3rd message of the 4-way handshake.

Quote from the paper:

  In particular, Windows and iOS do not accept retransmissions of message
  3 (see Table 1 column 2). This violates the 802.11 standard. As
  a result, these implementations are not vulnerable to our key reinstallation
  attack against the 4-way handshake.

But:

  Unfortunately, from
  a defenders perspective, both iOS1 and Windows are still vulnerable
  to our attack against the group key handshake. Additionally, because both OSes support 802.11r, it is still possible to indirectly attack them by performing a key reinstallation attack against the AP during an FT handshake.

1 This is iOS 10.3.1

So both OSes are still vulnerable, but not what is "commonly" known as KRACK - which would be exploiting the 4-way handshake. I guess iOS 11.0 had a correct implementation of the 802.11 standard and this is what was now fixed. (The description in the patch notes sounds a bit fuzzy to me tbh.)

Here's a screenshot of that fancy table:


how secure is an iPhone 4S+ with iOS 6+ in 2013?

AFAIK there is no DFU mode bug publicly available for anything later than the iPhone 4 or iPad 1.

In terms of typical lost device scenario, assuming that the attacker can't guess the passcode before the device wipes (so set a relatively strong passcode an enforce device wipe on 10-20 incorrect attempts), I'd say that the device should be relatively secure.

There are lockscreen bypass bugs in current versions which could disclose some information from the device and also allow attackers to do things like make calls from it, and in iOS 6.x there was the malicious charger issue, but in general there are no deep flaws that I'm aware of which would compromise the security of the device if it's correctly configured and the attacker can't get access to it when it's unlocked.


Mostly agree with Lynn.

And to add more - the public clouds are general purpose clouds and hence the privacy is not fully implemented. That's the core of HIPAA requirement. But if there are clouds that are equivalent to HealthVault or purely private clouds, its very much possible.

My point is - the current public clouds aren't designed with healthcare in mind. But a general purpose connected applications. And hence they aren't fully HIPAA compliant. Also people who manage the clouds needs to have HIPAA or CITI or similar certifications in order to be called as qualified personnel to manage the cloud! I strongly doubt if there are any public cloud offerings that have these basic criteria satisfied!

Of course this is my personal observation after working in this industry for long and no obligations.


What happens if my anti-CSRF token is compromised by an XSS attack?

If somebody has an XSS in your site, they can do anything they want on that origin -- submit any form, perform any operation, etc.  Javascript can modify the DOM in arbitrary ways, submit forms, modify the user's view, etc.

Consequently, stealing the CSRF token is the least of your worries -- the XSS allows anything on the same origin that a CSRF would allow.  The only time you might worry about this is if you share the same CSRF token across multiple origins (which would be a poor practice to begin with).


Is it as easy to crack a wifi password, as shown in the video?

Essentially, yes, the person in the video was cracking an "easy" WiFi. The password chosen to secure the network ("plausible") was simple enough to be contained in a short wordlist. They simply tested all of the entries in the wordlist until they found the correct password. In practice, this method is unlikely to work as effectively as it did in the video because most actual networks are, at least in theory, secured with longer and more complex passwords that would not be found in most wordlists. Brute-forcing takes exponentially longer as password length grows, so once a certain password length and complexity is reached this method becomes impractical.

Paid public WiFi networks generally don't use any encryption at all, so that is a separate discussion entirely.


POTS lines (Plain Old Telephone Service) are not encyrpted by the carrier.

While the signal can be intercepted by tapping into the physical line, as in some guy climbing up a telephone pole, the more modern method of intercepting land line conversations is by tapping into the line at the switch level, which requires access to the telephone carriers' facilities control systems.

https://en.m.wikipedia.org/wiki/Room_641A


How can I authenticate the origin of an email?

Yes, but all the reliable methods are optional:

PGP signing (also depends on you already having a large PGP network or directly having accepted the real senders public key)
x509 signing (S/MIME) - which few people use, despite relatively painless setup on the signing side
DMARC - which probably isn't supported in your mail user agent but is gaining wide adoption in MTAs.
Sender Policy Framework - again is probably ignored by your user agent but commonly implemented in MTAs - Not as robust as the methods above which use cryptographic signatures.

A further issue with all of these technologies is that they are designed as means for computers to validate the identity of other computers rather than as a way of proving to human beings that an email is genuine.


What bios passwords & settings are important to set on a Thinkpad X1 laptop?

BIOS passwords just deter the lowest possible attack vector: someone without any technical knowledge booting your laptop/from an external drive. If you do not have BitLocker in place, an attacker could then access data on your drives and affect the whole (Confidentiality, Integrity, Availability) CIA-triad. But even if you set a BIOS password, this would not hinder a more "advanced" attacker from simply removing your non-encrypted drive and plugging it into their own machine, again affecting the whole CIA-triad.

So to sum it up: BIOS passwords are easy to set up, so you might as well. Even if it's just to "protect" your device from someone who's technically completely unsavvy trying to boot from it.

Additionally, you definitely should use Full Disk Encryption to protect your local data from technically more advanced threats. Just make sure to use an adequately long and complex password. And keep in mind what @RubberStamp wrote in his comment: FDE only protects your data if your drive is not powered on. So power off* your machine whenever you're not using it/it is not in your direct line of sight.

EDIT: * Sleep seems to be an issue with BitLocker, according to this answer, hibernate seems to be ok according to this comment.


Can I convert a public key to a private key?

PGP is what is called an "asymmetric cryptosystem."  It is asymmetric because the key used to decrypt data is different than the key used to encrypt data.  In this case, you encrypt data with the public key, and then decrypt with the private key.  If you lose the private key, data encrypted with the public key is not recoverable.  In the case of a 2048-bit RSA key, chances are you will never be able to recover the key.

Take a look at the Wikipedia Article for a more thorough explanation.


Can I do a String Format Exploit for x64 systems?

If you're using the example code from the book (below), at some point you should reach the "AAAAAAAA" pattern (0x41). Note that, since you're running it on a 64-bit machine that stores elements in the stack with 8 bytes each, you should run it with $ ./fmtstr "AAAAAAAA %016x %016x %016x %016x %016x %016x %016x %016x %016x %016x %016x %016x %016x" instead, or you will miss part of each element on the stack.

#include <stdlib.h>
int main(int argc, char *argv[]){
        static int canary=0;   // stores the canary value in .data section
        char temp[2048];       // string to hold large temp string
      strcpy(temp, argv[1]);   // take argv1 input and jam into temp
      printf(temp);            // print value of temp
      printf("\n");            // print carriage return
      printf("Canary at 0x%08x = 0x%08x\n", &canary, canary); //print canary
}

You should pay attention to the quote in the book that states:

  The  fact  that  the  fourth  item  shown  (from  the  stack) was our format string depends on the nature of the format function used and the location  of  the  vulnerable  call  in  the  vulnerable  program.  To  find  this  value,  simply  use  brute force and keep increasing the number of %08x tokens until the beginning of the format string is found. For our simple example (fmtstr), the distance, called the offset, is defined as 4.

Remember that the parameter being parsed to printf isn't the string itself, but the address of the string. So it's position on the memory layout of the program in relation to the printf stack is what will define how further you'll have to search to find it.


Is there a way to prove authorship in a shared repository?

You need a trusted 3rd-party to witness and vouch for every commit. For example, if you use an online repository such as GitHub, then there is an online record of each commit. If you are not using a trusted 3rd-party service, then you need to use a notary or similar service. This could be a person who witnesses each commit, or an online notary that performs the work electronically. You could simply mail or email commits to a trusted person (perhaps an attorney).

All of these solutions rely on the trustworthiness of the 3rd-party.

Note that all of these solutions only prove that the work existed at a specific date. They cannot prove whose brain the work originated in. For that you'll need a mind reader.


how to debug for DNS changing Maleware on (linux/android)?

It is most likely that your router was infected and android takes its dns server from dhcp. There are a few driveby attacks that work on routers like this.


We can read the ssh private key with less id_rsa

You don't need the passphrase to read the file, just use it. If you created it with a passphrase then it is encrypted. However, the difference between a private key and an encrypted private key are invisible to the naked eye - both are effectively random bytes/strings.

As a result, you can't necessarily tell that it is encrypted just by looking at the file. You can't use it without the password though, which is the important part.

With older file formats, the presence of a key was visible as you say.  The new OpenSSH encrypted private key format does not indicate in a human-readable way whether or not there is a passphrase.  Therefore, the easiest way to verify if the file has a passphrase or not is by trying to actually do something with it.  As a simple test just run:

ssh-keygen -yf /path/to/private/key

If it asks you for a passphrase then the private key has one. If it doesn't then there is no passphrase.


There is no good way. What you say, is practically the measurement of the password distance in our mind. It is clearly impossible to have a direct method to do that.

Second thing, what you want to measure, depends heavily on the person, and contains often only for him known informations. For example, one of your collegues could use the name of its childs on the different company servers. It is not possible to build a software solution to find this, but some hacker / collegaues can have this information and use them to crack his account.

What you can do, were a step into the track of the NSA: although you can't spy peoples mind directly, you can use Big Data to emulate some very similar.

What you need: publicly available informations on the net. For example:

Thesaurus
Wikipedia (although there is no simple way to measure the link distance of two keywords, its database is simply downloadable and you can build a script to analyze its link connectivity).
Or simply you could do automatized google searches with the google search api, and get ratio of hits between the first, between the second password and between a dual query (for example, if the first password is "apple" and the second is "orange", then the Hits("apple")*Hits("orange")/Hits("apple", "orange")^2 had to be below an experimental limit set by you).

But beware: don't execute queries containing the passwords into an untrusted public cloud, it were a very serious security breach! Of course, it depends only on your viewpoints/considerations/responsibility, which public cloud is trusted for you. For me, none were.

In your place I did the following:

I get a wikipedia mirror (they have simple mysql database which is publicly downloadable)
Created a link distance map (it were very simple, although it were maybe big)
I created for the two passwords to compare to the their nearest wikipedia article title (it needed probably a massive levenshtein comparation, so you will need a lot of cpu)
Finally I used the following formula: D("pwd1", "pwd2") = Levensheiten("pwd1", Lev_nearest("pwd1")) + Wiki_Link_Distance(Lev_nearest("pwd1"), Lev_nearest("pwd2"))+ Levensheiten("pwd2", Lev_nearest("pwd2"))

Extension: wiki contains around of some 1million of text entries, which makes the shortest way search nearly impossible. You had surely implement this as a C++, and use very well optimized algorithms. Thus, it will be hard. As an alternative, you can do that you use from the wikipedia only the most common words (which can be found by getting their usage stats). Although the english wiki has around some million articles, a native english speaker knows only around some ten thousands from them.

Somebody should really write this, it were a wonderful opensource demon somewhere in the github :-)


Can Emule MD4 hashes and file sizes be faked?

There are different levels of insecurity in hashes.

The first is a collision attack where you can create two different files that result in the same hash. In order to use this in the attack you describe, the attacker would need to create two files which share the same hash: one which contains the illegal content and another which contains harmless content. The attacker could then somehow distribute the file with the illegal content in the hope that it will be tracked by law enforcement and then distribute the innocent file to the neighbor in the hope that he shares it and then get blamed by law enforcement.

Creating a collision for MD4 can be done very fast according to Wikipedia.

The other and harder attack is a pre-image attack. In this case, the attacker only knows the hash (or maybe has a file with the hash) and then tries to create a new file which results in the same hash. This is much harder than creating a collision (where the resulting hash value does not matter, it only needs to be the same for both inputs) but it is still feasible with MD4 as shown already in 2008. But the computation power needed for this is not trivial and an attack shown in 2010 has a complexity of about 295 which is not in reach of the average attacker.

Apart from that the attacker is further limited since the MD4 is for a fixed-size chunk of the file so the attacker must stay within this limit when creating the attack.

Because of this, I don't think that a preimage attack can be used to blame an innocent neighbor since it is too costly for the attacker to execute. But it is probably doable in theory. A hash collision attack is more likely but in this case, the attacker needs to "seed" law enforcement with some specifically prepared illegal content first.


Just as the error says, --increment and skip/limit options are not compatible. There's no way to do what you're describing with a single hashcat command and a single mask. But if you expand your approach to include multiple masks or multiple invocations of hashcat, you can get there.
I presume that for a fast hash like MD5, running through the shorter lengths wouldn't really take much time - so it's length 8 that you are probably most interested in splitting across systems.
I'd use three commands:

On one system, a script/batch file that runs hashcat incrementing lengths 3 through 7, and then runs hashcat again simply with length 8 (the full mask, no increment-related commands necessary) with your desired skip/limit options

On the other system, just run length 8 with your other desired skip/limit options.


Is there a simple and highly secure office suite?

Remember the I in the CIA triad - Integrity. It is also a security failure if these sensitive Word documents and Excel spreadsheets from outside don't convert properly when you open them in your office suite.

I fear you may have to accept that you need to run a proper copy of Office and put controls around that - for example, by opening them in a disposable VM that you throw away every time you use it. This is easier to do than you might think - most VM hosts allow you to easily roll back to a previous snapshot.


Is this a secure method of authenticating packets?

Vulnerabilities in your method

Your construction is most likely vulnerable to a length extension attack which allows for the modification of the message, as well as a replay attack which allows for the resending of previously send messages.

Obligatory: This is why you shouldn't roll your own. Instead, use existing solutions for the problems you want to solve.

Secure Approaches

A solution against the first issue is to use a proper HMAC. If you do need protection against replay attacks, you need to exchange nonces or similar.


What is sent between an HSM and an application?

Signing a CRL is like signing any other data and involves several steps:

Read the data you want to sign.
Compute a hash from the data you want to sign. This hash has the same length no matter initial data length and different data will result in different hash.
Encrypt the hash with the private key. The result of this operation is the actual signature.

This signature can then be appended to the signed data (added to the CRL to get the final, signed certificate to follow your example) to obtain the signed data.

As you see, the private key is only involved in the last step. It would be a waste of resources to send the whole data to the HSM, so the hash is calculated on the application side and only the hash is sent to the HSM to obtain the signature.


Relationship of password strength and key strength

Overview

Some examples for review/discussion.
Alternative: local password generated key for internal use in encryption-key access control API.
Entropy and increasing it.

1. Some examples for review/discussion

  In systems that take a user's pass-word/-phrase in order to derive the encryption key, what is the relationship between that password's strength and the key's strength?

This depends entirely on how you use the password to derive the key. If you're just hashing, or even hashing and salting your password, your function might look like this:

generateKey(password, salt) := cryptoHash(password, salt)

If I took a table of the weakest passwords I could probably calculate the potential hashes, and from there I would have the possible encryption keys. Then given a message I could crack it with this function:

crack(message, saltLen, weakPassList):=
    for salt in 0..saltLen:
        for weak in weakPassList:
            let k := generateKey(weak, salt)
            if decrypt(message, k) looks okay:
                return k

This might take a really long time to run based on how big saltLen and weakPassList are; however you could cache the k generated above in a table and just try them out one at a time, to eliminate the factor of experimental key generation.

There are definitely better ways to generate a password, but without a more detailed specification or use case it's difficult to analyze or recommend which method is best.

2. Alternative: local password generated key for internal use in encryption-key access control API.

  I know password strength matters in authentication; is it a moot point in encryption?

I feel like there's no difference unless you are talking about a particular method of key generation or some other specific use case. A key is a key.

Personally I'd go with using the password to generate a local encryption key kLocal, used to obscure a properly generated cryptographic key kMessage that can be used to encrypt messages for sending over non-secure channels.

This way the password is just an access control mechanism to filter users who can type commands into the system, and prevent them from using each other's encryption keys (if they can't get the right kLocal they can't discover the contents of kMessage). This is apt because you can use a secure API with, for example, a leaky bucket, to prevent people from doing password guessing attacks on the local machine.

It's not as straightforward to prevent password-guessing attacks against a ciphertext message that's already been sent out in the open! By making the password-generated kLocal for internal, API-restricted usage only, you bound the attacker's effective computation power, which is a really good thing in software security.

3. Entropy and increasing it.

  From my understanding, human-created passwords tend to have very low entropy, and that's why keys are derived from them.

Be careful! Entropy is just the measure of how unpredictable something is. If I take some values with X amount of entropy, and run them each through some adversary-known function, the results can have no more than X entropy. That is, if the adversary could guess the original values and also knows the function for manipulating/extending them, he'll have just as easy of a time guessing the manipulated/extended values as he would the originals.

You're not increasing the size of the domain unless you add more information; and you're not increasing the entropy unless the information you add has some entropy itself (is somehow unpredictable/secret).


Should SMS systems warn users of possible spoofing attacks?

Apple did not create the SMS standard that was provided to them by the industry.  The industry indicated that an SMS transaction can include a reply-to field.  Apple has designed their software to use what the industry told them existed (and what industry equipment is compatible with as it relays it intact).  As long as Apple isn't using the provided standard incorrectly, they can't be to blame.  If a reply-to field is included, and is different than the from field, what exactly is the purpose of the reply-to field if not to prevent you from sending the SMS to the "from" originator?  What would everyone expect behavior to be exactly?

An example: Apple builds computers.  My Mac Book Pro (running OS X 10.8) can be fooled by a spoofed MAC address.  Is that Apples fault?  My MBP can be fooled by a spoofed IP address.  Also Apples fault?  No in both cases, the IP stack was designed long before Apple had anything to do with networking computers.  MAC control being spoofed is not something easily detectable (unless you know for sure what MAC address you should expect to see).  Apple is using provided standards.

EDIT: If customers want a secure platform, they use iMessage. If compatibility is the concern, they use SMS. SMS is not, and has never been designed to be a secure platform.

I'm of the opinion that @blunder can not compare SSL (a known secure system) to SMS (a known insecure system).  Telnet is an unsecured communications medium and would be a closer comparison.  How many computers pop up a notification that says, "Are you sure you want to use telnet, it sends passwords in cleartext?"  Is Apple liable because OS X doesn't have a popup that notifies someone trying to use telnet that it isn't secure?

My turn to ask a question.  Why are we bent on finding excuses to make Apple fix this, instead of having the cause of the problem (the SMS standard) fixed?  Why is fixing the symptom logical when you can cure it at the cause (the carriers)?


What are the security risks of storing user roles/types in a cookie?

"Safe" is always a relative term. If the data in the cookie is not used for authentication or authorisation, and it is just a configuration item that is exposed and could be used by anyone without a risk of misuse/abuse, then why not? The letters a d m i n are not special in and of themselves.

At that point, it's just a UI toggle switch, just like dark/light mode.

The real question is about knowing the potential impacts of such a change.

The risks are about some future developer of the site misusing the cookie for sensitive things without understanding the consequence.


Bottleneck while creating XSS scanner for a website which returns API response instead of HTML

In this case the XSS is actually independent of the server side, which means you can actually ignore sending data to the API and instead focus on the UI. This means you could essentially write an XSS scanner that mocks the API, and then detect changes in the browser. You could use webdriver/selenium or similar to drive the browser.

Another alternative could be to use phantomjs, have it render the response, and then grab the full HTML from that.


Basically when you hijack someones session you take their sessionID and pretend its your own. Usually the sessionID is transferred in the cookie, meaning that if you can access the other parties cookie you can just put it in your own cookie and you've stolen their session.

This can be done in several ways, for example by sniffing the wireless network and looking at the HTTP packets being transfered or by XSS attack where you can tell the victims browser to reveal their cookie information to you.

I would like to mention that the example you describe in your question may also be vulnerable to SQL-Injection. If I change my cookie's sessionID to

asdf' OR 1=1--

I would most likely be authenticated as a valid user. To prevent this you have to make sure you have proper sanitizing on dirty data coming from your clients before you use the data for anything.


Why does a (D)DoS attack slow down the CPU and crash a server?

How does one crash a server using (D)DoS?

To specifically answer your question, to crash a server using only DDoS you need to target the Application Layer (detailed explanation below). These types of attacks specifically attempt to use up as much of the target servers resources a possible and bring it down, rather than just hammer it with network traffic.
However, to put this into context alongside other types of DDoS attacks, lets explore their major categories and their uses.
This article covers the 3 major attack types for DDoS. From the article:

DDoS attacks can be broadly divided into three types:
Volume Based Attacks
Includes UDP floods, ICMP floods, and other spoofed-packet floods. The attack’s goal is to saturate the bandwidth of the attacked site, and magnitude is measured in bits per second [sic] "(Bps)" [sic].
Protocol Attacks
Includes SYN floods, fragmented packet attacks, Ping of Death, Smurf DDoS and more. This type of attack consumes actual server resources, or those of intermediate communication equipment, such as firewalls and load balancers, and is measured in Packets per second.
Application Layer Attacks
Includes low-and-slow attacks, GET/POST floods, attacks that target Apache, Windows or OpenBSD vulnerabilities and more. Comprised of seemingly legitimate and innocent requests, the goal of these attacks is to crash the web server, and the magnitude is measured in Requests per second.

TL;DR - there are multiple types of DDoS attacks depending on what the attacker wants to achieve. Sometimes an attacker will just want to take up all the available bandwidth, other times they will try overwhelm the CPU.
It's worth noting that DDoS is just a distributed type of the generic 'Denial of Service' - it does not imply crashing a server at all, only preventing the server from doing whatever it's intended for, whether thats preventing actual business from taking place by using all bandwidth or otherwise.


Using the same base string for a password in several different logins

The biggest issue is that, if two of your passwords leak, an attacker will be able to find a pattern within your passwords. This would allow him to generate a dictionary attack by applying the found logic to other recovered hashes to try and find your other passwords.

p@ssw0rdstring1 and p@ssw0rdstring2 only differ one character. This means that an attacker can easily guess you will probably also have used p@ssw0rdstring3 or p@ssw0rdstring4.  While you can use a password base to increase the length of your password it will not increase security if an attacker knows you are always using this prefix.

While you are not specifying how you will choose the rest of your password, I must insist not to use anything which is site specific. So what comes after your password base should be completely random. If it's random you should remember it per website, which means that per website you need to remember a password, so why not remember something random completely without using a base?

If you have difficulties remembering passwords you should try to use a password manager like KeePass or Passwordsafe.  Passwords managers allow you to choose different passwords per website. These passwords are protected by a single keyfile and/or password. Choosing a unique password, which you do not repeat, is of the essence here.


Accessing Nmap flags in Nmap script (NSE)

There is not a way in the current NSE API to access that information. Most of the scripts that save output to files take a filename or target directory as a script-arg. I've just opened a new issue on Github to address this idea, so we'll see if we can improve this in the future.


Privilege escalation writing /etc/passwd but without SUID permission on su

On a standard Linux host (VM or server) if you have access to /etc/passwd or /etc/shadow then you can modify or add credentials to accounts, which would obviously be a serious problem (there's details on the consequences of allowing a modification to /etc/passwd in this answer
In a Docker container, usually the software which makes use of these files is not installed, so no Linux PAM, no SSHD, or other software which would make use of an authentication database.
Without that type of software being installed, there's not much you can do, as there are no programs which operate on those files.
Of course, if the application running in the container makes use of /etc/passwd then modifying it could have an impact, but that'll be situational depending on what the container does.


Can you set a custom x-forwarded-host header in a CSRF form page?

You cannot explicitly set any HTTP headers in a HTML form apart from choosing the Content-Type between two predefined values. This means you also cannot set some x-forwarded-host header.


Why is it a security problem not to update one's browser?

Because there are security vulnerabilities found in software all the time.  These vulnerabilities are sometimes publicly disclosed, sometimes not.  Either way, as developers find or find out about them they patch them.  Running old versions of browsers leaves you vulnerable to malicious websites trying to infect your computer.
Below are links to web pages listing vulnerabilities that have been fixed in relatively recent versions of the 3 most popular browsers.
Microsoft Internet Explorer
Mozilla Firefox
Google Chrome
All browsers are going to have bugs, and all of them will have vulnerabilities.  But staying on top of known vulnerabilities can help prevent attackers from gaining access to your system.
Edit
Thanks to kirb for these extra links to up-to-date blogs of browser security updates
IEBlog
Google Chrome Releases


Generate RSA2 key in OpenSSH format in windows

I just cited below from PuTTY manual:

  Selecting the Type of Key

  Before generating a key pair using PuTTYgen, you need to select which
  type of key you need. PuTTYgen currently supports three types of key:

An RSA key for use with the SSH-1 protocol.
An RSA key for use with the SSH-2 protocol.
A DSA key for use with the SSH-2 protocol.

  The SSH-1 protocol only supports RSA keys; if you will be connecting
  using the SSH-1 protocol, you must select the first key type or your
  key will be completely useless.

  The SSH-2 protocol supports more than one key type. The two types
  supported by WinSCP are RSA and DSA.

  The WinSCP developers strongly recommend you use RSA. DSA has an
  intrinsic weakness which makes it very easy to create a signature
  which contains enough information to give away the private key! This
  would allow an attacker to pretend to be you for any number of future
  sessions. WinSCP’s implementation has taken very careful precautions
  to avoid this weakness, but we cannot be 100% certain we have managed
  it, and if you have the choice we strongly recommend using RSA keys
  instead.

  If you really need to connect to an SSH server which only supports
  DSA, then you probably have no choice but to use DSA. If you do use
  DSA, we recommend you do not use the same key to authenticate with
  more than one server.

Reference link: https://winscp.net/eng/docs/ui_puttygen#generating


My question came from that little phrase "WPA key length". What does
  that phrase mean?

It appears to mean the length of the WPA key in bytes.

The "WPA Key" element is displayed in your figure right below the "WPA Key Length" element. The "WPA Key" appears in your figure as a string of 48 ASCII characters representing a sequence of 24 bytes. E.g., "DD" represents the byte 0xDD, which is 221 in decimal. E.g, "16" represents the byte 0x16, which is 22 in decimal.


Yes, malicious software running as your user can do anything that you can do as a user.  There is no privilege separation between different applications running as the same user.  This has also been noticed by xkcd.

To protect your SSH key specifically, you can add a passphrase, as mentioned by galoget, but that will only help marginally -- a malicious program could also scrape memory when the key is in use or log your keystrokes for your passphrase.  (Though there are some mitigations for those attacks.)

A better option is to use a hardware token of some sort -- I use a Yubikey 4 with an OpenPGP key for SSH authorization.  Though my key might be abused when unlocked, the key material itself cannot be stolen, and it is only unlocked for a short period of time.  (You could set this timeout to near 0 and require user interaction for each use of the key.)

Another option is to store your key under another user account and use su to switch to that account before SSHing.

In any case, if you have malicious software running as your user on your computer, you're going to have a bad time.


Mail server security: Authentication in SMTP servers for mails from same domain

how does SMTP server recognizes the tampered header?

They don't unless they require authentication. With authentication it can be determined if the authenticated user is the same as the acclaimed sender of the mail.

SPF and DKIM are not used in this case. These care only about spoofing the domain part of the senders address and do not care about single users at all.
Thus they can be used to find out if mails coming in from outside spoof the domain name. But they can not be used to find out if mails from  spoof the user part of the senders address.


How does the Intel CPU design flaw affect us?

The Spectre and Meltdown attacks are about programs reading data they're not supposed to.  They don't provide any way to make modifications or take control of computers.

If you're running a typical desktop system, the flaws aren't a major threat simply because anyone in a position to use them has so many other ways to do the same thing.  The real risk is to cloud services and shared-hosting providers, because the attacks let one user read another user's data.

The fix to Meltdown will slow things down by 5% to 30% (depending on your CPU and what you're doing); computer games appear to be at the 5% end of things.  There is no general-purpose fix to Spectre, but it's also much harder and slower to exploit than Meltdown.


IPs get changed and reassigned all the time. Any automatically collected reputation data gets stale fast. It's unlikely sites will archive that info.

But, I have been in the same situation. What I ended up doing was using my Google Fu and the Wayback Machine to search during that timeframe to see if forums, reputation sites, etc. mentioned the IPs in question and I was able to find enough information to help an investigation.

But there will be no guarantees that you will have similar success.


Encryption strategies for multi users access revisited

"What happens when a new user is added to the system."

A new user can only be added by an existing user who has already decrypted the disk.  This existing user supplies Kc, which only ever resides in memory (not on disk).

"Why does every user need their own self-encrypted copy of Kc"

Because none of the alternatives are acceptable.  Let's look at them.  Suppose all user records shared an unencrypted copy of Kc.  This Kc would have to be stored on disk somewhere to persist across reboots.  In that case, the disk's data is encrypted, but the plaintext decryption key is stored on the same disk, so anyone with a disk image can decrypt the disk.  Useless.

Suppose that each user had an encrypted copy of Kc, but all were encrypted with the same key.  Then each user would have to supply this key, presumably in the form of a password or passphrase.  But if every user has the same password, the system is not suitable for multiple users.

"What happens when a user is deleted from the system?"

A new master key Kc has to be generated, the disk's contents have to be reencrypted with the new Kc and every remaining users' Kui's has to be recreated.


SPF and DKIM passes for SPAM message when using SES and Google Mail

From the header you posted, "WEBSITE-DOMAIN.org" is your domain, correct? If so, then this spammer is spoofing the header from address. You can prevent header from spoofing with DMARC.

DMARC confirms that the header from address domain is "in alignment" with either the MAIL FROM domain or the DKIM signing domain. "Alignment" means that either the domains match exactly (strict mode) or it's a subdomain of the header from address domain (relaxed mode).

Implementing DMARC can be a little complicated depending on your specific configuration, and it's easy to accidentally block your own legitimate email if not done correctly. The details will vary, but here's the basic process:

Create a list of all legitimate sources of email from your domain. You mentioned SES and GSuite, but there may be others you don't know about (from shadow IT, for instance).
Create a DMARC DNS record, making sure to set the policy to monitor mode ("none")
Configure all known legitimate email sources to use your domain/subdomain for DKIM signing or use your domain/subdomain for MAIL FROM (preferably both, but not necessary), adding/modifying SPF and DKIM records as appropriate. Most mainstream email services accommodate this, including SES and GSuite, but unfortunately some still don't.
You'll start receiving aggregate DMARC reports at the email address specified in your DMARC record. Review these for at least a few weeks to make sure you haven't overlooked a source of legitimate email from your domain. Also review the headers from emails sent by your company to confirm that they're passing DMARC. Check the headers from as many different receiving services as possible, because they don't all handle DMARC the same way (especially if automatic email forwarding is involved)
When you're sure all of your domain's legitimate emails are passing DMARC, change the DMARC policy specified in the DNS record to "quarantine" or "reject"


Is there any risk related to allow user to give an URL as their image profile?

There's actually quite a few issues with the approach:

You need to worry about CSRF attacks (someone could use http://somesite.com/?q=deleteallmydata and anyone visiting the site will have their data on somesite.com deleted), and depending on your code, you're more vulnerable to XSS attacks.
This can be sophisticated - for example, they could show an image to begin with, then switch it with a redirect.

If your site is busy, it can be used to stage an effective DDOS attack - someone could enter a URL for some site they dislike, probably in a way that runs up server resources (a search page or similar).

Attacks on a user's image host are now attacks on your users.

If you don't want to handle images - you may want to consider a trusted host, like http://imgur.com/ (StackExchange does something like this) to direct your users to, and limit your URLs to that site only.


Reverse IP Lookup usually returns the domains under the primary Top Level Domains hosted on a given  IP address (since that is what is usually placed in the PTR record), not a complete list of websites hosted at the IP. This typically does not include sub-domains since the subdomains are not directly under the primary top level domains.

And how can I find all the websites on a particular IP?

If you want to enumerate all websites including sub-domains, you can try using a reverse IP lookup to find the domains hosted on that IP and then enumerate sub-domains for all of them.


Maintaining data security in motion with SSL

TL;DR: Any security policy is going to require basic trust of employees and business partners, which should be balanced with controls.

  In theory, isn't it possible for a rogue employee at the hosting company to use the private key to decrypt the data in motion?

Yes, it is possible.

  This seems to be a hole in our security policy

Not really.  You have a business relationship and a BAA with this company.  It is a reasonable assumption that they will act responsibly and not attempt to maliciously compromise you given their access.  You yourself acknowledge this when you say "rogue employee" - you're not worried about them being bad, you're worried about a bad apple that works for them.

By the same token, aren't you worried about the bad apples that work for you?  You should be.

Are there access controls on the key files?  Are there activity logs on the server?  Do you have access to them?  Do you audit them for inappropriate access?  These are all things you should do, whether it's your server and your employees or the hosting provider's server and employees.


HTTPS is not green and there is no padlock icon, is it safe for a debit card payment?

Note: this question is a few years old and browsers are changing how they report issues with secure sites over time.  The below answer still describes why some problems may exist but the way the browsers show it may have changed.  For example, sometimes browsers may block non-secure elements on a secure page instead of loading them and showing a warning.

The little "i" in a circle contains important information regarding why the HTTPS or site name is not green or shows no padlock.  In this case the explanation is that the connection is not secure - a suitably vague warning intended for average users, but nonetheless correct and worth trusting.  The "security" tab in the developer tools accessible by F12 key can often give more info if you are that way inclined.|

Why is it not secure even though it says it's using HTTPS?

Based on the notice it's giving, in this case the likely (but not definite; this is an educated guess) explanation is that while the page itself uses HTTPS, the page contains items on it which are fetched separately over plain HTTP.

This may sound like it's not a big security issue, but if any of the fetched items have been modified by an attacker it is possible for them to modify the page in misleading ways or even read and steal private information from the page, even though the rest of the page was initially fetched over HTTPS.  Scripts included in the page may read anything you type into forms or text fields.  CSS may modify the page to trick you into pressing something or submitting something.  Images may make the page appear to be something it isn't or try to exploit bugs.

Browsers respond to this problem in varying ways, but at minimum they indicate that the page is "not secure" in the location bar somehow.  They may display a warning and they may also block loading some elements on the page.

Why is it sometimes green but sometimes not?

This issue is usually an accident on the part of whoever built the website: they forgot to ensure some of the items on the page are using HTTPS.  Different pages can fetch different items: images, scripts, styles.  Sometimes one of the scripts fetched by a page can in turn fetch scripts of its own, and then the page is marked insecure due to a fault by a third party script that the owner of the website may not have anticipated.

What's an appropriate response?

On any page that is not marked as secure by your browser, you might choose to avoid entering personal or credit card information.  You could report the problem to whoever owns the site (as it could be losing them sales).


Accessing SQL database from Java Android app. How do I store the SQL database details so they are safe?

Since I assume your database will be accessible to the Internet, definitely use an encrypted connection between your app and the database. MySQL has documentation on this here.

One approach is to wrap the database with a web service. This lets you store the database password in the web service, instead of the app. This also gives you more power in deciding what queries are run against the database.

Another approach is to dynamically generate a user in the database for the app when the app is installed. This will allow you to make sure that the user's database credentials are unique, so that any action done against the database can be traced to a single installation of the app.

However, this leads me to a bit of a tangent:

Consider role-based access control for your app.

If your app only reads data from the database, and if you are concerned with the integrity of data in your database, consider creating two user roles in your database. One user role will be able to create, modify, or delete data: this user is for your use as administrator of the app. The other user role can only read data: this is the role of user your app will use to log into the database.

MySQL has capabilities to perform this sort of role-based access control, albeit awkwardly - check out the MySQL documentation to learn more.


I'm a bit confused about your vernacular. But I'll try.

I believe you are using the word "session" to refer to single sign-on (SSO). This is functionality that allows you to sign into one site and have that sign-on propagated to other sites without requiring re-authentication. There are several standards for this. They include OAuth1, SAML, and OpenID Connect.

SSO is not the same thing as a shared session. A shared session implies some shared state beyond authentication while SSO only affects authentication. Sharing state across multiple websites tends to be difficult due to security features in the browser, the same-origin policy probably being the biggest. Features like supercookies and Flash cookies are sometimes used to implement shared sessions, but they tend to be very limited in functionality due to the browser's security protections.

Regarding the example you gave of Google and Spotify, they do not share a session. Instead, they use OAuth to allow a Google login to be used to create a new Spotify account. Once created, the accounts are disconnected. For example, logging out of one service will not log you out of the other one.

1Technically, OAuth is an authorization protocol, but it frequently looks like an SSO.


You want to distribute the flash drives yourself, and forbid usage of any flash drive which has not been "approved". Reason: there are devices out there, which look like flash drive, but, when inserted, tell to the computer that they really are keyboards, and begin typing things wildly.

If only genuine flash drives are ever inserted in the computers you want to protect, you should be able to deal with malware, with a "reasonable" success rate, through the usual tools (antivirus & co). Malware risk can be drastically lowered by enforcing use of an operating system which is resilient to malware, e.g. NetBSD (resiliency being achieved mostly by rarity - malware developers do not bother with targeting NetBSD). But users may object...

Be aware that whatever you do, users will insert random USB devices into machines to which they have physical access, unless you pour glue into the USB ports. Best defense is probably a combination of up-to-date operating systems, good antivirus/malware detector, and, more important than everything else, security awareness training. You will be infected at some point -- but you can hope for a low rate of successful infections if the users, on average, exercise caution.


What are the implications of omitting the username in Secure Remote Password's verifier?

The original SRP specification used H(s,P) for a not-thoroughly-specified password derivation scheme, which includes the salt and the password together (but not the identity). The first draft for RFC 2945 (that's SRP-3, predecessor to SRP-6) already includes the user name in the hash.

Usually, the inclusion of the user name in a password derivation scheme is meant to simplify security proofs; with the inclusion of the user identity, one can study the security of SRP without having to bother (much) with what happens when an attacker plays tricks between a server and several clients (e.g. tricking two users into talking to each other, each one believing that the other is the server). Protocols with more than two parties are exponentially more difficult to analyze; the user name as part of the hash makes the security of the whole thing simpler (which does not mean simple, only possible to study).

Therefore, implications of omitting the user name from the hash are, at best, not investigated. So, do not do it. This is also the general advice about cryptographic algorithms and protocols: do not fiddle with them, because security is a subtle thing and you cannot know, by yourself (even if you are a competent cryptographer), if you still have it or not after any minor modification (it takes a lot of competent cryptographers to know that).

For your specific problem, I would advise registering several "users" for all the identifiers that the user may wish to use, and logically link them to the same identity once the protocol has run.


export-secret-key after Yubikey is plugged in

After the private keys are on the Yubikey, they are not exportable.  What you can export are secret key stubs, which practically only say this key is on a smartcard.  They were the main method of making the key work on a different computer (with the smartcard), but these days, as there is sufficient information stored about the key, all you need is to use --card-status to fetch the same stub from the hardware key, and import the public key.

You can try to see what information is in the exported stub with

% gpg --list-packets --verbose <exported-stub.bin

# off=1796 ctb=9d tag=7 hlen=3 plen=294
:secret sub key packet:
    version 4, algo 1, created NNNNNNNNN, expires 0
    pkey[0]: [2048 bits]
    pkey[1]: [17 bits]
    gnu-divert-to-card S2K, algo: 0, simple checksum, hash: 0
    serial-number:  <ELIDED>
    keyid: <KEYID>
# off=2093 ctb=89 tag=2 hlen=3 plen=293
:signature packet: algo 1, keyid <KEYID>
    version 4, created NNNNNNNNNN, md5len 0, sigclass 0x18
    digest algo 8, begin of digest 7c 79
    hashed subpkt 2 len 4 (sig created <YYYY-MM-DD>)
    hashed subpkt 27 len 1 (key flags: 20)
    hashed subpkt 9 len 4 (key expires after <TIME>)
    subpkt 16 len 8 (issuer key ID <KEYID>)
    data: [2048 bits]

While it looks like the "secret key sub packet" does have a pkey, but the gnu-diver-to-card entry signifies that there's no secret key present. The signature is the signature of the key, which is to ensure it can be verified with the public key, and does not contain the private key.

Another hint that everything is safe that gpg does not ask for a passphrase to export secret keys in this case.

Update: confirmed via the source that pkey is indeed referring to the public key, not the private key - those are called skeys inside gnupg.  A packet containing the passphrase protected skey would look like:

off=557 ctb=9c tag=7 hlen=2 plen=134
:secret sub key packet:
    version 4, algo 22, created NNNNNNNNNN, expires 0
    pkey[0]: [80 bits] ed25519 (1.3.6.1.4.1.11591.15.1)
    pkey[1]: [263 bits]
    iter+salt S2K, algo: 7, SHA1 protection, hash: 2, salt: HHHHHHHHHHHHHHHH
    protect count: NNNNNNN (NNN)
    protect IV:  <IV>
 >> skey[2]: [v4 protected] <<
    keyid: <KEYID>

So in conclusion, the private key is safe, the cryptographic content is not exported the way shown above when using a smartcard such as the Yubikey.


How much security is compromised if we accept other characters as login (other than the original password)?

These rules basically mean that instead of having one valid password for an account, there are three of them (two, if the original password already begins with an uppercase letter). Theoretically, this divides attacker effort by up to 3, but actually a bit less than that because "pASSWORD" is much less probable as a user-chosen password than "Password".

So, in short words: these rules make password-guessing attacks three times easier. This does not mean that they make attacks easy.


It is about risks weighted against benefits.

Requiring PIN for NFC transactions would reduce main NFC advantage - speed.
NFC payments without PIN are used only for payments of limited amount and usually for limited number of transaction and/or limited total amount of transactions per day. Maybe also other bank specific rules apply; at least I was informed by my bank that I may be asked for PIN without reaching any limit. This way card issuing bank or card company limit its risk (in EU you are not liable in the case of fraud for more then 150 EUR as long as you haven’t acted with gross negligence) and keep you using the card often and making profit for them.


PCI Compliance relating to 'other' passwords

PCI-DSS requirement 8.4

  Render all passwords unreadable during transmission and storage on all
  system components using strong cryptography.

If it's strictly sending passwords for configuration files, you should send them in an encrypted form through email. The best way would be by using PGP/GPG.


If you know what you're looking for, then you simply examine the data within the file to see if it matches the pattern you're looking for. How you do that depends on what sort of pattern it is; you could view the files in a text editor (for text-based patterns) a hex-viewer (for binary patterns) or you could write (or otherwise acquire) a program that will tease these patterns out of the files and present you with a report.

On the other hand, if you don't know what you're looking for, then you're obviously not going to find it.


Is automatic login with full disk encryption a risk?

No risk at all. A login prompt is only for preventing physical access to the computer. This protection role has been replaced by the FDE and the accompanying PBA (which is much more secure), so any attacker that would break or somehow bypass the FDE could bypass your OS login too.

However, running as a limited user (with autologin on) and having a password-protected administrative account (with autologin off) can be useful, since the account then protects against online-based attacks (virus infections, malicious scripts etc) while the FDE protects against physical (offline-based) attacks.

But remember that anything that your currently logged on account has access to (regardless of password protection), can be accessed by a virus or malicious tool too, so a good idea here could be to also put sensitive data in separate encrypted containers with different passwords. So if you get a infection or a malicious script, then they only get access to whatever you are working with at the moment.

This would mean those containers would be encrypted 2 times on the disk platter, first by FDE and second by the container encryption.


How can an IPS\IDS detect SQLi \ XSS (or other application layer) attacks?

An IPS just does the same TCP reassembly as the stacks on either end.

Retransmissions aren't a problem. When a hole appears in traffic (because of a lost packet), the IPS forwards the packets after that hole, but also keeps a copy of them. Because it's forwarding those packets, there is no loss in user experience.

When the retransmitted packet fills the whole (completing the puzzle), the IPS then analyzes all those stored packets in order. It combines these into a single stream, so requests split across packets are no problem. If any of those packets trigger a signature, the IPS blocks the retransmitted packet, and kills the TCP connection with RST packets. This caused the target victim to then drop the fragments after the hole that it was buffering.

Thus, packets stream through the IPS with essentially zero latency, but at the same time, TCP is reassembled with no problem.

Over the last decade, hackers have developed methods to attack this system, such as retransmitting TCP segments with different data. Likewise, IPS vendors have developed defense, such as matching their reassembly algorithms with the target system so that it can choose the correct segment to analyze.

Also note that the IPS does full HTTP processing as well. There are likewise ways of corrupting HTTP requests in order to evade the IPS, which IPS vendors defend against.

The upshot is that an IPS introduces only about 100-microseconds (or 0.1 milliseconds) in latency to web requests, while still handling such things as TCP reassembly with no problem.