Root server hacked via httpd - consequences and future prevention

The attacker had access to all other virtualhosts (what you call "domains") because they all run under the www-data user, once he got this user's privileges he could access all the other domains.

To mitigate this you can use mod_privileges on Apache to run each virtualhost under a different user account, and make each virtualhost use a different PHP-FPM pool that also runs under that same different user. If an attacker manages to gain shell access under one's virtualhost account and attempts to ls someone else's virtualhost directory, he'll only get a Permission denied error (unless that "someone else" ran chmod -R 777 on his directory, but then he just deserves to be compromised).

The PHP open_basedir directive may also help but isn't bulletproof and will only protect against reading files using PHP's usual file I/O functions, but not by executing shell commands.

  How can I strictly prevent php from sending emails by mail()

PHP has a configuration directive to disable certain functions, so as a dumb solution you can use that, but it won't protect against using raw sockets to connect to a mail server and dump spam in there. The best solution is to block outgoing connections to port 25. Authenticated SMTP can use the submission port (587) so legitimate users will still be able to use their own mail provider's server or the server your provide, but won't be able to scan for open relays on port 25 and try to send their spam.

  Is it possible to completely encapsulate the website of my friend

You can put each server into a chroot jail and running on a different port, and have a lightweight web server in front of them acting as a proxy to redirect connections from the standard HTTP(s) port to the correct chrooted server based on the domain name. You can go even further by using LXC (containers) or entire virtual machines/physical servers for this, depending on your security requirements.

  What is the ISP way of restricting customers' scripts and email sending behavior?

Some (bad) ISPs just block port 25 to anything but their own servers (from where they can track the user even if he doesn't authenticate because they know which IP belongs to who), some let you use the port freely and I assume just terminate your account if they get too many complaints about spam from your IP.

From reading OVH's (a popular french ISP) terms and conditions they appear to monitor your outgoing SMTP traffic and pass every main through their Spamassasin-based filter, and if too many mails are considered as spam they block your port 25.


Pre-shared key authentication between webservers for some security

There are several ways to do this, depending upon how complex you want to get. For example, you could go all the way of implementing an OAuth2 service and having your servers authenticate with each other, but that would probably be overkill. Given the description you gave of your situation, something simple would probably work well, in my opinion.

Essentially, what you need is a way of modifying the request, so that each server can be reasonably certain that the connection coming in is from a trusted/authorized source: your other server(s). There are many ways to do this. One straight forward way would be to simply inspect all connections and only accept connections from a given list of IP addresses, but this doesn't provide any authentication guarantee (anyone on that IP is given access), and is also vulnerable to IP Spoofing, but depending upon how much you care about things like that, that might be enough for you.

One way of doing the shared key thing you asked about is just to have a big random string/number (essentially acts as a token/API Key), and add it to all requests as a header (or URL parameter), and the receiving server can just extract the header and do a basic check on the string to see if the string is an exact match. If the "password" header is exactly the same as the password on the receiving server, it assumes that the request is valid, and processes the request. If not, it simply aborts and drops the request. However, if someone intercepts the request and manages to find your password, they can submit their own requests by simply passing the same password. I don't think you mentioned what language/frameworks you are using, but if you're using Java & the Spring framework, you can create Interceptors & Security Filters which automatically catch all requests as they come into the server before they're processed, which allows you to handle things like this, so you don't have to do it over and over again in every controller mapping. Other frameworks have similar features, but I have most experience with Spring.

If you wanted to get a little more fancy, you could go with a JSON Web Token (JWT) using a shared key, and append it to each request in the header. Then, just like before, you can inspect the request headers to extract the Token, validate it, and then process the request. JWT's have the added benefit of built-in validation & tamper protection, as well as allowing custom data fields to the token, so you could start to add more information to the token, like timestamps, types of requests, levels of clearance, user ids, etc. and they would all be encapsulated in the token, and protected by it's validation & tamper protection. If you're using Java, there is a pre-built, very easy to use JWT library called JJWT (i.e. Java JWT). Here is a video on it:

Starts @31:36

https://www.youtube.com/watch?v=sv0TUiYVimw

Here's the maven Dependency:

<dependency>
    <groupId>io.jsonwebtoken</groupId>
    <artifactId>jjwt</artifactId>
    <version>0.7.0</version>
</dependency>

As for the rolling (i.e. constantly variable) password, you could do that pretty easily. Something as simple as Encrypting a timestamp with a shared key, and sending it in a header on the request (or adding it to the JWT). Then the receiving server would just decrypt the header and check the timestamp. If the timestamp is older than 1-5 seconds (depending upon the latency between the servers and variance in clock times) the receiving server just aborts & drops the request; you could respond with an HTTP status code of "Unauthorized". If the server tries to decrypt the timestamp but fails due to corrupt data (or someone trying to spoof the header) or if the header is empty, the server should also abort, and drop the request.

Not to sound like a broken record, but if you're looking for a simple encryption API, Spring also has an easy one in their Crypto package, but it defaults to AES-256, which requires the unlimited strength encryption Policy files installed on your JRE. They're free & easy to install (just download from oracle, and paste them into the right directory), but it's something to be aware of. Without the policy files, it will throw an error: "Invalid key length" or something like that. It just means that it's trying to use encryption that's too strong for what the JRE allows; it has to do with US export control laws. But the short of it is that any American Citizen can download them for free.

What you're trying to do basically sounds like a small scale micro-service. You might be interested in these talks:

https://www.youtube.com/watch?v=ZyK5QrKCbwM

https://www.youtube.com/watch?v=rqQOSG0DWPY

https://www.youtube.com/watch?v=sbPSjI4tt10

https://www.youtube.com/watch?v=saiwZzE5IYg

Hope this helps.

-Yurelle

Edit: Also, you mentioned that one of your main concerns was the performance issues with people spamming your API. You're essentially referring to a small-scale Denial of Service attack. From your minimal concern with security, I get the impression that the project you're working on is somewhat small scale and you're not expecting to be the target of any serious attacks; and that it's more of a Better-Safe-Than-Sorry precaution, which is always good, I'm certainly not faulting you. However, if you ever do expand into the territory of being in serious danger of such attacks, there is one type of attack that many people often forget about; It's called a "Slow Loris Attack". It's very simple to pull off, and it's extremely effective given the resources necessary to run such an attack, and, to my knowledge, the only reliable defense is a non-blocking web server framework (often called "reactive" or "event-based"/"event-driven" frameworks). Again, probably not something you need to worry about, but since you brought it up, I thought I'd mention it.

Here's a video on it, if you're interested:

https://www.youtube.com/watch?v=XiFkyR35v2Y


WordPress site is infected with malware, cannot find dropper

If I understand what you mean by object name, most AVs use an object name that refers to a "class" of viruses. It may not be that the attack is specifically in revslider since you don't have it installed, but perhaps it was found using the same heuristic as is used to find the revslider virus.

I'd start by comparing the source code of the site against clean copies of wordpress and any installed plugins. That would be the quickest way to sniff out any modified code.  If that doesn't work check the database data.

Wordpress stores posts and pages in the database. These are by design allowed to hold arbitrary html/javascript. If an attacker got into the system they could have altered pages in the DB to include malicious scripting.


Does DH parameter file need to be unique per private key?

TL;DR Yes, parameters (not keys) can be shared.

First, don't say "parameter keys". Parameters and keys are related but different. The whole point of parameters is that they define a mathematical group within which keys are chosen and used. Generating parameters is costly (moderately so for DH and DSA, much more so for ECC); generating a key for given parameters is much cheaper. Further, a main point of DHE ciphersuites is that your SSL implementation generates a new ephemeral key for every connection. If you are running just one server with a fixed set of DH parameters, and that server handles more than one connection from one client using DHE, you are already using multiple keys with the same parameters. And similarly for ECDHE, but more on that below.

There is no harm (added) to connection security in using the same parameters for multiple servers as long as the parameters are not chosen or influenced by an attacker or otherwise grossly bad -- in which case they should not be used for even one server. The only case I see for multiple parameters is if you have some (possibly virtual) servers which must support (typically older) clients which can only handle DH 1024-bit, which now has a fairly small safety margin, while other servers want the greater margin -- and/or compliance with standards such as NIST 800-57 if applicable -- of 2048-bit or more.

(Added) One caveat: if you are trying to conceal that servers are under common administration, using shared but otherwise unique parameters gives a clue that their admins might be the same -- but not a strong one, because it could be they are simply using the same software. For example, Apache for a long time had a builtin default set of DHE parameters, and before Snowden and Logjam almost everybody used that default without thinking about it, so all those servers used the same parameters (when they used DHE at all) even though most of them were wholly unrelated and separate. (Similarly, researchers who have scanned the net for common RSA moduli or factors -- which is a security weakness, and a very bad one -- have tracked some of them down to particular hardware or software products, with no other commonality.)

For ECDHE (and ECDHA and ECDH and ECDSA, although those weren't your question) there are standard parameter sets -- commonly called named curves -- that are much easier to use and also more widely/reliably interoperable, so you should not even try to generate your own. As of now it appears best to always use P-256 or maybe P-384, the two standard curves (out of dozens) "blessed" by NSA in its Suite B profile.

See pretty-much-duplicates:

What are the implications of using the same DH parameters in a TLS server?
Can someone explain a little better what exactly is accomplished by generation of DH parameters?
How often should the DHE p and g factors be recycled?

UPDATE: since this has been necroed, I'll add that (for TLS) rfc7919 in 2016 recommends use of standardized parameters (the same for all users, chosen by a variant of the 'nothing up my sleeve' method from IKE rfc2412 & rfc3526), and TLS 1.3 rfc8446 in 2018 requires standardized groups for both classic/integer/Zp/modp/FFC DHE and ECDHE, dropping the 'explicit' EC option in 4492 that no one used anyway.
(And 1.3 generally requires [EC]DHE, because it dropped the plain-RSA and static-[EC]DH forms of keyexchange, though there is still PSK for the few cases that is manageable.)


Where can I practice web-security or network-security?

I would recommend that you have a look at some of the wargames out there. For example one of my personal favourites is Over the Wire where they provide several games of varying difficulty. Certainly makes for a much more entertaining and thrilling learning experience.

Another one more specifically targeting web-hacking is Hack This Site split into varying levels of difficulty.


Generate digital certificates for employees using the organization's certificate

No, you can't use your purchased signing certificate to sign other certificates. It is not allowed. Although, you can try, but signature will be untrusted. The ability to sign other certificates is governed by Basic Constratins and Key Usage certificate extensions and values of these extensions in your certificate don't meet criteria. For more details about Basic Constraints in action you can check my blog post: Basic Constraints certificate extension.


Run them all through a converter into PDF/A format.

It forbids any sort of active content.

If they won't convert, trash them.

  IDS/IPS with Anti-Malware license

Scanning is a losing battle. No matter what your scanner knows to look for, there will always be something new that it doesn't know to look for.

  dedicated sandboxed cloud environment

That's useful for generic malware, but not useful for targeted malware that will only execute it's payload under conditions that only exist at it's target.

If you want to make sure the file won't do anything Evil when opened, make sure it doesn't contain anything executable. You do this by running the conversion to non-executable PDF/A yourself. This creates a new, clean PDF file that by definition isn't infected because you created it.


There is no single way to mitigate time spoofing attacks. However, getting time information from several sources and ensuring none of the sources too strongly disagree with each other is a start.
NTP
The Network Time Protocol, with the common version 3 specified in RFC 1305, is the most widely used and most well known method of getting time. It is an unauthenticated, unencrypted protocol which uses UDP, making spoofing and tampering quite easy. Typically, a user will poll the time from a pool of stratum 3 servers. Stratum refers to the number of servers the time has to go through after leaving the source, usually a machine which measures the state transitions of an atom. A single second is defined precisely as 9,192,631,770 transitions of the hyperfine ground state of a single cesium-133 atom at near absolute zero, and the radiation emissions that correspond to these state changes are what gives the reference clock its accuracy. There are four commonly used strata:

Stratum 3 (pool) servers are used by end-users and are accurate to within a fraction of a second.
Stratum 2 (secondary) servers are used only by applications that require higher accuracy.
Stratum 1 (primary) servers attach directly to the source, and are accurate to the microsecond.
Stratum 0 (reference) servers are the atomic clocks themselves, and are perfectly accurate.

Each stratum connects to lower strata, in order to avoid overloading the services closer to the source. Stratum 0 is not a regular networked server, and is connected directly to stratum 1 (e.g. through RS-232). Only select services that require extreme accuracy (for scientific experiments, for example) are allowed to connect to stratum 1. Stratum 2 servers may have specific policies for who can connect, while sometimes they may just politely ask you to not use them if not necessary. Stratum 3 servers can be used by anyone. There can be up to 15 strata, with the 16th meaning unsynchronized, but in practice, stratum 3 is the last. Every stratum other than stratum 0 is capable of synchronizing itself with the other strata of the same level to improve accuracy and reduce the issue of clock skew.

That's a lot of servers to trust. While a server that majorly disagrees with the others will be kicked out, that only happens if it occurs often enough. A targeted attack against a single user or a group of users can easily go unnoticed. Your NTP client may average out the result of the time given by all members of the pool you are using, or it may ignore the ones which seem insane. For this case, a single malicious NTP server is not a huge issue. However, as NTP is unauthenticated and unencrypted, it is not difficult for a single entity to give you a false value from all the servers you are using.
There is no way to avoid this issue completely. Using multiple NTP servers helps against naïve attacks and accidental misconfiguration. Routing NTP through a VPS or VPN may also raise the bar slightly, especially if you suspect that your adversary is not willing to hijack NTP for everyone and is unable to figure out which server is connecting to NTP on your behalf. There is a type of authenticated NTP defined in RFC 5906 for NTPv4, but very few NTP servers support it, and it is not simple to configure.
GPS
The way GPS works is by transmitting a periodic, perfectly synchronized signal (each GPS satellite has three stratum 1 sources). When one signal is received earlier or later than another, you will know that you are closer or farther from that satellite than from the other (because the speed of light is finite). By taking into account these relative time differences from multiple sources, it becomes possible to tell your position in relation to the satellites in orbit, and thus in relation to the Earth.

A GPS clock works by combining the time given by multiple orbital satellites. This time signal can be used to synchronize a clock, but unless you know your precise location, it can take some time of averaging the signals to get accurate time. A high-end, dedicated GPS-based clock can, when calibrated properly, be accurate to within a microsecond.
GPS spoofing is not particularly difficult, but is easy to detect. The GPS signal is, like NTP, unauthenticated and unencrypted. Three is no simple way to prevent this.
RCC
A Radio-Controlled Clock, or RCC, transmits a time code from a radio transmitter which is connected directly to a highly accurate clock source, such as a stratum 1 server. The frequencies used can be either longwave or shortwave, and vary between 25 KHz and 25 MHz. Devices marketed as having "atomic clocks" typically use RCC, as it is simple and requires little power to receive. They are highly accurate, providing precision within several microseconds.
Like GPS, they are highly vulnerable to spoofing.
TLS
TLS is Transport Layer Security, used for encrypting and authenticating communications over TCP.

This is an interesting way to keep time. The TLS 1.2 specification provides a field in the ClientHello requests which contains the current time, in seconds since the epoch. This is not always reliable, as many servers either have inaccurate or purely random values for the timestamp. TLS 1.3 plans to do away with this timestamp entirely, so this is not a long-term solution. From section 7.4.1.2 of the RFC:
Structure of this message:

The ClientHello message includes a random structure, which is used
later in the protocol.

   struct {
       uint32 gmt_unix_time;
       opaque random_bytes[28];
   } Random;

gmt_unix_time
   The current time and date in standard UNIX 32-bit format
   (seconds since the midnight starting Jan 1, 1970, UTC, ignoring
   leap seconds) according to the sender's internal clock.  Clocks
   are not required to be set correctly by the basic TLS protocol;
   higher-level or application protocols may define additional
   requirements.  Note that, for historical reasons, the data
   element is named using GMT, the predecessor of the current
   worldwide time base, UTC.

random_bytes
   28 bytes generated by a secure random number generator.

Because the first four bytes of the "random" field sent by the server contain the current time, a TLS handshake can be (ab)used for obtaining time from an authenticated source. This is what is done by the tlsdate program. In order to obtain accurate time, one can connect to multiple TLS servers and average the time. This is used by Tails. Tails has hardcoded three "pools" of TLS servers:

Pal, which contains websites that promote privacy like the EFF.
Neutral, which contains websites that are neither extra nice nor malicious, like Stack Exchange.
Foe, which contains websites which tend to be seen as privacy-invading, like Microsoft.

Tails queries one server from each pool and averages the results out, bailing if any one of them disagrees too strongly. The idea is that the pools have different enough affiliations that they are unlikely to collaborate in providing maliciously incorrect time values in their TLS timestamps. The specific reasoning for this setup is unknown (and seems a bit like homebrew security), but the idea itself of extracting time from the TLS handshake is fairly solid.
RTC
The Real-Time Clock, RTC, is a component in any computer as part of the CMOS chip. It usually uses a quartz oscillator at 32,768 Hz for timekeeping and is typically powered by a button cell battery. Unlike a cesium atomic clock whose frequency is determined by fundamental constants in nature, the quartz oscillator is intentionally tuned to that frequency, resulting in slight imperfections. I actually didn't know the reason it was tuned to this specific frequency (it seemed too perfect, being exactly the maximum value of a signed 16-bit integer), but after looking it up when writing this answer, it turned out this value was chosen because it was always divisible by two, making it perfect for conversion into the 1 Hz signal required to control a digital watch. The frequency just stuck.
This is actually the biggest thing you have going for you. Your RTC is not perfectly accurate, but it cannot be directly controlled by an attacker (though there are indirect ways to influence it, such as strong electromagnetic fields or ambient heat). As long as it's set correctly at least once, you can always be sure to detect significant clock skew attacks, as your RTC will never drift too far.
Putting it all together
These are quite a lot of sources, each with their own benefits and drawbacks. It turns out there is an easy way to combine them all. The OpenNTPD project, designed by the same folks behind OpenBSD, is capable of combining all of these time sources at once. It is primarily an NTP server, obtaining the time from the NTP pool given to it and using that to set the system time.
OpenNTPD also supports Authenticated TLS constraints, where the timestamp obtained from a TLS handshake is used to ensure the NTP responses are not too inaccurate. It is not designed to be used as the primary time source. This is similar to tlsdate, but is not affected by the potential inaccuracy of timestamps in TLS handshakes. Hardware time sources can also be used, such as GPS or RCC. This, of course, requires the actual hardware, but it can provide much greater accuracy.
Unlike many other NTP implementations, OpenNTPD does not set the time immediately when it changes. Rather, if the time jumps forward or back, it will gradually change your system time, skewing it until it matches its view of the current time. For example, rather than instantly setting the time 5 seconds into the future or past, it will speed up or slow down the clock frequency, and push it forward or back in very small increments. While this is designed to avoid confusing applications that rely on relative timestamps, it is also useful to slow down attackers who want to immediately set your clock to the point where an expired and compromised certificate is still valid.
There have been criticisms (and responses to said criticism) of OpenNTPD. The simplicity of the project means it lacks several sanity checks and algorithms that improve time accuracy, leading to at best 200ms accuracy. This is not an issue for security, but it may not be ideal for certain applications. For the average computer user, it is more than enough.


A smart card is supposed to resist physical tampering.

Heavy hardware such as HSM and payment terminals use active protections: basically, their "secrets" (e.g. private keys) are stored in a SRAM module, with a battery, and they are never completely "off". If they detect a physical breach (there are many sensors inside the box), they proactively destroy their secrets.

Smart cards don't have room for such a battery, so their protection is mostly passive. However, some also keep their keys encrypted, using the PIN code as key. This is an extra layer which tries to thwart physical attacker who opened the case and are ready to plunder the EEPROM/Flash in which the card keys are stored. However, user PIN codes are rarely strong enough to make this encryption much of an obstacle.

For smart cards which use PIN-based encryption in addition to the physical shielding, the PIN entropy can matter (not the length, at least not directly -- secrecy of the PIN depends on how random it is, not on how long it is), but only if that entropy is high enough to be non-negligible. Think "passphrase" here, not PIN.

Smart cards achieve a resistance up to a (rumoured) breach cost of about 5000$, for a 5$ hardware cost. That's a 1000x factor, which is not bad when you think about it. I am not aware of tokens which do better. If you search one (there are many products on the market), look for active protections (with an inner battery) and certifications like "FIPS 140-2 Level 3" or "EAL4+" (certifications don't guarantee robustness, but they signal that the vendor, at least, put significant money on the subject).


Which services are affected by Heartbleed?

It is hard to say exactly which apps/services are affected. This is because OpenSSL is a collection of programming code (referred to as a "library") that can be used to add TLS support to an application or system. TLS (Transport Layer Security) provides secure connections, and is best known for being the security layer behind HTTPS websites.

So if a programmer were writing a program that needed to use TLS to connect to something, they can use the OpenSSL library to add that ability to their app.

The OpenSSL library itself is constantly being improved, like many other bits of software. During this process, the Heartbleed bug was accidentally introduced in OpenSSL version 1.0.1, which was released on 14th of March 2012. It remained present through to version 1.0.1f (inclusive) and was fixed in 1.0.1g, released on 7th of April 2014  .

This means that any application that uses those OpenSSL versions for TLS is potentially affected. No doubt the affected developers have fixes in progress.

The fix has since been "backported", meaning that it has been added to versions of OpenSSL prior to 1.0.1g. This is a good thing, and is commonly done for vulnerabilities, but has the side effect of making it harder to tell if an app is vulnerable (since you can't tell just by looking at the OpenSSL version).

To address your specific questions:

SSH is not affected (SSH is a different protocol to TLS)
HTTP is not affected (HTTP is also a different protocol to TLS), meaning that a HTTP-only server will not be affected.
Note that it's possible to provide HTTPS using other libraries - so Microsoft IIS Web Servers (which don't use OpenSSL) can provide HTTPS without being affected.

So in summary:

The only apps/services that are affected are those that use a vulnerable version of OpenSSL for TLS connections, and have TLS heartbeat support.

Other TLS libraries (like GnuTLS, SChannel, and JSSE) cannot possibly be affected by this particular bug, because it only exists in specific versions of the OpenSSL library.
If you are unsure, ask the person/company that wrote the application.
If you are a developer, find out what library your app is using for TLS connections and test to be certain.


What kind of damage does the HummingBad Android malware do when it fails to gain root access?

In the case of HummingBad, if rooting fails, the next step is an attempt to socially engineer the user into giving it the required access by posing as an Android update.


How to encrypt entire HDD without any password prompts?

The key has to be somewhere other than on the encrypted drive, because logic.

Typically the key is stored in one of three places:

In the user's head (or, since keys are hard to store in a head, a password is stored that can be used to derive the key)
in a TPM module inside the computer
on a removable token, such as a smart card or USB fob

From your description you want option 3, a removable token that you leave with the machine except when you send it off for repair.

However, you might have a bit of trouble finding a product that suits, as yours is a very unusual use case. Lots of software supports removable tokens, but typically they only use it with a password, either to

provide a backup of a password-derived key in case the user forgets the password
provide two-factor authentication, where you need the token in conjunction with a password.


How does the meet in the middle attack work on double DES?

There is a concept of effective/actual key strength. For Double DES the effective key strength is 2^57 even though double DES uses 2^112 keys. The below example will make it clear.

Assume that you are a cryptanalyst who has access to the plain text and encrypted text. Your aim is to recover the secret key. Assume AAA (plaintext) -> XXX (After 1st encryption) -> ZZZ (after 2nd encryption).

You start with AAA and try all the 2^56 combinations for secret key by encrypting AAA. This will give you a big list of possible values for XXX.
Next you take ZZZ and try all the 2^56 combinations for secret key by decrypting ZZZ. This will give you a big list of possible values for XXX.

The amount of effort you have put in 2^56 + 2^56 = 2^57.

Now do a simple lookup between the two lists to find a matching value. As soon you see a matching value XXX in both the lists, you have found out the secret key. So this means that with effort of 2^57 keys you have broken the encryption.


Up to date books for OpenSSL coverage of TLS 1.1 and 1.2

I think what you might be looking for is something like Bulletproof SSL TLS and PKI
 which is so new it's not even out yet!


sudo restrictions circumvention with pkexec: root shell

a big security hole: a user that is part of the 'sudo' group can always gain root access, and start a shell as the root user. That is, at least on a default Ubuntu server installation with sudo.
This renders sudo useless as a means to restrict elevated privileges to certain commands for selected users.

As far as I know that is no actual security problem, except maybe in your expectations. Your conclusion is completely unwarranted.

Since Ubuntu 12.04  administrator access is  granted via the "sudo" group.

Administrators need full and unrestricted superuser access to be able to perform their tasks.

This policy is consistently implemented in the default policies shipped by Ubuntu for both sudo and polkit, which allow all sudo group members to run both commandline and graphical applications as root.

(You then modified /etc/sudoers to remove the default policy for sudo group members but didn't modify the polkit policy. That led you to the mistaken conclusion that a member of the sudo group looks like unprivileged user, when they are (still) administrators with respect to other authentication systems. )
The solution is of course also obvious:

Don't make users a member of the sudo group when you don't want to grant them full superuser access.

For limited superuser access you will have to create your own policies in either sudo, polkit or both. You should create specific policies for particular users, or create new groups of your own for group based access controls.


No. The number of pages Google (and any major search bot) is willing to crawl on your domain (or indeed whether they are willing to crawl it at all) is based on how relevant they think your domain is.

There are plenty of sites with an infinite number of pages. This problem was solved decades ago.


Is an LDAP injection possible for a basic search query?

It's not the queries and filters you should be escaping – after all the complete queries and filters do have special characters in them and escaping them would cause the query or filter to break. Instead, you should sanitize the data coming directly from the user, here the username.

As you don't specify the programming language, it's hard to give examples on how this should be done, but if the language has some LDAP encoding functions, you should use them instead of implementing your own.

Although the distinguished name DN has less attack surface than the filters, it still has some special characters: according to OWASP LDAP Injection Prevention Cheat Sheet:

  There are certain characters that are considered special characters in
  a DN.

  The exhaustive list is the following: \ # + < > , ; " = and leading or
  trailing spaces.

  Some "special" characters that are allowed in Distinguished Names and
  do not need to be escaped include:

* ( ) . & - _ [ ] ` ~ | @ $ % ^ ? : { } ! '

E.g. with your "cn=" + username + ", dc=example, dc=com" one could descend
to organization units below the intended OU using "username" aiyan, ou=Administrators, ou=Secret, which would then find the user in

cn=aiyan, ou=Administrators, ou=Secret, dc=example, dc=com


Clickjacking: can't browsers just forbid/reject placing transparent elements over frames?

Banning opacity on elements containing an iframe would have negative impact of user interfaces (eg no nice fade effects when iframes are in play), but would not block clickjacking attacks.

Even if transparent iframes were impossible, you could still achieve all the same attacks by using a 1x1 pixel iframe that followed the pointer. Positioning inside that pixel can be achieved by scrolling a doubly-nested iframe. The potential one pixel difference underneath the pointer at rest would almost certainly be undetectable to the user.

So, yes, a browser could block it and temporarily be protected against a common form of attack. But as soon as a significant number of users have browsers that do that, the attacker will just change implementation technique. This is an example of a ‘fix’ that only works by making someone else the low-hanging-fruit; once everyone starts doing it, it is no solution at all. Consequently it makes sense to implement for a browser extension that not many people use, but no sense for a popular browser.


Enforce a secured protocol over public api

You have the control over what you are serving to the client.
This means that if you don't trust your vendor who provides the client then you face the fact that the vendor can see the data your API is transmitting for the client.
The vendor can do anything with that data.
In order to mitigate that, you can do the following:

Use a legal contract that covers this concern, you can't control the vendor client
Encrypt the payload with a key that the only user knows, but that is probably unlikely that the user will have to decrypt/encrypt the traffic, bad user experience.
Take control over the client or at least audit the code per version

As an API, you can only guarantee how the data and which data is delivered, you can't control over the processing of that data on the client side.


The private key that you store was encrypted by the TPM itself before it left the TPM, and can be decrypted only using the TPM's primary key (which only exists inside the TPM). The encrypted key you have is useless without the TPM, it cannot be compromised without access to the TPM. You can store it however you want.

Note that if you're keeping the key away from someone who has access to the TPM (can execute code/commands on the computer that communicate with the TPM, has access to the primary), this is another question and has nothing to do with TPM.

You do need to make sure you don't lose it, because the TPM cannot recreate it - you have the only copy. So multiple copies and backups might be a good idea.

The key should be RSA 2048 (or 3072 or 4096) or ECDSA NIST P-256. Unless your TPM has the ROCA vulnerability (or something like it), it is impossible to brute force on a classical computer using all the money in the world. Note: not "expensive" but "impossible". On hardware we can understand, breaking those keys will require some breathtaking new math, not brute force.

If in the future a large quantum computer will be built, it will be able to brute force the key, but first the owners of this marvelous device will go after higher profile targets and you will hear about it on TV. Nobody knows when such a device will be built, and there is scientific debate whether it will ever get built.


The visual fingerprint was introduced in OpenSSH version 5.1. The basic idea behind this feature is, firstly, humans are unreliable in comparing meaningless strings and secondly, it is easier to remember visual patterns.

Explanation of OpenSSH algorithm is given by Dirk Loss, Tobias Limmer, and Alexander von Gernler in "The drunken bishop: An analysis of the OpenSSH fingerprint visualization algorithm".

The approach is based on Hash Visualization: a new technique to improve real-world security.

If you are looking for a simpler explanation then this blog could be helpful.


Can the creator of Bitly links be determined from the link?

Bitly does not seem to be telling us how they generate the URLs, so we can not know for sure. However, they are so short that I doubt that they contain any information - a lookup in the database would be required everytime anyway to get a full URL from just seven characters. So probably it is just a random ID.

However, I would assume Bitly still knows what account generated that random ID, either from actively storing it in their database, or from just reading through the logs. If Bitly owns the data, they can also loose the data, be it from a data breach or because the US government sends them a letter demanding access.

So I don't think a normal user could figure it out, but I still would not consider it a secret.


Does ssh-keyscan tell you if a server supports RSA or DSA keys?

There are details... but, basically, early in the connection establishment, both client and server send to each other the list of algorithms (and thus key types) that they support. See section 7.1 of RFC 4253.

Try ssh -vvv the.ssh.server.name. The debug messages will show you a lot of information of what the server returned, including the full lists of algorithms.


As far as you're letting on, this isn't an "unrestricted file upload" vulnerability. "Unrestricted file upload" means uploading the file to their domain/server, not yours. SVG images sourced this way (via images) don't normally execute the javascript they contain. If you manage to upload an .svg file to their domain, and it's accessible via a URL, then you'll have a stored XSS vulnerability if you can get someone to visit the URL directly.
PHP files accessed on your domain would only ever execute code on the server they were hosted on, referenced inside an image tag or not. I think you're confusing sourcing a document as an image inside HTML with PHP file inclusion, wherein the server's PHP source specifies an attacker-controlled file for evaluation. Web servers don't go grab image links specified inside the HTML document they're responding with and interpret them as server side PHP code.


What problem does ECC seek to fix compared to RSA?

The main "problem" that ECC tries to fix is that RSA is RSA. Namely, people want to have a backup kind of algorithms, in case RSA gets brutally broken through a stupendous (and unpredictable) advance in integer factorization techniques. Integer factorization has been heavily studied for more than 2500 years (it was already all the rage in the Neo-Babylonian empire), and we can therefore say that ground-breaking advances in that field are not obvious, and don't happen often. However, this is mathematics: progress comes from ideas, which may occur without warning in the brain of people living in remote places that you have never heard of. Though such a break is improbable, there will be no prior warning. Hence ECC.

For the "backup algorithm" strategy to work, the ECC-based algorithm must be deployed wherever RSA is used. It takes a lot of time from the scientific article to code which runs and interoperates; if ECC is supposed to be able to replace RSA at a moment's notice, then it must be already there.

Apart from that point (often described as "a need for algorithm biodiversity"), ECC offers interesting performance-related advantages: smaller messages (an RSA-based key exchange with a 2048-bit RSA key requires 256 bytes, while the ECDH key exchange of similar strength may fit in 28 bytes), faster private key operations (by a factor of 10 or more)... Smart card vendors are much interested.

Furthermore, some very specific types of elliptic curves allow for efficient pairing computations, which open up the practical possibility of three-actor protocols. This is used, for instance, to implement identity-based encryption: you cannot do that with RSA.


Am I correct in having the unique service account per integration stance?

This is something I always recommend doing because you can set permissions per account. The least privilege principle should always be applied.

As you mentioned yourself, it is easier to perform audits when having separated accounts.

Additionally, service account should not be able to be used in order to login to a system and either get a shell or remote desktop connection (depending on the environment).


I could be wrong, but I'm fairly certain that your main problem is that you are attempting to do mysql injection wrong.  As a general rule of thumb, if a payload can trigger an SQL syntax error then your attempts at preventing SQL Injection are not really working.  For instance, if you were using prepared queries, there is no way for the payload to result in an SQL syntax error at all.

In this particular case the problem is that you are using the wrong kind of comment to hide the rest of the command.  In this case you would probably have more luck (making SQL Injection happen) if you used a double dash (--) or pound sign (#) which both start single-line comments in MariaDB.  The comment character you are using (/*) is a multi-line comment character that has no closing comment character.  This appears to be the source of the SQL error.  While I'm not personally familiar with sqlmap, my suspicion is that it may not think there is an injection vulnerability, not because you are safe, but simply because your injection payload is generating an SQL error, and therefore no injection is happening.  Try again with the proper comment character.

To reiterate, the fact that you getting an SQL error makes me think that you are vulnerable to SQL injection, you just aren't doing the SQL injection properly.

Edit

I think your next step is to ignore sqlmap for a bit.  The attack you are doing ' OR 1=1 would select all records.  Your URL implies that this is a view kind of page, which shows only one record at a time.  As a result, there are many cases where such an SQL injection payload may not actually have any results on the page in question.  This is the part that can make it tricky: SQL injection is only "successful" if you can actually do something with the attack and impact the application in a meaningful way.  So for instance if your payload causes it to load all records in the SQL command, but the page in question implicitly shows only the first result (because it is a view-type page), then you won't realize that your injection has been successful.

The short of it is that your given SQL injection payload is context-specific, and may not work in all places on a given website even if the page is vulnerable to SQL injection.  That makes SQL injection extra tricky.  Some sql injection tools can automate some of that trickiness, but I don't know if sqlmap is such a tool.

So ditch sqlmap for a moment.  Try a different kind of payload.  Instead of selecting ' OR 1=1 try doing ' AND 1=0.  This should (theoretically) result in zero records.  So if you get a 404 (or nor results in general), I would consider that evidence that the page is vulnerable to SQL injection.

Of course, if you're feeling mean you could just try variants on '; DROP TABLE users-- until something breaks :)  That would be mean though.


Question about IPv6, NAT, firewall, port forwarding, upnp and security

With IPv6 you cannot rely on NAT, and thus, yes, you have to configure reasonable packet filtering rules on the router or firewall. For stateful filters this is easy to accomplish: Allow all outgoing traffic and allow all incoming traffic that is related to the outgoing traffic, drop anything else. For stateless filters there's a bit more to do, in particular, you have to bear in mind that IPv6 requires several ICMPv6 packet types for its operation (mostly error messages; for instance, "packet too big" messages since with IPv6 routers don't fragment as they do with IP legacy).

Without NAT you don't need port forwarding, you simply add the according filter rule for incoming traffic.


Inbound Ip Visited "localhost" wordpress stack on AWS

While monitoring inbound connections with wordfence i noticed someone visited http://localhost ...

It is not fully clear for me what you really saw but I guess that somebody checked if your web server could be used as a proxy to access an internal host. A properly configured web server will not allow this and in this case you could ignore this message as the usual "noise" you get if you connect a system to the internet.

If instead your server would allow such a request then it could be used to bypass existing boundaries: a web server at localhost (or in some internal network) is often considered protected from outside access or authentication is omitted if the access comes from an internal IP address. If the attacker manages to use your web server as a proxy to access internal systems or localhost then the request to the internal server will come from the internal address of your machine (or from localhost when accessing localhost) - and thus from a trusted system. This way the attacker can access internal sites which should not be reachable from outside.

For more information see for example Server Technologies - Reverse Proxy Bypass.


Have you looked at rekall?

It is a fork of volatility, while I have had better luck with volatility it might be faster for the initial analysis


How to harden an iPhone/Android app so it's tough to reverse-engineer it?

Basically the objective '1' will not work; reverse engineering just works too well. It will be especially easy with the Android version, because Java is really simple to decompile (on Android, a specific bytecode format is used because the Java implementation is Dalvik, not the "true" Java, but the principle remains). However, it will not be that hard with the iPhone version either (Objective-C compiled to ARM code, not the hardest thing ever to disassemble and figure out).

Objective '2' will not work either. First of all, reverse engineering and data extraction are passive only, so the application is unaltered -- therefore, there is nothing which the application can do to detect that such reverse engineering took place. Also, it is easy to modify some application code to bypass any internal test; that's the number 1 cracking technique employed to work around copy protection of games since the early 1980s (at least).

The best you can hope for, is to split users into two categories: those who have jailbroken their phone, and those who did not. On an non-jailbroken iPhone, only apps dutifully approved by Apple can be installed and executed, so this prevents installation of a modified, non-official app which would use your "secret tokens" in a way you do not want. But jailbreaking is exactly what people do to install unapproved apps, and it seems to be relatively easy.

You can also try to make life harder for attackers by renewing your secret tokens very often, and constantly publishing new versions of your app. That's what game console vendors do, with constant new "firmware versions" that users must download to be able to play with the newer games, view the newer movies, or connect to a gamer network. This annoys users, too, so that's not exactly cheap to do.

Things would be much easier for you if you could arrange for the "secret tokens" to be user-specific -- this way, if a token value is unduly extracted, you can still blacklist it on your server (I assume that your app uses the tokens to connect to a server somewhere).

Note that keeping data secret in a deployed application, or, failing that, reliably detecting fraudulent usage of said data, is the Holy Grail of all game editors and movie publishers around the World. Last I heard, the big players like Sony and Warner have not found it yet, and that's not for lack of trying.


Should it considered a risk if an organization's server OS and version information is leaked?

In my opinion it is an indirect threat that is obtained during the first step in an attack, information gathering.

Know what you're dealing with, from an attacker's perspective, helps you prepare a successful attack. The more information obtained during the information gathering process, the more likely it is for an attack to succeed.

It is quit hard, especially for larger organisations, to avoid disclosing this type of information. An example is the human resource department that is instructed to hire technical people. Searching for job opportunities at your target company generally reveals specific details about the infrastructure or used operating system(s).

Having said that, it is import to properly secure your network (network segregation), harden your operating system(s) and have proper policies and procedures in place. When this is considered sufficiently done, an attacker that only knows the operating system should not be a big deal.

Ultimately, the attacker will determine what operating systems are used anyway, just not in the information gathering process.


They will absolutely need to do a key exchange so that A knows Kb and B knows Ka. There are different methods but they are usually based in the Diffie Hellman algorithm.

For convenience I will paste here wikipedia's example:

Alice and Bob agree to use a prime number p = 23 and base g = 5.
Alice chooses a secret integer a = 6, then sends Bob A = ga mod p
    A = 56 mod 23
    A = 15,625 mod 23
    A = 8
Bob chooses a secret integer b = 15, then sends Alice B = gb mod p
    B = 515 mod 23
    B = 30,517,578,125 mod 23
    B = 19
Alice computes s = Ba mod p
    s = 196 mod 23
    s = 47,045,881 mod 23
    s = 2
Bob computes s = Ab mod p
    s = 815 mod 23
    s = 35,184,372,088,832 mod 23
    s = 2
Alice and Bob now share a secret (the number 2) because 6 × 15 is the same as 15 × 6.

You will find several answers discussing this algorithm.


How many Stack Exchange logins should I have?

Adding logins means adding alternative methods to access your account; see this page for details.

Thus, additional logins cannot reduce the risk of hostile hijack; in fact, they can only increase the risk since they provide additional entry routes for the attacker. If you have several logins, then your "security level", formally, is no more than that provided by the weakest of the involved authentication systems.

Additional logins are meant to avoid being locked out if an authentication provider fails (e.g. ceases to operate). This can be viewed as extra "security", not against attackers, but against disasters.


connecting a secure computer directly to the internet without a router

What your ISP provides is a public IP address. When you use a home router, the router assumes that IP address, and forwards (with NAT) the connections from your actual computer. Any connection attempt to your IP address (from the outside) ends on the router, who will, generally, simply drop them.

If you do not use a home router, then your computer will own the public IP address. Connections from the outside will reach your computer directly. As long as your computer does not run services which can be contacted from the outside, and its IP stack is not bug-riddled, then it should be equally safe. After all, a home router (especially one running DD-WRT) often uses the Linux kernel, and therefore is no more secure than your Debian system.

Arguably, you may (and should) check for security updates for your Debian system on a daily basis, much more often than you update your DD-WRT router, so one may say that plugging the Debian machine directly to the modem is more secure than using a home router.

Just make sure that you know what runs on your machine. Use netstat -an to list the currently open network services: you are interested in TCP and UDP services bound to 0.0.0.0 (IPv4) or :: (IPv6); 127.0.0.1 and ::1 are "localhost" and thus do not denote Internet-facing services.


What are the disadvantages of using Shamir's Secret Sharing to implement a Partial Password scheme?

The source you cite actually already points out the problems, although it does actually considers these as problems but as advantage. To cite:

  Pros and Cons
  The highlights of this solution are:
  ...
  5. Faster to compute - verification is several multiplications of 32 bit numbers that is much much faster than computing a hash value.

The attack you are trying to prevent is that the attacker can guess correctly provided that the attacker knows the secret information stored in the database.

In your example you have a secret with 10 alphanumeric characters where the attacker needs to guess a specific 3 characters. This means 363 guesses are needed when only upper-case characters are allowed or 723 with also lower-case characters. Given that the check is very very fast the attacker can quickly find out these values (i.e. likely real-time) if the secret stored in the database is known.

If the attacker must choose more than 3 characters it gets harder to do in real-time but it can be easily precomputed. If 5 out of 10 characters are needed than 10*9*8*7*6/(2*3*4*5) possible "masks" for entering the characters must be precomputed with each has at most 365 (or 725) possible values, i.e. at most  10*9*8*7*6/(5*4*3*2)*725 really fast computations must be done. Assuming that 10000 computations could be done in a second this would take the attacker a day if he has a botnet of about 600 machines - which is not unreasonable. And it could probably be optimized. With 4 out of 10 characters one would only need 6 machines for a day and with 3 out of 10 a single machine could do this in less than 1.5 hours.

Note that this estimates also assume a random secret. If it is a user chosen secret it gets much worse since the search space is smaller. But, if the user can not chose the secret then he probably needs to write the random secret somewhere - defeating your original idea why this additional secret should be given (i.e. prevent shoulder surfing and storing in password managers) in many cases.


Does Ubuntu Server have a "phone home" feature?

Like its parent Debian, Ubuntu includes popularity-contest, a package which sends information about the system's installed packages back to Canonical (or for Debian, the Debian project) so that they can learn what packages and architectures people use most.

Both Debian and Ubuntu install the package by default, but disable its functionality; it must be explicitly enabled.


Command line Nessus/OpenVAS or NSE to find specific vulns?

Regarding nessus:  ./bin/nasl works well, especially given the “-M” flag.

It’s pretty easy to modify existing nasl scripts or to copy and paste a single nasl script to derive a custom check.

The "-M" option is key because it enables script dependencies in cli mode.  For example, the apache_http_version.nasl script has a dependency defined, which looks like:

  $ grep dependencies ../lib/nessus/plugins/apache_http_version.nasl
    script_dependencies("http_version.nasl");

If you run a single nasl script that has dependencies without the "-M" flag, your results will be empty:

  $ sudo ./nasl  -t 10.10.10.5 ../lib/nessus/plugins/apache_http_version.nasl
    $

But, if you add the "-M" flag, then you get the below (and a working check):

  $ sudo ./nasl -M -V -t 10.10.10.5 ../lib/nessus/plugins/apache_ssl_overflow.nasl
  ----------[ Executing ping_host.nasl ]------
  ----------[ Finished ping_host.nasl ]------
  ----------[ Executing http_version.nasl ]------
  ----------[ Finished http_version.nasl ]------
  [...]


Protect page sms verification from spamming

You have two areas where you can control this. First, you can control the user/number pair spamming by recording the number you sent an SMS to and limiting the number of messages your system can send to that number.

For the case where someone generates multiple users and multiple numbers, you can control that on the user-creation process before the SMS part. In this case, you can limit the number of accounts created by an IP in a certain timeframe. That prevents the SMS from being a problem in the first place.

And you always have the opportunity to rate-limit the total number of texts your system sends.


How to protect against the new malware Mazar Bot?

The text message to infect your device with the Mazar bot looks like this:

  You have received a multimedia message from +[country code] [sender number] Follow the link http: //www.mmsforyou [.] Net / mms.apk to view the message.

Don't run the link and you don't get infected.

You can also ensure that the option to not allow installation of applications from unknown sources (within Settings > Security) is enabled as this will also block the installation of the APK.

As noted in the post from Heimdal Security, Mazar won't install on a device with the language set to Russian,

  the malware cannot be installed on smartphones running Android with the Russian language option. Mazar BOT will check the phone to identify the victim’s country and this will stop the malicious APK if the targeted phone turns out to be owned by a Russian user

Mazar will "gain boot persistence to help survive device restarts". To remove, nuke the device from orbit and start again.


Is it possible to insert data into UUID?

No UUID have a specific structure which doesn't take into account appending any custom user-generated information.

Also consider how bad this idea is, UUID are a type of identification which should somehowe resolve to the actual data, identification is a mean of decoupling the desired entity from the actual content of it.
An identification should be unique and unchangeable, while the actual data may change over time.

If you start putting data into the identifier you're messing with the very goal of an identification, being unique and unchangeable.

EDIT:

The version 3 or 5 UUID is meant for generating UUIDs from "names"
   that are drawn from, and unique within, some "name space".  The
   concept of name and name space should be broadly construed, and not
   limited to textual names.  For example, some name spaces are the
   domain name system, URLs, ISO Object IDs (OIDs), X.500 Distinguished
   Names (DNs), and reserved words in a programming language.  The
   mechanisms or conventions used for allocating names and ensuring
   their uniqueness within their name spaces are beyond the scope of
   this specification.

Source: http://tools.ietf.org/search/rfc4122 paragraph 4.3


Is an Initialization Vector (IV) required with unique data?

The role of the IV depends a lot on the encryption system, e.g. whether it is a stream cipher or a block cipher, and, in the latter case, the mode of operation.

IV means Initialization Vector: this is a generic term for the start value of any looping process. In cryptography, we use the term IV to designate a parameter that is used at the start of a computation, changes often, but is not secret (otherwise we would call it a key).

In general, reusing an IV is a mortal sin, but the amount of suffering implied by an IV reuse can vary. In the case of a block cipher in CBC mode, IV reuse immediately leaks whether two messages starts with the same sequence of block; and, more generally, the IV for CBC must be generated randomly, uniformly, and unpredictably, otherwise you are open to Chosen Plaintext Attacks, as was demonstrated in the case of SSL/TLS (where it was popularized under the name "BEAST attack").

If you use a block cipher in CTR or OFB mode, then IV reuse is deadly (it becomes the infamous "two-times pad") leading to easy recovery of the encrypted data. The same applies to stream ciphers that use an IV. In CFB mode IV reuse is a bit less comprehensive in its devastation but it still is dire.

As general rules:

An IV shall never be reused for a given key.
You can get away with using a fixed IV if (and only if) you never reuse a key. If each key value is ever used for a single message, then a fixed IV can be used. However, since keys are secret while IV are not, changing the IV for each message is considered easier than changing the key for each message (you can convey the IV by including it in an unencrypted file header, for instance).
Most attack models that make encryption desirable also allow for malicious alterations, so you also need checked integrity. Combining integrity and encryption is not as easy as it seems; the sane way is to use an encryption mode that does both, which, in practice, means GCM. GCM has its own requirements on IV, namely that an IV shall never be reused, and that is important.
Anything which handles credit card numbers is likely to fall in scope of PCI-DSS, and if you ask the question you asked, then chances are that your system does not comply at all with PCI-DSS, at which point it would be wiser to get your hands off your keyboard and rethink your business choices.


Can WhatsApp, Signal or Telegram be hacked through a SIM SWAP attack?

I can only tell about WhatsApp because I do not use or specifically know about the others.
Basically, with a SIM swap attack, an adversary can get future messages intended for the target, but not access the chat history.
With WhatsApp, conversations are protected with a "security code" which enable the safe recognition of participants.
In a SIM swap attack (with a different phone used), the security code for the forged participant would change, so the others would get a notification - given they enabled that, which, unfortunately is not by default.
And it only notifies them once they receive back a message from the victim.
Read the Security Code Change Notification FAQ about it.
Furthermore, you can halt the attack altogether for 7 days if you use two-step authentication on WhatsApp.


The syntax is hash:message

Note that the length of the message is limited to 51 bytes.

See hashcat example wiki page for details: https://hashcat.net/wiki/doku.php?id=example_hashes


How to avoid scripts with hardcoded password?

As mentioned in other answers your credentials will be somewhere accessible to your script.

But I would put them is separate configuration file which will be read by your script.

You will have the following advantages:

you can use the same script for several systems (with different configuration files)
you will be able to share the script with others (or show it to someone without compromising your credentials)


Is there a security tracker aggregated file for Ubuntu server like Debian has?

There is the Ubuntu Security Advisory. It provides a mailing list, RSS feed, and Atom feed.

There is no API for historical data, according to a similar question on AskUbuntu.


Securely analyze and format a USB flash drive

Assuming that the USB flash drive is in fact just a flash drive:

Do a Physical Image of the device without ever mounting it.

Analyse the Image on an isolated machine or sandbox.

You don't say why you are dong this, but the above method allows analysis of unallocated and slack space that your file copy approach will not.

If this is for more than your own use in an attempt to clean up, such as for legal action, there are additional steps necessary.

       ----------- edit response to comment ---------

"The goal of the operation is to build a usb cleaner station"

Since you are not interested in unallocated space or performing an analysis as stated, but rather a standard virus scan and clean, why do you feel you need to perform a format and repopulate? Most similar systems I've dealt with consist of a standalone workstation with multiple virus scanners that run against the inserted media. Simply deleting, or overwriting if concerned, detected malware is no different than not finding it in unallocated space in the first place.

Note also that formatting and repopulating can create its own problems. It will disable bootable devices. It may also disable DRM files dependent upon time stamps.

Also, there may be multiple file formats. Within plain Windows 10, you may encounter: NTFS, FAT16, FAT32, or exFAT.There may even be multiple partitions.

I'd suggest that any reformatting be punted to the user/owner of the media after providing scan results.


Yes, the Windows passwords saved inside the SAM file can be changed, provided the operating system is shut down, you have physical access to the machine and the disk is not encrypted.

A tool that never failed me is ntpasswd, which can be executed from a CD or a USB key and it's very easy to use.


php-fpm.log shows a mass of unusual attempts to open primary script with random filenames

This is an attack OWASP refers to as Forced Browsing.  The attacker has a dictionary of words, phrases, etc. that they're going to try on random servers in order to find php files which exist, but aren't directly referenced by links, or other scripts on your site.

The goal of this is to identify potential vulnerable PHP files.  Filenames such as a, b, or 123 could indicate code which was still in development that was forgotten about.  Code in development could leak information in debug outputs.

Conf.php is looking for scripts which are intended for initial configuration of a web app, which was intended to be removed after configuration, but wasn't.

These types of attacks are extremely easy to perform, and usually don't indicate that you're the target of an advanced threat.  I would liken them to port scanning, where an attacker just tries a few common files on everyone and sees if anything interesting is returned.

Depending on the webserver you're using, you may want to install a module or configure it in a manner that limits the number of 404s it returns to a particular IP in a specified time.


OpenSSL certificate revocation check in client program using OCSP stapling 2

how OpenSSL actually handles OCSP stapling response

OpenSSL does not do anything by its own in this area. You have to explicitly deal with OCSP stapling in your code, both for signaling that you support stapling and for validating and interpreting the response.

Does the OpenSSL check the signature, issuer key/name hashes of the response?

If the proper functions are used you can do it. Validating a OCSP response is in most ways similar to validating a certificate or a CRL, i.e. validating the trust chain. Apart from that you need to verify that the OCSP response actually is for the correct certificate.

Does the response include OCSP responses for the whole certificate chain? If so, is there a way to know that one of the validations have failed?

This depends on what the server sends and what the client requests (i.e. single OCSP response or multiple). And since you have to implement all the details anyway you have also the information what part of the validation failed.

To sum up, can I simply rely on 'Cert Status: good' field of the response? :)

No, you cannot simply trust any OCSP response sent by the server.
You have to do all the validation of the OCSP response yourself. Apart from the already mentioned parts this involves also checking if the response is stale (too old). And of course the certificate might have been revoked in the last minutes but the response is still valid, i.e. OCSP does not provide real-time information about the status of a certificate.
Note that only very OpenSSL based tools or libraries implement OCSP and/or OCSP stapling at all and even if they do it is usually not enabled by default. This is not much different with other TLS implementations.


Where are good sources of information on current tactics that are used in cyber attacks?

From -- https://attack.mitre.org/wiki/Main_Page

  Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK™) is a
  model and framework for describing the actions an adversary may take
  while operating within an enterprise network. The model can be used to
  better characterize and describe post-compromise adversary behavior.
  It both expands the knowledge of network defenders and assists in
  prioritizing network defense by detailing the post-compromise
  (post-exploit and successful access) tactics, techniques, and
  procedures (TTPs) advanced persistent threats (APT) use to execute
  their objectives while operating inside a network.

The work is further expanded from red-team exercises in The Fort Meade eXperiment (FMX).

For on-going cyber-offensive tradecraft, please see the following resources:

Initial Entry-Point Access

Conference Material

irongeek.com
infiltratecon.com
blackhat.com
gsec.hitb.org
media.defcon.org
recon.cx

SANS

pen-testing.sans.org
https://lists.sans.org/mailman/listinfo/gpwn-list

RecordedFuture

https://www.recordedfuture.com/shared-poc-exploits/
https://www.recordedfuture.com/russian-apt-toolkits/
https://www.recordedfuture.com/top-vulnerabilities-2016/ -- https://www.recordedfuture.com/top-vulnerabilities-2015/
https://www.recordedfuture.com/recent-ransomware-vulnerabilities/

Research Labs

labs.mwrinfosecurity.com
immunityproducts.blogspot.com
blog.trendmicro.com
community.rapid7.com
threatpost.com
zscaler.com/blogs/research/
code.google.com/p/google-security-research/ -- security.googleblog.com -- googleprojectzero.blogspot.com
blog.coresecurity.com
contextis.com/resources/blog/
info.lastline.com/blog/
fireeye.com/blog/threat-research/
blog.skyboxsecurity.com
nccgroup.trust/uk/our-research/ -- nccgroup.trust/globalassets/our-research/
endgame.com/blog/
labs.bluefrostsecurity.de
signalsec.com/blog/
talosintel.com
blog.vectranetworks.com
blog.checkpoint.com
sekoia.fr/blog/
payatu.com
researchcenter.paloaltonetworks.com
symantec.com/connect/blogs/
heimdalsecurity.com/blog/
proofpoint.com/us/threat-insight/
cgsec.co.uk
expdev-kiuhnm.rhcloud.com
keenlab.tencent.com
blog.netspi.com
exploresecurity.com
silentbreaksecurity.com
sensepost.com/blog/
blackhillsinfosec.com
utkusen.com/blog/english-articles
blog.fortinet.com
blogs.mcafee.com
blog.morphisec.com
srcincite.io
blog.malwaretracker.com
nakedsecurity.sophos.com

Research Individuals

shellntel.com
cybersyndicates.com
rotlogix.com
malware.dontneedcoffee.com
blog.skylined.nl
blog.innerht.ml
d.uijn.nl
j00ru.vexillium.org
blog.malerisch.net
blog.malwareclipboard.com
malwarejake.blogspot.com

Access Expansion

This list probably includes some of the above, but these tend to focus on what to do after you already have access (i.e., Post-Exploitation techniques)

harmj0y.net/blog/
fuzzysecurity.com
breakingmalware.com
labofapenetrationtester.com
crowdstrike.com/blog/
blog.comae.io
sixdub.net
en.wooyun.io
insinuator.net
subt0x10.blogspot.com
exploit-monday.com
secinfodb.wordpress.com
moyix.blogspot.com
nightlionsecurity.com/blog/
talosintelligence.com
repret.wordpress.com
astr0baby.wordpress.com
woumn.wordpress.com
travisaltman.com
bhafsec.com
crowdshield.com/blog.php
labs.nettitude.com/blog/ -- nettitude.co.uk
carnal0wnage.attackresearch.com
countuponsecurity.com
adsecurity.org
clymb3r.wordpress.com
enigma0x3.net
kernelmode.info/forum/
labs.portcullis.co.uk/blog/
trustedsignal.blogspot.com
marcoramilli.blogspot.com
jumpespjump.blogspot.com
hexacorn.com/blog/
diablohorn.com
toshellandback.com
alex-ionescu.com
thepcn3rd.blogspot.com
blog.tanium.com
trustedsec.com
attactics.org
shelliscoming.com
hackerhurricane.blogspot.com
leeholmes.com/blog/
birendraraj0.blogspot.com
dsinternals.com
cyberarms.wordpress.com
riosec.com
insomniasec.com
darkoperator.com
buffered.io
g-laurent.blogspot.com
sternsecurity.com/blog/
xlab.tencent.com
bluescreenofjeff.com
rvrsh3ll.net/blog/
room362.com -- attackerkb.com -- cfdb.io
securitynik.blogspot.com
obscuresecurity.blogspot.com
blog.didierstevens.com
blog.varonis.com
c0d3xpl0it.blogspot.com
implicitdeny.org
cutawaysecurity.com/blog/
jimshaver.net
cyberark.com/blog/
binarydefense.com


Two factor authentication cannot protect you from a site breach. If someone gains access to the sites password database odds are they have access to any confidential data on that site too.

Two factor authentication protects you from somebody else obtaining your password. If you view this as impossible 2FA is unnecessary. With most TOTP systems this relies on you having possession of a device with the relevant key installed as well as the password to log in. Often this device is separate to the device used to access the service (i.e. a smartphone for the token generator and a laptop for the service) which also offers some protection (your session can still be hijacked) should the device logging in from or your password manager be compromised.

The main risk for yourself is probably the password manager. Vulnerabilities do occur in these. This becomes twice as important if the passwords are stored online - since they are visible to machines you do not control.


Using AES to securely store sensitive pictures

I am not good at crypto either so edit or comment below if i get it wrong.

The first and foremost thing you need to get right is APPLICATION LOGIC.

By that i mean the logic should have effective controls such that my image should not accessible by anyone else to begin with.For example if http://example.com/imgid=(GID),Now no matter what the GID is,my image shouldn't be accessible by anyone else,at all.That is two people could have the same GID .So the back end logic should be that my SESSION is associated with the images i have uploaded and the image to my account.DO NOT USE THE SECRECY OF GID AS A LOGIC

The next part is storing images at the backend

Here you should use symmetric encryption(AES256).The encryption key should be randomly generated with sufficient entropy using a cryptographically secure random number generator.The key is generated whenever a user signs up|uploads and then gets associated with that user.Store it in a database in the user collumn its fine.Then use the encryption key associated with that user to encryt and decrypt the image.

SO TO SUMMARISE.

The user signs up.He is allotted an encryption key.
The user requests the image.
The back end logic check's if the image belongs to the user asking for it.
If it is true then decrypt it using his and only his decryption key and
send the image back.


Sql injection bypass against KONA(akmai) WAF

There is a potential for automated tools to provide false positives in case of blind SQL injection even if there is a slight difference in responses. This may also be the case where WAF's are involved.

I would normally consider that anyone putting their site up for a bug bounty program have done their homework and secured their site reasonably to make it resilient to automated tools. If you get even a slight indication that the site may be vulnerable through automated tools, this must be followed by manual investigation to see if the site is actually vulnerable and what will be the impact if the vulnerability can be exploited.

The following SQLi resources may be useful for further manual investigation: http://pentestmonkey.net/category/cheat-sheet/sql-injection
https://www.netsparker.com/blog/web-security/sql-injection-cheat-sheet/

Also, be wary of extracting data from the database once you know the SQL injection is exploitable and may result in data extraction. If the website stored sensitive information, the site owner might not want you to exploit the vulnerability to extract any potentially sensitive data. Always good to review the terms & conditions of the Bug Bounty program before hand.


How IDS and Firewall Logs are aggregated and feed aggregated log to SIEM?

The logs do not need to be the same. They do not even need to log data about the same types of information (although it can help). Aggregation is simply to gather the information together.

In a SIEM, what you are most concerned with are identifiable entities: users, IPs, domains, services, etc. Firewall, IDS, and Antivirus logs will contain information about these things, and that's how a SIEM can stitch together information.

For example:

IDS log: Known malicious traffic pattern from Source IP 10.10.10.9 to Destination IP 10.10.10.10 at 12:00 pm
Antivirus log: User Sun-IT disabled Antivirus scanning on IP 10.10.10.10 11:50 am
Firewall log: Source IP 10.10.10.10 sent a high volume of data out of the network at 12:05 pm

The logic in the aggregator pulls out the data fields to provide a timeline and a logic flow:

User Sun-IT disabled anti-virus on 10.10.10.10, which was then sent malicious content from 10.10.10.9, and then 10.10.10.10 sent large amounts of data outside of the organisation.
10.10.10.9 and Sun-IT are the suspicious actors (a decision support step)
quarantine 10.10.10.9, 10.10.10.10, reset Sun-IT's credentials and forensically investigate 10.10.10.9 and Sun-IT (this is an automation/orchestration step)

The logs themselves do not need to record the same information or even be in the same format. The data fields just have to be recognisable enough for the log aggregator to identify and stitch the data points together.


Decent programmer looking to learn to Hack

When you say "Hack" I'm personally wondering what sort of hacking you mean - it's a fairly varied skill with many different interpretations.

Firstly, by far and away the biggest domain going forward in security will be web. That means SQL injection, javascript bugs, browser bugs, studies of authentication schemes etc. So as others have mentioned, OWASP and the like are fantastic resources.

Not my particular favourite area though, so here's my guide to "things to know" if you want to start looking for vulnerabilities in compiled code on operating systems. One of the first things you'll begin to realise is that there is a lot to know - you are not going to become an uber cool h@x0r!11!! overnight, or actually if you do nothing else for the next 6 months but read up on all of this, top to bottom.

Programming knowledge

You need to know assembly. Contrary to popular belief it is not that hard - most assembly takes the form instruction register, register and translates directly to machine code. You need to know an assembler, such as NASM, YASM or GNU AS. There are two different syntaxes in assembler - AT&T and Intel. They're not that far apart.
You need to know your processor's instruction set. The Intel Software Manuals for IA-32 and Intel 64 are a great resource and explain every instruction you could ever need. AMD publish equivalents; since both AMD processors and Intel processors use the same x86 instruction sets, there are, mostly, a lot of commonalities.
Knowing a debugger and a disassembler will help you. GDB is the canonical debugger used on Linux platforms, WinDBG is one such debugger from windows. Other people like OllyDbg. In terms of disassemblers, many Linux ones are powered by objdump. I personally like objconv from this author. The tool in the field of disassemblers is IDA Pro, which provides much more than just disassembly.
Have an understanding of how to use hex notation and the various word sizes.

Program Internals knowledge

Know C. Know C++. Know the difference between C/C++; for example what effect a template keyword has. Know what name mangling is and why it exists. Know what C cannot do which assembly can.
Understand how your CPU represents data and which C types most closely represent a register.
Binary file formats: knowing how executables work is very important, especially what formats they exist in and how to manipulate them. You don't need to know the internals of the ELF, COFF or PE formats unless there's a really hairy exploit going on, but knowing how to extract symbols from these files and how data is laid out in them generally will give you an advantage.
Shared object loading. Understand how shared objects or DLLs work and load.
Understand the basic run time constructions of your program. Where is the "heap"? Where is the stack, what is on it and how does it work? Where is your vtable and how does that work?

Program Internals for other languages

Nobody said you'd be working with C/C++ programs, although if you're looking at traditional, OS software it probably is C/C++. That said, you may be interested in python or jvm bytecode and how each of these runtimes work. Or perhaps the CLR.

Program environment - Operating Systems I

Basics. You should know that there are permissions and what they mean. You should know who the administrator account is, what the default is and how you acquire administrator access normally (e.g. root vs Administrator group).
Program environment. You should know where and how config for programs and the OS is laid out. You should know how services run and as what user. How do programs run automatically/on boot? How is media loading handled?

System subversion - Operating Systems II

Know the difference between the various CPU Rings.
Know how drivers are loaded into the operating system and what they can do.
Have an accurate idea of how the OS handles permissions, resources, filesystems etc. Better if you know exactly how.
Know about mechanisms for intercepting system actions.
Understand how to perform shared object injection on your system (1, 2).
Know how various subsystems in both user and kernel space work and how to attack programs over these subsystems.
See here for OS internals.

Networking & Network Services

Knowing something about networking is important. What does a packet
look like? What is the OSI stack? How do you inspect packets and
network traffic; how do you connect to a network from your system?
Know common network services for your target platform, most commonly HTTP, SSH. Understand how they can be exploited.

Common exploits and their defences

Know what smashing the stack is, ret2libc.
Know what ASLR and stack canaries are.
Know about vptr exploits.

Analysis tools

Know about fuzzing, static analysis and possibly instrumentation.
Know some tools for achieving this, like pintool, splint and peachfuzzer.

This list is probably incomplete and reflects a set of knowledge that should be helpful for understanding how reverse engineering works and how you go from there to finding vulnerabilities and exploiting them. I do not, myself, know absolutely everything in the above list inside out and I've been researching this stuff for a while. As I said, becoming good at this takes a lot of time, dedication and patience.

Once here, you can start to study applications and how they process data and begin to work out how exploits against them work, such as PDF/Flash/Java vulnerabilities.


I appreciate where you're coming from, but this is actually very dangerous in a production system.  While it might be kosher if 100% of all failed password attempts were in fact random evil people trying to break into your system, the fact is that a lot of bad passwords are actually going to be legitimate users making simple typos.  And there's the rub: if someone managed to get a hold of your bad password cache, they'd likely have a bunch of barely typoed legitimate passwords that they'd otherwise have no way of knowing.  In effect, every user who mistyped their password would have dramatically weakened their account in the case of a breach—quite possibly fatally.

Logging which accounts get bad passwords, and how often, can be very useful in auditing.  So for that, go ahead.  Just don't log the failed passwords.

(In practice, the passwords you'd get from doing what you're proposing are likely the same ones available already to tools like John the Ripper, anyway.  If you're concerned about users' passwords being compromised, it's likely sufficient to use a tool like John and run known passwords against your user accounts.)


Cryptowall files found, but nothing encrypted

In your case, I would backup all my files if it's not already done (It's the only reliable solution against ransom-ware).

According to http://www.sentinelone.com/blog/anatomy-of-cryptowall-3-0-a-look-inside-ransomwares-tactics/ you can look at

  HKCU/Software/(CUUID)/(sorted 2nd half of CUUID)/(FullFilePath) =
  (Volume Serial Number)

And you will see the list of encrypted files if any. Continue scanning your workstation with antivirus solution. You can also launch procexp from Sys Internals and check if you see any wired process running. Click on Options->Verify Image Signatures to be sure that a process is not masquerading as a legit one. Malware are used to be named as Microsoft binaries (svchost or else).

Hope this helps.


How to properly hash a key? big vs small digest

In practice it literally doesn't matter.

If your key size is 512-bit (I'm not sure what cipher you're using, as none that I'm aware of use 512-bit keys, but whatever) then you've got two scenarios:

In a small digest you've got so many collisions that discovering the original key by looking for matching values will give you a silly number of results. Not to mention trying to execute any remotely computationally complex algorithm 2512 times, or even 2256 times, is practically impossible.
In a large digest you've got less collisions (a SHA-512 hash of all values in the 512-bit space will, in all likelihood, produce a number of collisions - it's just really hard to prove) but a much slower hash function. Again, with an input space of 2512 or 2256 it's practically impossible to brute-force.

There's a lot of incorrect thinking on both sides of your argument, though:

Brute-forcing a single 256-bit value on a conventional computer would require more energy than we can observe in the known universe due to Landauer's principle. More context on this can be found in a post on Bruce Schneier's blog. A 512-bit value is even more ridiculous - it just won't happen, even if you've got an efficient implementation of Grover's algorithm on an incredibly advanced quantum computer.
Rainbow tables aren't useful on these kinds of attack. The idea of a rainbow table is to enumerate all possible input values ahead of time for a known hash function, and store the plaintexts and hashes in a chain. This makes it very easy to perform lookups against a hash and get the original value. However, this is only feasible for sensibly sized input spaces, e.g. passwords. With a 512-bit input space your number of states is so large that storing even an 8-bit hash for each input value would require 2512 bytes (i.e. 1.190853×10139 petabytes) of storage. Even if you could somehow compute all those hashes (Landauer's principle again) you'd need to store them somewhere. Even if you could store an entire byte, somehow, on each photon in the known universe, you'd have nowhere near enough - only about 1050th in fact.

All in all, just pick a strong cryptographic hash like SHA-256 and don't worry about it.


How to identify user created vs. external functions in disassembly

If the program is dynamically linked, functions are usually pretty easy to identify since you will see imported functions with their correct name. E.g. a user created function will be shown as sub_xxxxxx, and printf will be shown as printf.

However, if the program is statically linked, it will be hard to tell the difference between user-created and imported functions. Tools like Ida's FLIRT exist to recognize known functions; there may be some free alternatives as well.


Implementing Secure Remote Password with Remote Desktop Application

RDP is a protocol with a rather long history, and multiple layers. Recent versions can mix with SSL/TLS in two ways (which can be used concurrently):

Since RDP v5.2, the protocol can embed a TLS session (the TLS records are transported as so many messages by the RDP transport layer); the TLS session protects the connection.
Since RDP v6.0, the whole protocol can be embedded within a TLS session opened from the client to a specific gateway (called Terminal Services Gateway).

Both TLS usages can conceptually use SRP, since there is a standard for that. However, this will not happen until you get an RDP client and an RDP server who agree on using SRP with their TLS tunnels. I am not aware of any existing RDP implementation (either client or server) which currently does that. You could launch an opensource project on the subject, building on FreeRDP (for the RDP protocol) and GnuTLS (for the TLS+SRP part). This is not a five-minute task, though...

Another possibility is to establish a kind of SRP-powered VPN and tunnel the RDP protocol through that. The SSH protocol can help; it includes features for port forwarding through the SSH tunnel, and at least one SSH implementation supports SRP. This kind of tunnelling will be clunkier, but easier to assemble than patching RDP implementations (in particular if you want to use close-source RDP client and servers, which is usually the whole point of RDP: to use Microsoft's client and/or server).


Is shred -n1 /dev/sda demonstrably worse than Bleachbit?

No

The wipe free disk space tool will overwrite free blocks. Here you are overwriting the full disk.

While with the wipe free disk space tool there could be remaining data in slack space of used blocks¹ (and perhaps some other blocks that cannot be overwritten), with shred you are overwriting the contents of the whole disk, so it seems that the shred solution would perhaps be more reliable.

Note that you are not defining what you consider reliable, and both solutions will probably be similar, but given that you asked is shred is demonstrably worse, the answer is clearly no.

¹ Ok, they could be overwriting that, too (editing every file file to enlarge with random data, and then shrink back), but it would be an unusual move (and could conflict with other processes, not to mention files the OS won't allow to touch...).


Is there any reason to not use my real name in my email address?

Many contact sites or services use email for communications. While it might not matter in most cases, having your real name as part of the email address may be concerning sometimes, such as:

an AIDS Clinic
an Abortion Provider
a Left/Right wing organization
a Car Salesman
Others

It may also lead to cyber stalking, as in people finding "John_Smith@whatever" and not differentiating.


How up-to-date is the plaintextoffenders list?

The plaintextoffenders.com list is built from user submitted reports along with screenshots censoring sensitive data.

The FAQ says they rely on their community to contact companies. They also rely on companies contacting them to be removed from the list if they think they are listed in error, or if the issue has been corrected.

The validity of the claims could be somewhat dependent on how old the report is. It's possible that the company has since updated their security but not submitted a request to plaintextoffenders to be removed.

If you are thinking of using a company listed, you can always contact them yourself and ask if the situation has been resolved.


Test registration form using brute force

I'm positive there are quite advanced tools available for this. Form fuzzers come immediately to mind.

If you're just trying to check for what happens with 10,000x form posts come in, you might be able to script something fairly trivially.

I haven't tested this, but you might have luck with something similar to:

for ((i=1; i<= 100; i++)); do
    curl -s "localhost:1337" -c "cookiejar" -d "user=foo&pass=bar"
done

Edit: I wanted to circle back on this w/a couple of links: You can find all of these on OWASP's Testing for Brute Force page which has a lot of information on what you're trying to accomplish.

Bruter: http://sourceforge.net/projects/worawita/
THC Hydra: http://www.thc.org/thc-hydra/
John the Ripper: http://www.openwall.com/john/
Brutus: http://www.hoobie.net/brutus/
Basic Auth Bruteforcer: https://market.android.com/details?id=com.firebird.basicauthbruteforcer
Cain & Abel: http://www.oxid.it/cain.html

Note: Chrome gave me some trouble when trying to go to thc.org. If you want to get past all that, just see the Hydra GitHub Project instead.


What concepts / technologies does Ultrasurf use?

How do tools like Ultrasurf work?

They use a second computer (proxy), actually many such computers.

Then the browser is instructed not to connect to the real website you want, but to the proxy, sending it the request. The request itself is heavily encrypted so that nobody except the recipients may know its contents.

Imagine you're underage and you still want to get drunk. You give money to a over-age friend to buy alcohol in his name, then get the booze back to you. The bartender sees a guy of twenty-four and has no qualms in giving him alcohol, nor has he any knowledge of who is actually going to drink it.

At a very high level, what concepts / technologies are used by tools like this?

They use proxying (a proxy is, unsurprisingly, the computer that receives your requests and acts on your behalf), through an encrypted tunnel (or a full virtual private network or VPN, which is actually a more advanced implementation of basically the same concept).

How does it prevent others (employers/ censorship bodies) from seeing end user traffic?

These are actually two questions, because depending on who the "others" are, you might need to hide not only

what you are communicating or who you're communicating to

but also

the fact that you're communicating privately.

The encryption takes care of the "what", and the fact that you're only communicating with a proxy, and only the proxy knows who to forward your requests to, takes care of the "who".

But while your employer does no longer see requests for nsfw.com, he now sees encrypted and unreadable requests directed to hidethisfrommyemployer.com. Depending on the employer, this might still be undesirable or have nasty side effects.

In some countries, using encryption to sneak past censorship will actually have the opposite effect, since encrypted communications are usually extremely easy to recognize. Also, the networks used on the other side to supply you with a proxy service can be identified and blocked, so that this strategy no longer works.