I worked with an international logistics company that had an SQL vulnerability in the 5250 emulator on a handheld wireless barcode scanner.

Theoretically you could craft a barcode that would amend the database and alter stock and inventory levels etc...

Theoretically:-)


Didn't understood what the purpose of the newly discovered TCP Faking attack

The idea here is to open a TCP connection with a spoofed address (i.e. the IP address seen by the server is not the real IP address of the client), thus bypassing any IP black- or white-listing. It has severe restrictions on what can be done: only one packet can be sent, and it should not depend on anything coming from the server.

Practically, it could for example be used to send a HTTP request.

Now, I don't think this is really new, IP spoofing with TCP has always been about guessing the sequence number sent by the server in the SYN-ACK packet. In the past, it was mostly done by using weaknesses in the generation of this number, here it is a simple brute force attack.

It doesn't avoid the usual difficulties faced when spoofing a connection neither, the real host using the spoofed IP still need to be unable to respond, and there is no way to see what the server sends back. It is also limited to a single packet.


Allow read-only access to server-side pages for security review?

This is feasible to set up. But it only superficially solves the problem.

It doesn't necessarily create extra trust. Why? It's not too hard to redirect FTP to another server (or to another folder). An untrusting user could claim that you have set up two servers; one with the live, malicious code and one with innocuous code. You direct FTP requests to the innocuous server, and let the web server run the malicious code. Remember, you get to choose what they see when they ask for an FTP connection. Spoofing it isn't too complicated.


Can back doors be reasonably well hidden in end-to-end encrypted communication?

Your assumptions are wrong: neither are governments good at keeping secrets nor do they have no rouge employees.

Yet, let‘s assume your assumptions do in fact hold.

How such a back door might look exactly is highly dependent on what protocol you are trying to compromise the secrecy of.

So let’s keep it abstract:

For eavesdropping on end-to-end encrypted channels, there has to be a weak spot in one of the following parts:

The crypto itself.

Using weak cryptographic systems such as the infamously fast Double ROT13 encryption scheme allow for eavesdropping. Yet, everyone implementing it must know about this and everyone sniffing their own network traffic can see it.

There are however more “stealth” ways of using weak cryptography, though crypto systems are usually very thoroughly researched before they go into production. They are also usually open source and well reviewed; anyone can check them out and see wether there is a weak spot - and switch to another product using strong crypto if they do.
The trust model.

For the key exchange to work (and knowing you talk to the person you are supposed to, not e.g. a government as a MITM), you do often have to rely on a third party that vouches for the identity of the counterpart of your conversation. Being it a CA or the web of trust or the apple or Facebook or WhatsApp servers, you do base your trust on something. So this seems like a good point to introduce false keys, MITM’ing all conversations.

The problem here is that most end-to-end encryption capable clients do allow the user to check the key fingerprints in person or over another channel with the entity they are supposed to belong to; Threema even denotes this - and only this - as “green”, where getting the keys from the server alone is just “yellow”, in their trust model. Whatsapp allows that too and warns users when keys change (e.g. when the phone is switched). So this would be easily detectable as well, with the same consequences as in 1. - the users just switch to a different Service wreaking havoc on the services business plan.
The endpoints.

Compromising the endpoints might be the most plausible way of breaking end to end encryption, yet this amounts to a massive undertaking in a broad deployment as a single point of failure does not exist - unlike on the other options.

Now, the real problem is: even if such a back door is implemented, there will be researchers finding it sooner or later. Maybe they will think it’s a bug, not a feature, but it will be found and will be exploited, especially when used in the wild and on a broad basis.

Remember that this technology is neither delivered by only one vendor nor is software bound to laws of a specific country and can be easily reproduced even if forbidden in a country.

Simply put: you cannot have your pie and eat it, too. If you put in a back door, it is there - and others will find it.


Can I protect my router from a Mirai Worm and how do I know if I'm vulnerable?

Like your quote says; change the password. It would be much harder to fix if Mirai used 'actual' vulnerabilities (software bugs i.e. memory corruption). Then you'd have to hope there's an update available and apply that. But it seems like it's only taking advantage of people who leave their devices with the proverbial 'changeme' as password.


Can single-purpose PCs be useful as a security measure in this situation?

The basic idea of restricting what can be done with the system does improve the security, because it decreases the attack surface and it also reduces the risk that the user will cause harm accidentally.

Unfortunately you have added one software which has a very great attack surface - the web browser. To limit this risk you should explicitly limit what sites can be reached from the browser. This could be done with a restrictive proxy or by configuring the browser to use a proxy.pac file which denies access to all but a few sites.


Are there any security risks in replying to an SMS message?

Some telephone or SMS numbers allow for an additional charge that is automatically recovered by your phone provider and reversed to the owner of the number. This is mainly used (legally) for some TV games where each participant pays a little money when calling a special number or sending a SMS. At the end, either one of the players earns something, or the answers of participants were used for an election (a miss election for example).

So a not so good company could set up a system like that and send tons of SMSs asking for a reply to such an overcharged number - and optionally omit to say that it is overcharged... The cost of mass sending SMS is low, so if they get an acceptable return rate, they will earn some money with it.

When you are aware of that, and willingly send an overcharged SMS to participate in the election of the song of the year, all is fine. But when you receive a SMS just pretending to come from a friend and end in paying more than the simple SMS it is robbery.

But as this kind of company hides themselves abroad or vanish as soon as you reclaim to have your money back, it is much better to never send them any overcharged SMS.


Vulnerabilities common only in mobile solutions?

Yes.  Mobile browsers are generally at greater security risk than full-sized desktop browsers, for a number of reasons.  Here are some examples:

Tap-jacking. Tap-jacking is more powerful on modern phone browsers (compared to full-size desktop browsers): see this paper.
Phishing and UI spoofing. Mobile phone browsers don't reserve screen real estate for the "browser chrome" and its corresponding security indicators: e.g., the address bar, padlock icons or signs of HTTP/HTTPS/EV-HTTPS, etc.  In particular, web pages can go fullscreen and scroll away the browser chrome, leaving no browser chrome.  Once the web page has done so, a malicious web page could draw a spoofed address bar.  In one user study, almost every single user was fooled by a phishing attack of this sort.  This may pose increased risk: e.g., to phishing, clickjacking, and other social-engineering or human-factors related attacks on the user.
Vulnerable mobile web sites. Many websites offer separate sites specifically for mobile clients.  Those mobile-optimized versions of the site often have vulnerabilities not present in the normal site; see, e.g., this paper for some examples of this (relating to clickjacking).
Webview-related vulnerabilities. Many mobile applications use webviews, to seamlessly embed a web page into the application.  However, at least on Android, if webviews are not used carefully, they can introduce vulnerabilities that allow a malicious website to misuse the application's access to the user's phone and information.  This risk is specific to mobile systems.


Safe erase of a Toshiba SSD with Absolute Secure Disk software

(As Rory noted, this isn't strictly a password-cracking approach, so may be better suited to Superuser or Server Fault. But the question is relevant to not knowing the password, and the original data isn't generally accessible, which has information security implications that may make it worth answering here anyway.)

It sounds like the Absolute Secure solution may be setting a controller-level ATA password. With high-security configurations, drives locked in this way cannot be unlocked without the original password, and can only be re-wiped.

If you attach the drive directly to a system with a SATA or other direct cable (and not via USB), and then boot from your Linux live CD, you may be able to at least restore the drive to service, as follows.

First, check to see if the drive is in a frozen state:

hdparm -I /dev/sda | egrep -i frozen
        frozen

If so, you may have to tell the OS to sleep the system, and then wake it back up, to un-freeze the drive.

Next, issue the following to set a new password, and then erase the drive:

# Set a new password to prepare the drive for erasure.
hdparm --security-set-pass password --user-master m /dev/sdX

# Erase the drive.
hdparm --security-erase password --user-master m /dev/sdX


Does it make more sense to encrypt whole backup disk or only encrypt backup itself?

It depends upon your level of paranoia.

Short answer: all 3**

Minimal answer: encrypt the backups individually

Longer answer: The FS-level encryption and file-level (or backup-level) encryption are protecting against two different (although overlapping) attacks.  The FS-level is protecting against the unauthorized mounting of the backup drive; whether that's the physical theft of the drive, or the attaching of the drive to your system when you do not intend for that to happen.  The FS-level really only provides protection if you are only mounting the disk when you are either performing a backup or restoring data.  Otherwise, if you leave it mounted to the OS, the backup files are available to whomever has access to the machine they are mounted.

The file-level encryption protects your files for unauthorized reading whether they are still on the drive or not.  So in the case that the backups are exfiltrated from the drive, they are still unreadable w/o serious effort.  Hence, at a bare minimum, you should encrypt the backups.

** There's actually a 3rd level of encryption you can have, which would be at the physical disk layer.  This seems to be a growing feature among SSD drives.  In many respects, the Disk-provided encryption is somewhat redundant to the FS-level encryption.


Create a page to experiment with SQL injection

From SQL injection to shell - this will walk you through manual exploitation of sql injection, download the vm and follow the step by step pdf.

there is a difference in understanding and doing. this will help :)


2 factor authentication - cost effective solution for a web startup

If you're willing to delegate your user authentication function to someone else, then Google's two factor authentication is a good option.


What does Django's ALLOWED_HOSTS variable actually do?

It took me a bit of digging to find it, but I found this which seems to answer the question:

https://www.djangoproject.com/weblog/2013/feb/19/security/#s-issue-host-header-poisoning

  Issue: Host header poisoning

  Several previous Django security releases
  have attempted to address persistent issues with the HTTP Host header.
  Django contains code -- and some functionality shipped with Django
  itself makes use of that code -- for constructing a fully-qualified
  URL based on the incoming HTTP request. Depending on configuration,
  this makes use of the Host header, and so an attacker who can cause a
  Django application to respond to arbitrary Host headers can cause
  Django to generate, and display to end users, URLs on arbitrary
  domains.

  Previous iterations of this issue (see CVE-2011-4139 and
  CVE-2012-4520) have focused on tightening Django's parsing of Host
  headers, to eliminate various means by which attackers -- using
  techniques such as username/password pairs in their submitted Host
  header -- could exploit this.

  Ultimately, however, Django alone cannot ensure that an attacker is
  unable to submit, and cause Django to accept, arbitrary Host headers.
  Properly securing this in a deployed Django instance additionally
  requires configuration of the web server, and both the configuration
  and the achievable level of security vary with the server being used.

  In light of this, the get_host() method of django.http.HttpRequest
  will now validate the Host header against a new setting,
  ALLOWED_HOSTS. This setting is a list of strings (by default, an empty
  list) corresponding to acceptable values for the header, with some
  support for wildcards.

  Code which does not use get_host(), or Django configurations which can
  determine the current hostname without recourse to HTTP headers (i.e.,
  when Django's sites framework is enabled) will not be affected by this
  change.

  Since this is hardening/tightening of a previous issue, it does not
  have a new CVE number.


TLS Downgrade attacks consist of a MITM threat actor forcing the weaker TLS/SSL protocols because the server permits the weaker protocols. Server-to-Server is still possible but client-to-server is the most likely.

A good candidate for an attackers success:

In this scenario, the attacker in the simplest form, drops the packets of the victim requesting the higher protocols, thus the downgrade attack, then the client retries lower protocols. This downgrade gives the attacker a high probability of being able to acquire session cookies, modify traffic, steal credentials, intercept emails, etc.

Victim   - Supports SSL 2.0, 3.0: TLS v1.0, v1.1, v1.2, v1.3
         - Supports High, Medium, Weak, CBC, and Null Ciphers
Attacker - Has Successful MITM
Server   - Supports SSL 2.0, 3.0: TLS v1.0, v1.1, v1.2, v1.3
         - Supports High, Medium, Null Ciphers

A good candidate for an admin's success:

In this scenario, the attacker can attempt to try and modify the negotiations but the server is hard set, use high protocols and ciphers, or nothing. The attacker really has no chance to decrypt the communications. Or at least not yet... Computers in 5-10 years will be different and these protocols will become outdated. IETF already has a draft for decommission of TLS v1.1.

Victim   - Supports SSL 2.0, 3.0: TLS v1.0, v1.1, v1.2, v1.3
         - Supports High, Medium, Weak, CBC, and Null Ciphers
Attacker - Has Successful MITM
Server   - Supports TLS v1.1, v1.2, v1.3 or just TLS v1.2 & v1.3
         - Supports High Ciphers only


A few thoughts about that:

Biometric data is easy to access and should not be used as a password, only as additional authentication.
As Freedom explained quite well your government already tracks you.
Biometric data like fingerprints are mostly not stored as raw images but in form of hashes. An algorithm extracts certain characteristics. You cannot restore your fingerprint from this data.  Still, it might be possible to trick some authentications with this data.
Biometric data could be misused. DNA markers could contain information about medical issues or ancestry. Currently forensic teams are trying to reconstruct how a person looks from their DNA.
Data that is not needed should not be stored. Especially not centralized.

I am against this because: Freedom before Fear.

Update because of comments

With "Freedom before Fear" I tried to translate the German "Freiheit statt Angst" slogan. We tend to give up freedom because we fear terrorism and crime. The Freedom is in this case the control over your private data. Data you can not change. Data that defines who you are. This data might stop someone from copy a passport or it might not. In anyway it does not stop terrorism but it does create the risk that the data is miss used.

A database of DNA where you could look up all foreigners. What could a dictator do with this? A database that tells an insurance that you could get a sickness. Who knows what will be possible with this data in the future?

If you create such a database there is always a chance it will be used for evil. How and in what way is impossible to tell but it would not be good for you.


Is KeePass's method for key derivation secure?

Based on a 2014 tweet from Malik Mesellem, who created a KeePass master key cracking software named KeeCracker, he was getting a bit over 1,000 password guesses a second with his tool on an Intel i7 CPU.

In an answer to this similar question about KeePass key storage from back in 2011, Tom Leek discusses the possibility of 32,000 password guesses per second with a quad core CPU.  That may have been a best case estimate that is further degraded due to inefficiencies in the cracking software managing those AES operations, like we see with the KeeCracker figure.

Either estimate is pretty slow as far as password cracking goes, where you regularly see hundreds of millions or billions of hashes per second on a single GPU with a bad choice of algorithms like MD5.  The KeePass implementation is a bit faster than a 'typical' modern implementation of bcrypt or scrypt, but probably not by so much that you'd consider them to be in a whole different class.

I'm not aware of any criticisms of their choice of algorithms or implementation, and I'm not really qualified to evaluate them myself, but it appears that it is not considered a bad choice.  I think it's fair to question whether 6000 rounds is sufficient if you want a 'future proofed' password archive, but it doesn't seem that this is necessarily an irresponsible number to use today.

So the biggest concern of KeePass users should be choosing a master passphrase (e.g. not a password) that is going to withstand simpler cracking attempts.  Don't use known phrases, don't make it too short, don't make it predictable based off someone's personal knowledge of you, etc.  If you properly avoid these things it becomes unlikely that your passphrase will be discovered through dictionary, hybrid, or brute force cracking attacks.


why encryption result using 3 des is different between website and openssl commander?

You're using OpenSSL incorrectly. The obsolete -k parameter specifies a password, not a key file. This password (literally "key.txt" in this case) is then run through a key derivation function (PBKDF2, per your command line). However, if you're providing a key already, (as opposed to a password), you don't need (and shouldn't use) a slow key-derivation function. Since you are using a key as though it's a password in one case and actually using it as a key in the other, of course the output will be different. The ciphertext will differ, as it's created using a derived key (rather than one provided directly), plus KDFs require additional parameters (work factor and salt) which must be stored and transmitted in plain text as well.

Indeed, looking at the base64-decoded output of the openssl command, it is relatively long (40 bytes) contains the plain English word "Salted", which is unexpected if it were nothing but ciphertext of English text:

$ echo -n 'U2FsdGVkX1/CxK/qRQDXLsl/pTE650K/xcHnPWbhb3Z1EtyrNPyzVA==' | base64 -d | hd
00000000  53 61 6c 74 65 64 5f 5f  c2 c4 af ea 45 00 d7 2e  |Salted__....E...|
00000010  c9 7f a5 31 3a e7 42 bf  c5 c1 e7 3d 66 e1 6f 76  |...1:.B....=f.ov|
00000020  75 12 dc ab 34 fc b3 54                           |u...4..T|

The website output also contains a handful of printable ASCII characters:

$ echo -n 'c9FxRbKYE/We1igMIU58833njnVjs6KH' | base64 -d | hd
00000000  73 d1 71 45 b2 98 13 f5  9e d6 28 0c 21 4e 7c f3  |s.qE......(.!N|.|
00000010  7d e7 8e 75 63 b3 a2 87                           |}..uc...|

but no plain English words, and (as expected) a significantly shorter length (just 24 bytes, which makes sense if you take the 20-byte input string "hi all I am someone" and pad it out to a multiple of DES' 64-bit block size). I don't know exactly what the format of the OpenSSL command is, but that's definitely your problem.

You're also using the website you linked incorrectly. It expects the key in plain text, not base64 (or hex) encoded. Since your key contains non-printable characters, this is tricky to paste into the text box, but I tried it with a key made only from printable characters and it worked correctly.

To actually use the key you want with openssl, you first need to convert it to a hex representation (there might be an easier way to do this on the command line but this uses tools most likely already installed):

$ echo -n "CiC4IOH/ASAHd7dxOSVIw4VSG/yG33FN" | base64 -d | od -A n -t x1 | tr -d '\n '
0a20b820e1ff01200777b771392548c385521bfc86df714d

You can then pass this hex-encoded key to the -K (note: capital K) parameter in openssl enc:

$ echo -n 'hi all I am someone' | openssl enc -des-ede3 -base64 -K 0a20b820e1ff01200777b771392548c385521bfc86df714d
Lv0OrqobWvh/u681nIe3ONhSy0PYnhlK

I can't find any way to pass the symmetric key directly to openssl from a file, unfortunately. OpenSSL is a cryptographic library, more than a command-line tool; its command-line utilities are often convenient, but it's mostly used for its API.


The soft limits can be used to implement rekeying. If either limit is reached the kernel will send expire messages to anyone listening for events on XFRM or PF_KEY sockets. But if a hard limit is reached the kernel also removes the state/policy automatically.

A keying daemon that installs SAs will usually set the limits so that there is enough time between the soft and hard limit to rekey the SA and it will use the expire event for the soft limit as trigger the rekeying. This mechanism is defined in RFC 4301 (Security Architecture for the Internet Protocol).

While a keying daemon could implement its own timer to trigger rekeying after a certain time, this isn't possible for packet/byte limits without relatively expensive polling of the usage counters of each SA.


Is it safe to plug an SD card or USB flash drive in my sytem if I turn the write protection on using the switch?

you look at the device and sees it has a write protection switch, so
  you move it to turn it on and the storage media is now read only.

  Now does that mean malware can't spread from it, can the malware
  bypass that protection?

No.  Malware acts by being read and executed from media, not by being written to it.  Setting the media to read-only merely guarantees you can't add or delete anything from it.  If your operating system wants to, or is tricked into, reading and running code from that device, you're vulnerable.


Is it possible to get the Qubes OS security concept on other distros?

I'd give you a "yes" to each of your three questions (2 being easiest, 1 hardest to achieve), but with a very large footnote.

You can get some (or most) of what Qubes OS does with the technologies you mention (e.g. virtual machines, container/sandboxing technologies, app armor profiles etc), at least if you're on Linux, because this is basically how Qubes OS does it.

However, getting it right is a tremendous amount of work, and you'll never get the same level of isolation that Qubes OS provides you with, because in order to achieve that, you'd have to run your OS as a virtual machine on a hypervisor, and you'd need to hack the windowing system.

The last is the relevant part. Qubes OS gives you a desktop with individual windows that can run in their own virtual machines, and keystrokes, mouse events etc are isolated from each other.

With current X technology, this kind of isolation is simply not reproducible while keeping the same level of usability. The X Window system is designed in a way that makes it pretty much impossible to isolate user events without code changes. Things might change once distributions come with Wayland, though.

  I don't know if Sanboxing is similar to this

There is quite a difference between Sandboxing and Virtualization. Virtualization, depending on the degree of virtualization, either simulates a whole computer in software or uses special CPU hardware to virtualize it (which is much faster). Sandboxing means that you're putting some software in an environment where it can't do much harm if it misbehaves. Virtualilzation is one specific way to achieve this, and it provides the best isolation apart from actually using a second computer. Using Linux capabilities to reduce access to specific kernel functions for a given process is another, and chrooting the filesystem so that a process sees only a specific part of the whole filesystem yet another. User Mode Linux (e.g. running a Linux kernel as a process in another Linux) and solutions such as Linux Containers (LXC) are ways to build sandboxes that aren't quite as resource-intensive as full virtualization, but still provide good isolation.


What do I do, I got a virus warning on a webpage

I think it's just a scary tactic from the site to make you download something that can be just an annoying adware to a very bad malware. It can be also a common method called Clickjacking where a user is tricked into clicking something which will then execute a malicious action or start a malicious download.

Do you have any AV on your OS? Did your browser downloaded anything automatically?

If you have AV and haven't downloaded anything I would say you are safe. Just ignore this kind of messages.

They are just a scary tactic, no different than messages like " you just won $1000 dollars" or pop-ups saying you are "infected with a nasty virus"  so ignore every message like that on websites: they are fake and aim to trick you to do something malicious.

But anyway since you are using Safari, I recommend this amazing open-source Blocker for Safari. I recommend going to its settings to add more lists to it. (Just don't use EasyList without element hiding rules or  Anti-ThirdpartySocial list since they can break lots of pages). While I'm doing recommendations I could say you should also use https everywhere.

It appears to be Safari specific, because when I visited it with other browsers I didn't see anything. However I still think this is a fake pop-up intended to lead you to download their software. Macupdate.com appears to be a store for apps for MAC.

I visited the website with my iPhone too and I couldn't reproduce what happened to you. I'm not sure why this but I would keep the usual : if nothing has being downloaded on your PC then ignore these "warnings".

Some useful tools if you happen to come to another websites like this one : Google Safety Report and VirusTotal.

Do not execute or open any file that you did not expected to download. Keep your browser updated and use ad-blocking software if possible.


What could an attacker do on a server where Source Control Management (SCM) files are accessible?

For my answer I presume that this is the git repository of the application that the server is running, that it contains the full git repository, and that you are able to download the whole thing.  This is both surprisingly common and surprisingly dangerous.  The git repo contains not just all the code actively deployed on the server, but also every line of code ever written (deployed or not) for the application.  There are a few things to do with that:
Search the Repository for Access Tokens/Api Keys/Database Credentials
A common refrain is that you shouldn't store passwords/keys/credentials in git.  This incident is one big reason why.  When attackers find them it can end very badly..  Finding the git repo publicly available on the website means that it is time to go treasure hunting.  Every key you can find is basically a treasure map, and you never know what you'll find!  There are even tools out there that try to automatically find credentials in git repos for you.
As an example, the best case (for you) scenario is if the database password and host are stored in code.  If the database accepts remote connections and isn't properly firewalled, you may end up easily upgrading yourself to full database access too.
Whitebox Penetration Testing
Whitebox testing can be much more effective than blackbox testing because you have more information to aid in your search.  With full access to the source code, you're basically now a whitebox tester for this site.  You can spend a few hours getting to know the way their application is put together and organized (which is obviously easier if you happen to be familiar with their technology).  From there you have substantially more information available for bug finding.  Of course you don't have infinite amounts of time for reading their code, so I'd start simple.
I would Check their routing tables for very juicy looking endpoints (admin/users, admin/orders, admin/reset_user_password), and check those endpoints for holes in access control that you can exploit.  Similarly, I'd pick some "high impact" pages that I already know about and dive through their source code, looking for any vulnerabilities that might be hard to find while trying to blindly exploit things from the outside.
Social engineering attacks
This one is a bit much for your typical bug bounty scenario, but I thought I'd mention it anyway.  git records not just all code every written, but who wrote it.  It requires both a name and email address to be attached to every commit (although it doesn't check that an actual email address was provided).  This means that you have also gained access to a list of the names and email addresses of everyone who has ever done any programming for this system.  This could lead to many opportunities for more informed social engineering attacks, which increases the odds of success.
Summary
Finding someones git repo is definitely bad and a reportable bug in-and-of itself.  This can also lead to the discovery of even bigger vulnerabilities.  However, like anything else in penetration testing, doing so can still take some effort and isn't guaranteed results.  If they don't keep keys in their repo, if they have good application security overall, then their git repo may not practically benefit you in the slightest.  Of course, allowing the repo to be publicly accessible is a big security blunder in the first place, so I'd be surprised if there aren't more mistakes waiting to be found.
Also, to answer your last question: the hash you found won't help with penetration testing.  git uses hashes as ids for both files and commits.  That is just the hash (aka the id) of the commit which is the HEAD of the active branch.  In other words, that hash just tells git which code is currently "active", since git keeps around the full code history.  It is built from data that is otherwise easily accessible in the git repo in plain-text format.


Disabling external DNS lookups - is this really so beneficial for security?

When firewalls are correctly configured, DNS is our way into and out of the network. Depending on your security level, blocking DNS where it is not needed can be useful hardening.

As a security consultant, it is not that uncommon to find yourself in a system with a limited server-side request forgery or some other server-side vulnerability. Some customers have very well-configured firewalls which prevent us from using it to get much further, but through DNS we can typically still learn more about the network and sometimes setup useful data tunnels. In such a case, disabling DNS would be the final nail in the coffin.

  it's causing me issues

That is the risk: if you disable DNS and someone does need it (for example for apt update), you risk that sysadmins use ugly workarounds, making the network less secure instead of more secure. If you can't do your work properly, then disabling DNS altogether is not the right choice.

Might a limited resolver be a solution? It could run on localhost or perhaps on a dedicated system, and it could be configured to only resolve a whitelist of domains. Since you mention you're moving your data and applications to other people's computers ("the cloud"), it sounds like you might only need to resolve the domains belonging to whatever SaaS/*aaS service your company uses.

The pitfall there is that whitelisting something like *.cloudCorp.example.com probably allows an attacker to buy a VPS at cloudCorp and get a matching domain name. That would be something to watch out for. But even if this is unavoidable (and that's not a given), it's better than allowing all DNS queries.


How are Tor packets decrypted on the client?

There is no need to know the key of the relays to decrypt the data they send back. TOR works the in the same way as SSL/TLS does: when you connect via HTTPS (HTTP over SSL/TLS) to a web server, you don't need to know the key of the server to decrypt the data you receive from it, this because during the SSL handshake phase, your browser and the server agreed on a symmetric encryption key that will subsequently be used with a symmetric-key cipher (e.g., AES). TOR works with the same principle, in some way it is like having three SSL connections one inside the other (although it is not exactly that...).


Authentication handshake between two independent apps

JWT can either be signed, encrypted or both. Please use the case of encrypted and signed for secrecy of the App2 user-id (and other details) transmitted via JWT.
Please refer this for more details.
It's recommended to sign and then encrypt but vice versa is also possible.


The client supports old ciphers, but the server not. Is it possible to exploit?

Given how old these ciphers are I could imagine that the client also supports SSLv2 which is broken. Also it will probably accept certificates signed with MD5 or even MD4 or MD2 which are broken too so one should be able to create a faked certificate accepted by the client. That is if the client checks certificates at all because even a few years ago it was still very common that programs did not check the SSL certificate. Thus I think an active man in the middle attack should be possible.


Protection against clickjacking in Firefox without installing addons?

Firefox doesn't have an option to turn of iframes in about:config, see this bug report. It's also not that likely that it will get that option anytime soon, seeing that this issue was first reported 15 years ago.

You could disable iframes without plugins by using custom CSS (iframe { display: none !important; }). This would be done via an CSS file in /~/.mozilla/firefox/<random-id>.default/chrome/userContent.css. This setting should not be overwritable via a websites CSS (but future Firefox versions might of course allow a website to overwrite custom CSS, which would allow an attacker to bypass this).

ClickJacking should not be possible without iframes (although there might possibly be a way via the embed tag, but it didn't work for me).

Personally, I would probably trust NoScript more than this approach, but if you definitely do not want to install plugins, this might be the best way to do it.


Is it better to have a camera hidden or visible?

From security point of view is it better to have camera hidden or visible

Yes.

Hidden and visible cameras emphasize different security values.

Visible cameras provide deterrent value as much or more than recording value:

They may cause less prepared or less dedicated criminals to think twice.
They may encourage actions or routes which benefit the defender (e.g., walking around the visible camera field may force the attacker into the field of another, hidden, camera).

Visible cameras are more susceptible to avoidance or disabling, though, because they are obvious.

Hidden cameras provide improved recording value, in that they can be more survivable than visible cameras.  However, they may have more limited fields of view, and they don't provide any deterrence.

The security decision to go with hidden, visible, or both, should be dictated by the site and the threat.

A convenience store is going to want to emphasize visible cameras, as deterrence is more valuable in that threat environment.  A museum might emphasize hidden cameras, partially because deterrence is less of an issue and partially because obvious cameras detract from the atmosphere they want to provide for their customers.

In all cases, the Digital Video Recorder (DVR) needs to be better protected than it was in this case.  It should be protected well enough that legitimate employees can't tamper with it - certainly the attackers in the case you describe probably knew video stayed local to the site, knew it could be disrupted, and possibly even had "inside information" that allowed them to go straight to it.


Is it possible to prove which public key was used to encrypt a message?

"Proving" depends on whether the recipient (Bob) cooperates (i.e. accepts to reveal his private key to the verifier), and also on the type of cryptographic algorithms and details of the key.

If Bob cooperates, then he may decrypt the message; this may show that the message "makes sense" when decrypted with Bob's private key, which is a rather strong hint that Bob was intended as the recipient, or at least as a recipient. This depends on what qualifies as "making sense", of course. Most asymmetric encryption systems are hybrid: an asymmetric key exchange yields a message-specific key shared between sender and receiver, and the key is then used to encrypt the actual message data. If the encryption uses (as it should) a MAC to detect alterations, then this can be turned into a convincing proof that the sender really worked with Bob's public key in mind (but it depends on the exact algorithms employed).

Note that, depending on the used algorithms, it can be possible for Bob to take an existing message, and compute his own key pair so that the message (as a sequence of bytes), when decrypted with his brand new private key, decrypts to a message of his choosing. Thus, asymmetric encryption cannot, in general, be considered equivalent to an ownership claim.

If Bob does not cooperate, things can be tricky. We are now in the model of traffic analysis, where an attacker tries to see through anonymous communications. In that model, Bob is not a potential attacker, but a potential victim. Hiding the recipient identity is not a primary property of asymmetric encryption and key exchange systems. If we consider asymmetric encryption with RSA, then the key size is leaked, since it matches the encrypted message size; this can reduce the number of possible recipients.

As a more extreme case, consider Diffie-Hellman. DH is computed in a given group, consisting of a prime modulus p, a subgroup size q (q divides p-1 and is normally prime) and a subgroup generator g (g has order q, which means that gq = 1 mod p). These values (the "group parameters") are part of Bob's public key. The message will first begin by a value B = gb mod p, with a randomly chosen exponent b: that value is a subgroup element. For a given asymmetrically encrypted message, it is easy to look at the header, see the B value, and check whether it matches a given (p,q,g) DH group specification: simply compute Bq mod p; for the right group, it will yield 1, but (with overwhelming probability) not for a distinct group.

So if potential recipients use Diffie-Hellman and each one generated his own group, then this simple test will pinpoint the actual recipient. On the other hand, if several recipients decided to generate their respective key pairs within the same group (which is allowed in DH and does not incur any security weakness), then outsiders will not be able to determine which recipient is the right one: doing so would entail solving the Decisional Diffie-Hellman problem, which is hard (we don't know of any way to solve DDH, in a normal DH group, except through Discrete Logarithm, which is thwarted by large enough p and q).

Also, pinpointing is not proving. Anybody can take a copy of Bob's DH group parameters (they are part of Bob's public key, thus public themselves) and generate his own key pair within the same group. Therefore, being able to recognize Bob's group in a given message does not demonstrate that only Bob could read the message; the message could be meant for somebody else whose key works in the same group.

(Having many keys in the same group is a very common occurrence when using the elliptic curve version of DH, because generating your own curve is hard work and severely limits interoperability.)


Strange activity on Apache httpd access log

The first one looks like CGI injection.

Is your PHP updated? Are you sure it doesn't exist in the cgi-bin directory? These two should be enough to protect against this attack....

The second one appears to be targeting some D-Link vulnerabilities. I wouldn't worry about that one on your webserver.


Potential issues with Kim Dotcom's new proposed "encrypted webmail service"

Let's assume here that we are talking about a serious design for emails, i.e. one without a centrally trusted server (we already have Gmail; if we are to trust a potential Evil Overlord, we may as well choose a competent one).

When using a Web site, you use the Web site interface, which can do nasty things behind your back. The same holds for every piece of software, but the Web is especially dynamic: by definition, it is downloaded anew each time you connect. With a locally installed application, you can make some review once (e.g. compiling GnuPG from source); if the application is "clean", it stays clean; and if it was backdoored, then, at least, you may keep a copy as proof for the post mortem analysis. Such things are much harder with a Web site.

Therefore a Web-based solution may cheat on you easily.

Also, we know that key management, including generation and storage, is subtle. Since keys concentrate secret, they are high-value targets, and should be protected accordingly. GnuPG uses mlock() (or its Windows equivalent) to prevent the key from being written to the swap space. You cannot do that in Javascript... Similarly, some cryptographic tasks related to passwords (e.g. password-based encryption of a key ring) kind of need some slow and salted hashing (bcrypt & co) to cope with the inherent weakness of passwords being passwords. Slow hashing requires some muscle. Javascript is very feeble in that respect. And if we want to be serious about key management, then we want smart cards, which again will be hard to access from a Web application.

Therefore I claim that it is very difficult to make a correct Web-based solution for secure emails, even when all the developers are top-notch.

And, of course, how can you know if a given Web site was properly developed, by people knowing their trade ?


Servers/Websites for vulnerability scanning

There are quite a few sites available for this. Some are online, e.g. Google's Gruyere.

As Sam mentioned above, some are designed to be run within a virtual machine like virtualbox or vmware, or even in a container like Docker. bWapp & OWASP's own Juice Shop and Webgoat spring to mind.

If it's a downloadable web app best to ensure that you're VM network is isolated from the host so that you don't unintentionally cause issues for yourself.

In any case here are some lists of legally compromisable sites and apps for learning pursposes.

15 Vulnerable Sites To (Legally) Practice Your Hacking Skills
and
40+ Intentionally Vulnerable Websites To (Legally) Practice Your Hacking Skills

Some of these tools are old, but that shouldn't be a concern as many of the most leveraged vulnerabilities have been around for donkey's years.


Why did Poitras and Greenwald put their batteries in the fridge?

With its insulated walls and rubber door seals, a refrigerator is the most soundproof box commonly found in ordinary living spaces.  If it's running, even better, it provides white noise.  If one was worried about listening devices, then a fridge would be a reasonable and available place to stash them.  And Snowden has alleged the NSA can do lots of things even with a phone that's turned off.

Minor update: Out of curiosity, I put my phone in the fridge, and later in the freezer, with batteries still inserted.  In both cases I was able to call it quite easily - the refrigerator did not act as a Faraday cage - but the ringtone volume was noticeably subdued.  So I do think the fridge in the quoted story was a soundproofing issue, not a Faraday Cage as has been suggested.

I know it's just an experiment of one, but it was a fun experiment :)

Even more minor update: I was curious and checked... my microwave, which is a Faraday Cage, doesn't block cell signals either (good explanation here).


Meta: Doesn't add much to the dupe but on rereading I notice I missed the "D".

Key establishment and encryption/decryption and authentication processes for TLSv1.2 are specified in RFC 5246 TLSv1.2 plus for ECC key exchanges (not here) RFC 4492 TLS ECC as amended by appendix A.7 of 5246.
These processes also apply to DTLSv1.2 RFC 6347 DTLSv1.2, although the record headers, actual transmission and some error handling differs for DTLS compared to TLS.

SSL/TLS/DTLS key establishment has two parts: one which varies based on (only) the key-exchange algorithm used and produces a "premaster secret", and one which varies based on (only) the bulk-cipher and MAC algorithm(s)/mode selected and uses the premaster secret to produce a master secret and then the several working keys, IVs and nonces.
The encryption/decryption and authentication processes vary similarly.

For TLS_RSA_WITH_AES_256_CBC_SHA256 the key-exchange algorithm is RSA. The premaster secret is chosen by the client, RSA encrypted under the server's public key and sent to the server which RSA decrypts it and checks it. See in 7.4.2 the item for RSA, 7.4.7.1, and 8.1.1. To do this outside the server you need (a copy of) the server's private key.

The premaster secret is then used with the nonces to derive the master secret as specified in 8.1 using the PRF (PseudoRandom Function) defined in sec 5; this is then used again with the nonces to derive working keys as specified in 6.3 using the same PRF for the sizes in appendix C. For TLS_RSA_WITH_AES_256_CBC_SHA256 the cipher is AES_256_CBC which requires a key of 32 octets in each direction and the MAC is HMAC-SHA256 with a key of 32 octets in each direction.

Each data record is MACed and encrypted by the sender, and decrypted and verified by the receiver. The MAC/verify process is specified by 6.2.3.1 (for all non-AEAD ciphers/suites, and this suite is not AEAD) and the encryption/decryption process for CBC-mode (block) cipher is specified by 6.2.3.2; it does indeed choose a per-record IV and prepend it to the encrypted record data.

For another ciphersuite the outline is the same but in some places different specific sections of RFC 5246, and for ECC-based key exchange (ECDH, ECDHE, or ECDH_anon) RFC 4492, apply.


That seems to be a NetGear log entry, there are two possibilities for this:

SYN Port Scan: Someone (very likely automated, by an infected machine) attempting to scan your machine. They send a SYN packet to you, then your machine responds with an ACK packet. In order to prevent a connection from being established, they send you an RST (Reset) packet. (More likely)
DoS Attack: Someone (again, very likely an infected machine) attempting to flood you with RST packets. (Less likely)

In both cases, your router's internal firewall is responding by dropping those packets. You don't need to worry about that at all. You're on the Internet, something will always attempt to attack you.

And just to be sure, I've checked the IP address bothering you, it seems to be blacklisted for

  Portscans or hacking attempts were seen against an UCEPROTECT-System

Update: Just to keep the information up to date, I'd like to mention that the IP address is no longer blacklisted, which suggests that the infection is likely to have been cleaned.


XKCD #936: Short complex password, or long dictionary passphrase?

I think the most important part of this comic, even if it were to get the math wrong (which it didn't), is visually emphasizing that there are two equally important aspects to selecting a strong password (or actually, a password policy, in general):

Difficulty to guess
Difficulty to remember

Or, in other words:

The computer aspect
The human aspect

All too often, when discussing complex passwords, strong policies, expiration, etc (and, to generalize - all security), we tend to focus overly much on the computer aspects, and skip over the human aspects.
Especially when it comes to passwords, (and double especially for average users), the human aspect should often be the overriding concern.
For example, how often does strict password complexity policy enforced by IT (such as the one shown in the XKCD), result in the user writing down his password, and taping it to his screen? That is a direct result of focusing too much on the computer aspect, at the expense of the human aspect.
And I think that is the core message from the sage of XKCD - yes, Easy to Guess is bad, but Hard to Remember is equally so.
And that principle is a correct one. We should remember this more often, AKA AviD's Rule of Usability:

Security at the expense of usability comes at the expense of security.


Your text is just very badly written. No wonder you got confused.

1)

  a user may encrypt a portion of a document using his or her private key.

Sounds like a badly written description of RSA signing. With RSA signing you essentially encrypt a hash of the message with your private key.

2) You use the senders public key to verify the signature. And since the signature contains a hash of the whole document, you know it hasn't been tampered with.

3) Is a bit unclear. If you decrypt with plain, unpadded RSA, it will succeed for most blocks, and give gibberish. When using padding you'll get a padding failure. But since this is a nonsensical operation, you don't need to think about that.

The way messages are encrypted in a public key system is:

Sign the hash of the message with your private key
Generate a random symmetric key, and encrypt the message(and potentially the signature) with it.
Encrypt the symmetric key with the receivers public key and combine it with the ciphertext.

Encrypting the key with the receiver's public key, means that only the owner of the associated private key can decrypt the symmetric key, and thus the message.

Signing the whole message with your private key, means that the receiver verifies that the signature matches the sender's public key, and that the hash embedded in the signature matches the document. Thus he can be sure that the whole message is unaltered, and was sent by the owner of the sender's private key.


How do I write my own authentication protocol and add that to OpenWrt?

You might want to consider writing a pluggable authentication module (PAM) that you can use with OpenWRT.  From the Wikipedia article linked above, you can see:

A pluggable authentication module (PAM) is a mechanism to integrate multiple low-level authentication schemes into a high-level application programming interface (API). It allows programs that rely on authentication to be written independent of the underlying authentication scheme.

From the links below, tt looks like OpenWRT supports integration with PAMs:
https://openwrt.org/packages/pkgdata/libpam
https://forum.openwrt.org/t/installing-google-authenticator-for-ssh-openconnect-openvpn/31439


How should privileged accounts be secured in both Linux and Windows?

For Linux, disable root if you don't need it. Use sudo with a properly configured sudoers file. This allows fine-grained control over what users have what superuser permissions (i.e. you can allow sudo only for Nessus). Alternatively, you can set permissions so the account running Nessus has permissions to access the resources it needs (this will be more involved, especially if it involves things like network devices)

For Windows, it's a little more complicated, but if you set up your ACLs correctly, you should be able to run Nessus as a (custom?) service account that has permissions to whatever resources you normally need admin for (so you won't be prompted for passwords).

The password vault idea is dangerous. If you trust your users enough to give them an admin password, they should be admins in the first place (I'm guessing this is not the case).


Why is Pastebin still widely used by malicious actors for sensitive data dumps, given that it's public and easily parsable?

Here are the reasons I know why Pastebin is popular in terms of doxing among Malicious actors :

It can handle large text files
Easy to use
Doesn’t proactively moderate postings
Publishing there doesn’t require registration
Its heritage is rooted in IRC networks

They might be other reasons I'm not aware of. This SE site community should comment/edit this answer or post a better one if it's the case.


How can I keep track of the packages to check any strange behaviour

As far as setting up SNORT goes, here is a video I found on Youtube that may be helpful (it appears to be difficult to set up):
https://www.youtube.com/watch?v=_ic07nBoweg

It appears you would want to at least practice good computer security to make being hacked more difficult.  As far as tracking packages, I am not aware of what the best solution is, but it sounds like what you are looking for would be an Intrusion Detection System, and/or a file integrity checker.  For your situation, I would especially recommend "good computer security" to include at least the following:

Make sure you have a firewall turned on.  Note: you can also configure your firewall to be more secure, but do your research here first if you choose to do so because this can be tricky and can "break" things if you are not careful.  It appears you have already done this.
Make sure you have antivirus software installed and updated and set to update itself, and run a full scan.  Set up a periodic scan schedule if it is not already set up.
Change your passwords (in case you have already been hacked), and change them to long passwords (harder to hack).  Adding numbers and special characters to your passwords helps make them harder to hack too (although some sites don't seem to deal well with special characters in passwords).
Keep your programs up to date (especially internet browsers).  Vendors continue to patch security holes that hackers can use as they are found.
Keep backups of your files (e.g. on an external hard drive).
In e-mail, don't follow links or download attachments unless you know who they are from, and you need to do so.  Verify "urgent situations" with the actual place of business.
Here is an example of a phishing attempt and how to avoid it: an attacker pretending to be your bank sends you an e-mail that says they are going to close your account unless you go to the link and "verify your account".  Do NOT follow the link.  You should first verify that it really came from the place of business (e.g. visit your bank and make sure they really sent that).  Furthermore, in this example, you should report it to the bank as a phishing e-mail once they verify that they did not send it.  Usually, e-mails like that are from scammers / hackers, etc... and you should not do what they say.

There are many more things you can do to increase your security, but this list is a good start (e.g. fairly easy and adds considerable security).


The nonce doesn't have to be random, can also be 0 for example, but it has to be unique in combination with the key

There is no requirement for the nonce to be random or secret for CTR mode. One nonce value is as good as any other potential value. The only requirement is that a nonce is not repeated for a given key.

This is the norm for almost all common algorithms, but check an algorithm's specification to know for sure.

Outside of cryptography, the word might be used differently. WordPress calls its anti-CSRF token a "nonce", despite the fact that they're reused. (And may need to be secret. Double submit cookie tokens need to be unpredictable.)

  XOR-Encryption needs a key with good entropy

I think by "XOR-Encryption" you mean the process by which each bit of plaintext is XORed with a corresponding keystream bit. (And not a "repeating key XOR cipher", which are especially far from being secure.)

No. These type of ciphers need to have an unpredictable keystream, but that doesn't require much entropy in a key. An attacker can only be 50% certain about the value of each unknown keystream bit. Even if they learn other keystream bit values, have a large number of ciphertext bits, or (if relevant) have multiple messages encrypted with the same key.

The reason keystream bits might become known is that a person can guess portions of the plaintext, then XOR it with the ciphertext to get raw keystream bits.

Though for serious stream ciphers, like AES-CTR, learning any number of those keystream bits doesn't aid you in finding the remaining unknown bits. Not unless an attacker can brute force the 128-bit to 256-bit key; an impossible task for securely generated keys.

Do not equate stream ciphers (XOR encryption) with OTPs. The OTP algorithm is one kind of stream cipher algorithms. Stream cipher algorithms in general are not wannabe OTPs.

OTP is just the most trivial version of a stream cipher. You assert that a key is only used once (barring using multiple messages to break the key). And you assert that each keystream bit is statistically independent of one another and that each bit has exactly 1-bit entropy. (Trivially making it so knowing one keystream bit can't help you learn the value of the rest.)

But this comes at the cost of needing exactly as many bits entropy as the length of the message you're trying to encrypt, making OTPs very impractical.

Other stream ciphers are less trivial, but allow you to use smaller keys. You get 128-bit security out of something like AES-CTR, assuming your key has 128-bit entropy. As long as key secrecy is maintained and nonce reuse is avoided, the only thing you have to worry about in practice is the internal counter rolling over. (Simply putting a hard limit on the maximum message length you can encrypt.) ... And, well, software glitches.

It's usually recommended, however, for CTR mode with 128-bit blocks, that you restrict the message length to Gigabytes or Terrabytes. It's also usually recommended that you re-key after encrypting a large number of messages. All versions of AES use 128-bit blocks, even if the key size is larger.


Why do we allow SSL certificates to be replaced before their expiry date, without revokation of others?

There are several normal situations which would entail the existence of several simultaneously valid certificates for a given SSL server:

The server has several front-ends, each having its own private key and certificate, though all certificates contain the server name. Having a per-system private key is a good idea because it avoids moving private keys around, and moving private keys is rarely a good idea.
The private key was lost, but not compromised or stolen; e.g. the machine was destroyed in a fire. A new certificate should be issued for the new server key, but there is no point in revoking the old certificate (see below).
A server certificate will soon expire; a new one is issued. There shall be some overlap during which both certificates are valid, so that the transition can be enacted without any downtime.

Conceptually, the X.509 model is one of positive assertions of local properties. An X.509 certificate says "this key is owned by that entity". There is nothing in X.509 which says "that entity has only x active certificates"; this is not part of the properties that X.509 tries to maintain, and it would actually be impossible to enforce, because nothing would prevent me from building my own CA and issuing certificates for other people. I mean, if I find a certificate issued by some CA, then I can extract the certificate contents and put that in a new certificates which I can sign. I thus can arbitrarily make thousands of new certificates for any server I wish -- and that's not even "wrong" because I am not writing any fake assertion in any of these certificates.

Correspondingly, CRL are not designed to be a way to publish information about any list of "active certificates". A certificate is revoked not because it will no longer be used by the owner, but because verifiers must no longer accept it, which is a very different thing. You do not revoke a certificate when the private key is destroyed; you revoke it when you suspect that the private key is in the wrong hands.

That X.509 might not be the exact right model for HTTPS servers on the global Web is a valid point. That's the gist of Convergence: replacing the X.509 model (positive assertions trickling from trusted authorities) with another model where servers have some contextual state and, in particular, don't switch certificates every five minutes. With X.509, a given SSL server could perfectly have one million valid certificates at the same time, and use a randomly chosen one for each connection; X.509 validation is totally up to it. But "normal HTTPS servers don't do that" and Convergence tries to milk extra security out of that idea of "normal servers".


Is it dangerous to publicly share a scopeless token?

Some issues I can think of are:

Exhausting the request limit, preventing any user's depending on that token from using it
Violating the terms of service agreement by sharing a token which is supposed to remain secret/in the possession of a single user
If the API is subject to such a vulnerability, exploitation of 'session fixation', where another user authenticates or calls a privilege escalation method which upgrades the privileges available to the token without changing it (thereby enabling access to that user's account for all holders of the token)
Some API methods might vary their response based on calls to other methods (or the same method). If you know applications are depending on certain formats for a response (e.g. XML or JSON) you could leverage this to force the trusted end-point (the exposed API) to attack whichever application is using it, because you will be sharing a session.


PRO: strict control over everything that moves out of your network (and by extension, much of what moves in it).
CON: Your users feel the control. Considering that you're a development business, your users are pretty much technical, and would probably resent it.
CON: Considering that you're a development business, and your users pretty much technical, they are likely to legitimately install new programs, in the course of their work (how long was it till skype was allowed through?), and it will be difficult to keep track and update the filters.
PRO: your users are in effect restricted in what they can install :)
CON: Don't even consider this for each machine, while not completely pointless, its much easier to bypass.
PRO: As you mentioned, it can block certain usages of botnets and other unwitting malware.
PRO: It can hypothetically assist (as a first step) in data leakage prevention.

UBER-CON: It is trivial to bypass in most malicious scenarios by simply tunnelling over any ports/protocols you DO have open (this includes many types of adaptive worms), whereas in the non-malicious cases it's a real **tch to deal with.

Bottom Line: Egress filtering is a very good tool to give you control, but not very good for security.


Nmap scan list of domains and provide boolean if up/down

Consider this command:

nmap 127.0.0.1 -T4 -p 111 -sT -oG - | awk | awk '/^Host: .* Status:.*Up/ { print $2" UP" }'

A common pattern using shell scripting is to use grepable output.

Use the flags "-oG -", the final dash is to redirect output to stdout.

Output from "-oG -" in this example outputs the following when I ran it at my own server:

Host: 127.0.0.1 (localhost) Status: Up

The output is redirected to awk, where it searches for strings starting with "Host:, and contain open ports.

When this pattern is found, awk will print:

127.0.0.1 UP


The Walmart clerk takes pictures of my IMEI # and Phone numbers

Get a new phone, seriously. If someone has your IMEI they can authenticate with your IMEI to whatever cellular provider (this is not easy, but provenly possible with the required skills). If your IMEI is used for any kind of illegal operation and police start investigating, this can get you into deep trouble. E.g. someone threatens to blow a bomb using your IMEI. It is very likely that police discover the fraud in such a case but i'd not want to get into something like this.


UPDATED

I cannot answer for every ISP or even most of them. I'll look at what they could store and talk about reasons you'll never really know what they actually store.

Bottom line: If this is a worry for you, then assume that yes. Your plaintext traffic is stored. And take measures to protect it.

TL;DR

What's an ISP

Broadband provider
Cell service provider
Coffee shop hotspot
Your ISPs upstream ISP
Your work ISP

What they can know.

IP Address - this is the connection information and required to make the internet work.
DNS requests - these are sent in plain text and many ISPs log this information.
HTTPS SNI - this is the subject name identifier information that's sent in plaintext when establishing an HTTPS connection.
They know who you are and where you are.
Any other plaintext traffic (eg http).
Amount of transferred data.

What they can store, but can't utilize:

HTTPS traffic - other than the SNI. It's encrypted and I don't know why they would store this.

Do they log it?

Probably. It's useful for making their service better, and it's easy money for them to do so. This depends on your TOS; however, I wouldn't trust your TOS.

An ISP may claim not to sell your data or to de-identify the data. But their upstream providers might. De-identification is not guaranteed to be effective. They may not understand they are selling it. They may be giving it away, storing it, or analyzing it to "improve your service," or "enhance your security."

GDPR is a great lever to feel safer, but the truth is logs are hard and mistakes happen.

Top google result:

Ars Technica Writeup

Summary

You use the internet in so many places that you should assume that someone's ISP is logging the information. Sure, they may claim to de-identify the data, but it's not guaranteed to be effective because it's hard to do right. They may claim to keep the data to better the service, improve performance, and the like. Often they sell the data to third parties that help them improve security.


Is it a bad idea for a firewall to block ICMP?

Compared to other IP protocols ICMP is fairly small, but it does serve a large number of disparate functions. At its core ICMP was designed as the debugging, troubleshooting, and error reporting mechanism for IP. This makes it insanely valuable so a lot of thought needs to into shutting it down. It would be a bit like tacking >/dev/null 2>&1 to the end of all your cron entries.

Most of the time when I talk to people about blocking ICMP they're really talking about ping and traceroute. This translates into 3 types

0 - Echo Reply (ping response)
8 - Echo Request (ping request)
11 - Time Exceeded

That's 3 types out of 16. Let's look at a couple of the other ICMP type that are available.

4 - Source Quench (send by a router to ask a host to slow down its transmissions)
3 - Destination Unreachable (consists of 16 different kinds of messages ranging from reporting a fragmentation problem up to a firewall reporting that a port is closed)

Both of which can be invaluable for keeping non-malicious hosts operating properly on a network. In fact there are two (probably more but these are the most obvious to me) very good cases where you don't want to restrict ICMP.

Path MTU Discovery - We use a combination of the Don't Fragment flag and type 3 code 4 (Destination Unreachable - Fragmentation required, and DF flag set) to determine the smallest MTU on the path between the hosts. This way we avoid fragmentation during the transmission.
Active Directory requires clients ping the domain controllers in order to pull down GPOs. They use ping to determine the "closest" controller and if none respond, then it is assumed that none are close enough. So the policy update doesn't happen.

That's not to say that we should necessarily leave everything open for all the world to see. Reconnaissance is possible with ICMP and that is generally the reason given for blocking. One can use pings to determine if a host is actually on, or Time Exceededs (as part of a traceroute) to map out network architectures, or Rory forbid a Redirect (type 5 code 0) to change the default route of a host.

Given all that, my advice is, as always, take a measured and thoughtful approach to your protections. Blocking ICMP in its entirety is probably not the best idea, but picking and choosing what you block and to/from where probably will get you what you want.


What does the browser keep, the CA copy or CA public key?

If browser uses public key of CA, why does It keep complete certificate?

When creating the trust chain from the servers leaf certificate to the local root CA certificate the browser the next certificate in the chain is found by looking for a certificate where the subject matches the issuer of the last certificate in the chain. Therefore it is not enough to only have the public key of the root but one needs also the subject of the root certificate.

  and how browser decrypt key from CA certificate? isn't encrypted?

No, the certificate is not encrypted and contains the public key in clear. The main idea of public key cryptography is that the public key is is public and only the private key needs to be secret. Therefore the public key can be contained in clear in the certificate.


How Google Detects State Sponsored Attacks

Google states on their security blog that:

  These warnings are rare—fewer than 0.1% of users ever receive them—but they are critically important. The users that receive these warnings are often activists, journalists, and policy-makers taking bold stands around the world.

And:

  We can't reveal the tip-off because these attackers will adapt, but this happens to less then 0.1% of all Gmail users.

So I think we can only speculate about how they do it, but you shouldn't forget that google has a lot of information on a lot of people, and enough serverpower to make feasible predictions.

Source: https://security.googleblog.com/2016/03/more-encryption-more-notifications-more.html


real time

Looking at the signed message, the reason gets very obvious. gpg --list-packets takes the input and lists all packets contained in a somewhat readable fashion:

$ echo "foo" | gpg --sign | gpg --list-packets

[gpg asking for passphrase]

:compressed packet: algo=1
:onepass_sig packet: keyid 8E78E44DFB1B55E9
    version 3, sigclass 0x00, digest 2, pubkey 1, last=1
:literal data packet:
    mode b (62), created 1421012528, name="",
    raw data: 4 bytes
:signature packet: algo 1, keyid 8E78E44DFB1B55E9
    version 4, created 1421012528, md5len 0, sigclass 0x00
    digest algo 2, begin of digest 96 e3
    hashed subpkt 2 len 4 (sig created 2015-01-11)
    subpkt 16 len 8 (issuer key ID 8E78E44DFB1B55E9)
    data: [4096 bits]

The "literal data packet" and also the "signature packet" contain the creation timestamp, here 1421012528. The actual signature data also takes the literal data packet's creation timestamp into account, thus will also be different for every signature calculated.

faketime

To verify this, use the glorious faketime program and calculate the checksum of the signature (which is always the same, no matter how often you run the command):

faketime 5pm /bin/bash -c "echo "foo" | gpg --sign | sha256sum"


How secure is setting mongodb to 0.0.0.0 but blocking all incoming requests to 27017 on ec2?

A few thoughts in no particular order:

It is as secure as your (or your companies) ability to leave the firewall configuration unchanged and not come back later and open it up to the whole world because, "This is inconvenient".
It is as secure as your machine is.
It is secure until your IP changes, you forget to update the firewall rule, and your current IP is reassigned
It is secure until someone is doing something else and accidentally changes the configuration without realizing it.
It is not as secure as putting the instance in a private subnet and connecting through a bastion host
It is not as secure as a database server that has no internet access at all and is sunk to the bottom of the sea after being encased in 10 feet of solid concrete

The key is to accept the fact that there is never perfect security, understand the risks posed by your chosen configuration, and find the right balance between security and usability for you.


Can Apple Devices Give Away Location to Hackers?

Theoretically, yes they could track you, provided your phone was turned on.  There are actually several potential ways for them to do it, but they vary in terms of the level of sophistication (i.e. how technically advanced they are and the amount of money they have to spend).

For a more technically sophisticated attacker, the protocol which mobile phone use, Subsystem number 7 (SS7), has known vulnerabilities which could allow an attacker to locate you.

If they have access to your iCloud account, they could potentially use the device location feature to find your location.

If they have already compromised your phone/laptop, a VPN won't do you any good.  The attacker could potentially access your phone/machine, see what wifi networks are nearby, and use that to geolocate you.

Travelling with that new iPhone/macbook will only help if the attacker is not able to link you as an individual to those new devices.  How easy it is to link you to a device is going to depend on the sophistication of an attacker.


Does data inside a VPC needs to be encrypted in trasit too?

In general, you use a protection mechanism when there is a threat that you want to counter. So, if there is a threat within a cluster or between RAM and mounted volumes, then, yes, you should consider encryption.


What's the down side of a dynamic numbered radial keypad?

There is a Security User Experience (SUX) downside, which you might consider to be minor.

As someone who is more kinestheticly inclined, I don't memorize things like phone numbers or PINs: I memorize patterns. If I was forced to use this keypad, I would have to use a compensating method to remember the actual digits (like writing it down).

While not everyone is strongly kinesthetic, it is a factor in how people learn, which means that others might need to write down the passcode, too. This, of course, defeats the purpose of having a dynamic keypad.

As I say, it might be a minor point, but it is a downside that impacts security.


Can all internet traffic from a smart phone be encrypted?

Using a VPN to tunnel all internet traffic will encrypt the traffic until it gets to the VPN servers, after that it goes over the internet as protected as it would be without a VPN (possibly unprotected). This means that it will help secure you against local tracking (for example people listening in on the hotspot you are connected to). But it will not help against tracking on websites with banners or scripts (which is also a whole other conversation).

In order to choose which VPN to use for encrypting all data coming from your phone I would recommend having a look at this site, which compares a whole lot of VPN providers.

Tor is also an option to encrypt outgoing traffic but it is probably slower than a VPN. The data gets decrypted at the exit node and travels on from there (possibly unprotected).
This means you entrust the exit node with your internet traffic.


Should updated CRLs be published even if no new certs have been revoked?

Yes, CRLs must be reissued periodically in all cases. Even if no certificates were revoked. This is because CRLs have validity period with specific end of validity date determined by Next Update (or NotAfter) field. You must publish new CRL prior to that time.

If the CRL is expired, certificate chaining engine will reject that CRL and report RevocationOffline error.


Did a virus like this really ever exist?

I can't say that this specific virus existed, but it is not surprising at all.
Once the Internet burst on the scene in 1995, people were trying to figure out what to do with all these new tools at their disposal. There were all kinds of hobbyists and tinkerers who would explore every avenue of possibility, simply because they could.
It wasn't so much about trying to be evil, but more about exploring what kind of evil was possible. You ended up with all kinds of malware that did all nature of strange things. Most of what I saw didn't have any specific point but to determine if some idea was feasible. A lot of it was, indeed, quite clever.
And because there was no cost to create and experiment, and the Internet made a perfect, anonymous testing ground, there were a lot of tinkerers.
So, yes, the example virus is indeed possible and yes, someone may have spent months tinkering with it to see what would happen. But the point was likely not to "waste their vast technical skills and large brain capacity on something so pointlessly evil" but to enhance their skills and get smarter. Evil was just a byproduct.


How does UEFI Secure Boot prevent "evil twin" spoofing attacks?

The attack you describe sounds more like a "diamond heist" attack of sorts (a bit elaborate for the everyday criminal). Before I address UEFI, I wanted to answer if that is feasable. No, not really. If we are talking about a server running a VM, the end user may not notice a performance issue (given enough physical resources) but more than likely the monitoring software in place would detect a new process running and a system administrator would respond. A good sys admin would hopefully be able to tell once he investigates.

If it were a desktop user, he would most likely notice very quickly when he reboots his machine and it looks a bit different. Assuming the VM could spoof the old desktop, the hardware installed, and not display information about the VM running, he may be fooled.

But I think the question was more about the technology than the individual, so my thought would be yes, it appears this type of attack would work for the reasons you stated: the host OS is trusted and secure boot loaded that OS. The VM process could be hidden inside of another process (svchost, maybe?) and both user and monitoring tools could be fooled (for a time).

Your last question (what it does and doesn't guarantee) I found some information at the home page http://www.uefi.org/home/ for UEFI. The answer to the question, "What is UEFI" states:

  UEFI (Unified Extensible Firmware Interface) will be a specification
  detailing an interface that helps hand off control of the system for
  the pre-boot environment (i.e.: after the system is powered on, but
  before the operating system starts) to an operating system, such as
  Windows* or Linux*. UEFI will provide a clean interface between
  operating systems and platform firmware at boot time, and will support
  an architecture-independent mechanism for initializing add-in cards.

From this answer, I believe you are promised nothing. UEFI is a framework that vendors and coders can implement. I believe it would be up to the vendors and coders to make the guarantees.


Transparent pixel in nginx error log...Is this hacking?

Someone sent a request asking your server to fetch that image from Amazon, and your server interpreted it as a local file which it then failed to find (which is a good thing).

They were probing for an open proxy, which would return any requested URL to them.  This would let them visit any web site and as far as the site could tell the traffic would all come from you (although proxies and the true originating address can often?sometimes? be detected).


JAVA (Web Application) Public Data Assigned to Private Array-Typed Field

The issue is that the private data of the object becomes a reference to the data which is passed as parameter in the setter (which is a public one).

Arrays in Java are passed by reference (See this question), so that means that :

public void setUserRoles(String[] userRoles) {
    this.userRoles = userRoles;
}

String[] param = ...;
User.setUserRoles(param);

Actually puts the reference of the param variable into the private member. If someone were to modify this param variable afterwards, it would modify the private member as well. The problem in this case is that the array is mutable as you can see in this SO question.

This behaviour defies the principle of encapsulation and therefore is wrong.

Finally, this has nothing to do with how many instances of the object you can create. It's a problem that is related to the object itself (so all object you would create would have the same problem).

A solution to the problem would be to make a full copy of the parameter array into the private member array (using clone() for example).


How can I enable opportunistic encryption for my web-site?

None of the mainstream user-agents (IE, FF, Chrome, etc.) support either of those proposals, so there's currently no way to do opportunistic HTTPS.  Why not just serve all of your content over HTTPS to everyone?  Seems like that's the best way to offer your users confidentiality and integrity.


How can memory problems lead to code execution exploits?

The memory a program uses includes both what you would think of as "data" (text you wrote, an image you're viewing, the script source code your web browser downloaded, a string of text which is being built by that script...), what you might think of as "metadata" (a mapping of free and used memory blocks, pointers to the start of those memory blocks, pointers linking that text string to the functions that can be executed on it, pointers to loaded libraries such as IE's jscript.dll that parses and compiles JavaScript), and executable code (the actual binary code the CPU runs, including the DLL contents, the result of compiling the JS, and the functions on "objects" in data).

When memory corruption occurs, an attacker can usually exert some control over where and how it occurs. For example, if the attacker finds a way to cause some data to be deleted but the program to treat it as still present (called a "use after free" vulnerability, and relatively common in JavaScript engines), the attacker's script could then create a new data object (such as an array of carefully-chosen bytes) that is a similar size and will be created in the same place as the deleted-but-still-usable object. The attacker then might set some value on the deleted object, but because the memory that defines the object holds some totally new data now (the attacker-crafted array) it actually changes some value that the attacker isn't supposed to be able to control such as the extent of the array. The attacker then uses the array - which the program now thinks is super-long, extending well beyond the allocated chunk of memory that it is actually stored in and into memory used for other things - and does something like change the "return pointer" metadata that says "when this block of code finishes, continue executing code over here". The attacker changes this return pointer so that the code that will run after the current function ends does something like launch a program the attacker wrote, executing arbitrary code on the victim's computer.

This type of attack (use-after-free to gain an arbitrary-offset memory read/write, using that to overwrite the return address into some attacker-desired function) is but one of many, many ways to exploit memory corruption. Some other memory corruption vulns are double-free (trying to delete the same block of memory twice, which is dangerous because you may end up treating attacker-controlled data as though it is actually metadata about the memory allocations and thus change data the user shouldn't have access to), buffer overflows on the stack or heap (two different places program data goes; the stack is where you'll also find things like return pointers and the heap where you'll find things like function tables and arbitrary-length arrays) where you can write past the end of a memory allocation and change other data, integer overflow where the program does some math on two numbers that ends up with a result too big for the program to hold in the space it reserved for that number, so it loses part of that number and treats the totally-different number as a pointer into safe memory when in fact it could be anything at all, and so on.


Drop the -sn flag to let nmap do a port scan or specify a port.

The robots.txt is specific to a HTTP service running on a given port. Without checking ports, nmap is unable to identify HTTP services where it could apply the http-robots script.

From the docs:

  With the -sn option it is possible to run a script scan without a port
  scan, only host discovery. In this case only host scripts will be
  eligible to run.

Drop the flag and you're fine:

$ nmap -Pn --script http-robots.txt.nse nmap.org

Starting Nmap 7.31 ( https://nmap.org ) at 2016-11-06 17:13 CET
Nmap scan report for nmap.org (45.33.49.119)
Host is up (0.21s latency).
Other addresses for nmap.org (not scanned): 2600:3c01::f03c:91ff:fe98:ff4e
rDNS record for 45.33.49.119: ack.nmap.org
Not shown: 991 filtered ports
PORT      STATE  SERVICE
22/tcp    open   ssh
25/tcp    open   smtp
70/tcp    closed gopher
80/tcp    open   http
113/tcp   closed ident
443/tcp   open   https
| http-robots.txt: 4 disallowed entries
| /favicon/tiles/ /favicon/dim/ /favicon/dim-nogzip/
|_/mailman/
554/tcp   open   rtsp
7070/tcp  open   realserver
31337/tcp closed Elite

If you know which ports you're looking for you can skip the enumeration:

$ nmap -p 80,443 --script http-robots.txt.nse stackexchange.com


Would DNSSEC and certificates for IP addresses be enough?

Domain names are a different addressing scheme than IP addresses: There might be  multiple IP addresses for the same domain name and there can also be multiple domains on the same IP address. And these domains can serve different contents too, even if they have the same IP address (at least with HTTPS).

Only if there is a 1:1 relation between IP and domain name you could replace one with the other, i.e. connect to some domain by lookup up the IP address and then check if the IP address is contained in the certificate instead of looking for the domain name. But neither plain DNS nor DNSSec provide you with the information if this is a 1:1 mapping. Even if you get only a single IP address when looking up a domain it might still be possible that you get the same IP address when looking up other domains.

Because of that you have to check the subject/SAN of the certificate against your original expectation (domain name) and not against some derived expectation (IP address).


What are the disadvantages of a partial encrypted file system?

Assume an attacker Grace who has physical access to your machine.

She doesn't know your password and wants access to the data stored on the encrypted disk. You will only type in your password if the machine seems not to have been tampered with. Plugging a device which stores everything you type on your keyboard between the keyboard's USB cable and your computer or hiding a microphone which records what you type (yes, that's actually possible) is risky because those modifications can be found.

So Grace decides to go a different path. She assumes that you won't swap out your read-only disk (bwt.: I hope it's a read-only disk because if it's only a read-only FS like you describe in your questions, you have a lot more to worry about) and on day α copies your read-only disk onto a disk of her and also writes down which read-only disk you're using.

She then figures out what program receives the password when type it in and modifies it so that it's stored in a place of the encrypted hard disk which probably isn't accessed during normal usage if the password is correct. She then swaps out the program on the copy of the read-only disk for her modified version and copies the FS to one of the same model of read-only disks you're using and makes it unwritable.

On day β, she takes her read-only disk to your computer and swaps your disk out for hers.

She then just waits until you use the computer and subsequently on day γ copies the contents of your encrypted disk.

Afterwards, she can decrypt everything.

This would not have been possible had you only used a single encrypted disk.


If you renew (same key, same name) the Root CA certificate then the leaf certificates will still validate.  A certificate's identity is defined by its key and name, and if neither change then it's effectively the same certificate.  All a renewal does is change the validity period of the original certificate.
Contrast the above with a re-key (new key, same name) where the identity has now changed and you'll need to resign the leaf certificates (or preferably just the online CA you have in place for these sort of situations).


ShellShock vulnerability and Java Web Applications

This is very difficult to answer, as no one here does know how your application is built.

As long as you are not executing (either directly or indirectly) any system commands your application should be save.

There is only a risk when you execute commands (either directly or indirectly) in the shell out of your application.

This risk is very high if you execute shell commands (either directly or indirectly) that your users are entering as then your system is vulnerable.

Executing shell commands out of a Java web application is very unusual and not needed in almost all cases.


Accessing iPhone data without passcode - how difficult?

Theoretically, an iPhone can be configured to encrypt all the "user data" with a key that is ultimately derived from your passcode. However, that does not really save you:

If the attacker limits itself to the unmodified iPhone and just enters some potential 4-digit passcodes on the screen, then the attacker has a probability 1/1000 of success (10 tries, 10000 possible codes). That the data is encrypted is irrelevant here.
If the attacker opens the iPhone case and dumps the Flash chip contents, then he can try all possible 4-digit passcodes; there are only 10000 such codes and each try will need much less than one second on a basic PC, so he will be done within less than 3 hours. Since, in that scenario, the attacker works on the data dump, not with the phone itself, there is no automatic erasure. The encryption layer does not save you; at best, it delays a bit the attacker, but not by much (3 hours instead of immediate access). Note also that whether the encryption layer was done properly is questionable.

With a 4-digit passcode, your safety relies on how hard it is to physically open the case and access the data directly from the Flash chip. It will deter most phone thieves, who, after all, wanted the phone, not your data. However, a determined attacker will be able to do the dump. If your attack model includes such a determined attacker, then:

you need to activate encryption;
your passcode must have a lot more entropy than a 4-digit passcode (12 digits would be a minimum, more are better);
you still have to pray that Apple did not goofed up with the encryption.


Which of the TLS/SSL and IPsec protocols is effective?

Both will do what you need effectively.

Whether you need to use one over the other depends on your use case. From what you have described, as long as there aren't other requirements, I'd go with TLS, as it requires very little additional work on most network architectures.

IPSec is just a little more convoluted to set up - but it does provide additional security options.


Can a PC hotspot signal be monitored for website URL address and content?

Most people use their personal hotspot because there is no way for their employer to see their traffic. I'm assuming that's the case here.

The hotspot becomes its own network, which means you have no legal or inexpensive way to break into it. "Stingray" is a device that is designed to do what you are looking for, but it is limited to law enforcement.

The only option here is to install something on the PC.

If you are the employer and not the employee (I assume it's actually the other way around) if you have endpoint protection software already installed, you might be able to prevent the use of hotspots, or simply pull the user's browsing history from the machine.


Securing REST API for use by mobile apps

your API may not be as public as you thought. One can sniff all of the traffic from the mobile and get all of the data sent from the mobile app to your server.
even with htpps or basic auth, one can sniff the data.

https://en.wikipedia.org/wiki/Man-in-the-middle_attack


Why does pinning a CA root certificate not present a security risk?

TL;DR: No. That's not how certificates or pinning work.

All certificate pinning does is limit the root of the trust chain to a smaller set, for a specific domain. It doesn't change how the certificates work. It only adds a new constraint on acceptable roots, without disabling any others, and they all have to match -- domain name, date, etc., and now pinned root.

The only way a MITM like you describe could occur is if someone managed to get a certificate, through Amazon, of your domain. Such a thing is technically possible, of course, because people run the systems and people are fallible. However, if anything, pinning a cert makes that attack harder. Without it, they could attack any trusted root certificate. With cert pinning, they have to attack the one(s) you've pinned.


Can one prevent stack overflow by storing strings in memory backwards?

If your code doesn't protect from writing into adjacent memory spaces, then no you can't reverse the order and be protected from a buffer overflow. Reversing the way the string is stored just changes the shellcode payload arrangement (i.e. the payload may come first and the NOPS [x\00] may go in last). Like this:

  x\DEx\ADx\BEx\EF\x00x\00x\00

vs

  x\00x\00x\00x\DEx\ADx\BEx\EF

As you mentioned, there are much better coding practices that protect against buffer overflows.


What is the point of using stronger hashes?

Cryptographic hashes do not exist in an independent world. They have actual use cases and for most of these use cases it is not important that a hash cannot be decrypted. But it is important that one cannot construct two inputs with the same hash (collision) or a different input for an existing hash (preimage attack). A very important use case is for example the use of cryptographic signatures as proof of a (trusted) origin when digitally signing documents, certificates, programs ... .

For example a preimage attack would allow to create a faked document, certificate or program which is different to the original one but as much trusted as the original one since the signature still matches. In these cases stronger hashes provide better protection against collision and preimage attacks.


How and when is a MAC sent across during a TLS connection?

Is a MAC sent only during the handshake (the async bit) or is one sent with each piece of transferred information (both the async and sync bits)?

Both.

The handshake is MACed via the PRF (pseudo-random function).

Each record is MACed by the TLS-MAC. (Unless you're using one of the fancy new AEAD cipher suites like GCM, which take care of the MACing themselves.)

When a client talks to a server, does it always send a MAC?
When a server talks to a client, does it always a send a MAC?

Yes. Direction does not matter. (Only cipher suite type: One of Stream/Block/AEAD. And stream and block do have the external TLS-MAC and AEAD does the MACing itself.)

In regards to TLS, is a MAC created via EtM, E&M or MtE? In short, is the MAC sent as plaintext?

The default case is to use the MAC-then-encrypt. And this is not really regarded as hip anymore. You can switch to Encrypt-then-MAC only via a newish TLS extension

If a MAC doesn't match-up is the connection terminated? Does the whole handshake process have to begin again?

Yup. And this is quite reasonable if you are communicating over a protocol like TCP that is supposed to guarantee integrity against things like line-noise. This bad_record_MAC alert is always fatal and will abort a session.


Passing a hash of a secret to a server instead of the plain text

No, this approach is not widely used and here's why:

If the client hashes the password and sends this to the server, then doesn't that hash become "the password"? As an attacker, can't I just intercept the hash and use that in a replay attack?
You said "A salt can be added also". This one actually is done in some protocols, when you add in a random value to make the hash different each time, it's called a nonce.
Most importantly, the server knows the plaintext password!? Bad Idea! If your server knows the plaintext password, that means it is storing the plains in a database, that means a database breach leaks all your users' plaintext password. Only storing password hashes in the db protects you against this. Are you really willing to risk a total and complete breach in order to make your protocol a little bit simpler?

Bottom line: I'm happy that you're thinking about protocol design from a security perspective. I would encourage you to have a look at the TLS handshake because it is a very well thought-out protocol that uses a lot of the concepts that you're thinking about.