Are there any established secure coding guidelines available for mobile devices?

The issues are mostly the same as for secure software development in general.

Note that the most critical issue for secure software development is (1) integrating security into your software development lifecycle (so security is integrated into each step of the process: design, implementation, maintenance, ops), and (2) training of developers.  It is a mistake to think that a software library or framework will magically ensure the security of your systems.

Mobile-specific issues. That said, here are some issues that are worth special attention in the mobile space:

Use secure communication channels.  Be wary of HTTP or other unencrypted communications from the mobile app to servers elsewhere.  Mobile phone users often use open Wifi networks for network connectivity, and those are at high risk for eavesdropping and man-in-the-middle attacks.  Therefore, if you are sending anything confidential or security-critical at all (personal information, session cookies, authentication tokens), you should be using HTTPS, not HTTP.  For custom network protocols, use TLS.

In general, if in doubt, be skeptical of any use of cleartext communications.  Several high-profile apps had vulnerabilities of this form (they sent session tokens over cleartext HTTP); don't make the same mistake.
Store security-critical data securely. If you have a security-critical app, be wary of storing highly sensitive data on the mobile phone.  For instance, you wouldn't want to store online banking passwords or that sort of thing on the mobile phone, and you might want to be careful about things like credit cards -- because if the mobile phone is stolen, that information becomes accessible to the thief.  Theft and physical security are especially relevant, as it is so easy to steal, lose, or borrow a mobile phone.
Be careful about where to store application data. When storing application data, be careful about where you store it and what permissions you use.  You don't want to inadvertently make sensitive information available to others apps.  For instance, on Android, apps can store app-specific data in internal storage, on the SD card, or in a Content Provider.  These have different security implications.  Internal storage is secure by default (no other app can view the data) and is a good standard choice.  The SD card is visible to other apps by default, and thus you need to use more care.  If you store data on the SD card, at least set the file permissions so other apps can't read it, unless the data is intended to be shared.  Content Providers can be protected using Android permissions.
Be respectful of user privacy. Don't do things that will creep your users out, if they find them out.  Be up-front about what you app does.  For instance, don't grab a copy of the user's address book and upload it to your servers (even if you have good intentions, like helping them connect with their friends) without user consent.  Don't upload photos to a server behind the user's back.

One technique, when dealing with unique identifiers like device IDs or email addresses or phone numbers, that sometimes helps is to hash these identifiers before letting them leave the mobile phone.  Don't just upload them to your server.  Instead, hash them together with some app-specific value, then upload the hashes.  In some cases, this can help improve user privacy.
Static analysis. Consider using static analysis tools that know about mobile-specific risks.  For commercial tools, Fortify has a good reputation for this.  For free tools, Comdroid and Stowaway check for a few risks that are specific to Android applications.  However, beware that the free tools are not as comprehensive or polished as commercial offerings like Fortify.

Know your platform. Educate yourself about security risks specific to your particular mobile platform.  For instance:

On Android, watch out for Webview related vulnerabilities.  If you use webviews in your application, and you expose any of your application via the Javascript interface (e.g., webview.addJavascriptInterface(), webview.setWebViewClient()), then there are subtle security risks.
On Android, watch out for Intent-related vulnerabilities. Android Intents are great; they allow for flexible cross-application communication.  However, they also introduce new security risks.  In particular, they share many of the potential risks of other communication channels: eavesdropping, spoofing, and man-in-the-middle attacks.

To avoid these problems, you know the risks.  When using Intents for communications internal to your app, use only explicit intents (not implicit intents), and make sure all your intent receivers are internal only and are not exported to the world (beware that under some circumstances, Android will implicitly export components; you want to avoid that for ones which might receive your internal intents).  When using Intents for cross-application communication, beware that Intents can be injected and eavesdropped if you are not careful.  Therefore, when receiving an Intent that may have come from another application, treat its contents as untrusted and apply proper input validation/sanitization; you may want to check the sender of each Intent you receive, too.  When sending implicit Intents that could be received by other apps, beware that they can be intercepted by malicious apps, so you should avoid including any sensitive or confidential information in the Intent.  Avoid mixing internal communications with external communications.

For an introduction to Intent-related vulnerabilities, read here and here.  More technical details can be found in the technical paper.
On Android, avoid permission re-delegation attacks. A permission re-delegation vulnerability is one where your app has a permission P, and is willing to respond to requests from other apps by performing operations that require permission P.  If the other apps don't have permission P, you've just leaked the privileges associated with P to those other apps.
On Android, beware SQL injection vulnerabilities. On Android, Content Providers can be accessed through SQL queries.  You'll want to avoid classic SQL injection vulnerabilities.  In addition, if you build your own Content Provider and implement your own methods (like delete, execSQL, rawQuery, update, updateWithNoConflict) to handle queries on the Content Provider, be careful when implementing those methods to avoid inadvertently introducing SQL injection vulnerabilities -- experience has shown that developers often introduce problems when implementing those methods.  The best way to avoid these problems is the same as on non-mobile platforms: use parametrized queries (prepared statements).

Other resources.  For Android, I highly recommend the following presentation:

Seven Ways to Hang Yourself with Google Android (slides)

For mobile security in general, I also recommend looking through the following questions:

Android API/development security pitfalls
Secure Software Development
Vulnerabilities common only in mobile solutions?
Top Mobile Device Security Concerns?
Any useful tools for Android source code review?
Do mobile OS's provide crypto-quality randomness?


The ASN.1 structure we are talking about here is:

DigestInfo ::= SEQUENCE {
    digestAlgorithm  AlgorithmIdentifier,
    digest           OCTET STRING
}

AlgorithmIdentifier ::= SEQUENCE {
    algorithm        OBJECT IDENTIFIER,
    parameters       ANY DEFINED BY algorithm OPTIONAL
}

When encoded in DER, the actual hash value (the raw bytes) will be located at the very end of the encoded structure. So the DER encoding of the structure, for a given hash function, is equivalent to concatenating a fixed string of bytes with the hash value. The string of bytes for various hash functions are given in extenso in PKCS#1 (section 9.2, page 38 of version 2.1 of the standard).

So, on a Unix-like system, to get from hash value contained in file hash-value.bin to the DER-encoded structure specifying SHA-256, you would do this:

(printf '\x30\x31\x30\x0d\x06\x09\x60\x86\x48\x01\x65\x03\x04\x02\x01\x05\x00\x04\x20' ; cat hash-value.bin) > encoded-hash.der

The Devil hides in the details, and ASN.1 is so much full of details that it should rightfully be burnt at the stake. In this case, notice how the parameters field is marked OPTIONAL. In a much older version of another standard, a typographic error resulted in the omission of that keyword; hence, some implementers found it that they had to include some parameters, even so the OID was enough to designate the hash function. Hence, they added an ASN.1 NULL value as parameters (it shows as the 0x05 0x00 bytes in the encoded headers). However, some later implementers found it fit to "correct" the issue and omit the unneeded parameters altogether, as is permitted by the OPTIONAL keyword. The bottom-line is the following: for a given hash function, there are two versions of the "fixed header string" to be concatenated with the hash value. For instance, for SHA-256, PKCS#1 says that the header is:

30 31 30 0d 06 09 60 86 48 01 65 03 04 02 01 05 00 04 20

but one may also encounter this alternate header:

30 2f 30 0b 06 09 60 86 48 01 65 03 04 02 01 04 20

which some implementation use. According to PKCS#1, you shall use the first one when producing signature, but you also shall accept both when verifying signatures.


What does Zerodium do with their bought exploits and bugs?

Zerodium is not "well-known for their big bounties", but they are well-known for aggressive marketing. They are actually fairly small players in the whole exploit broker market, and they pay up far less than they advertise. In general, companies like these will buy weaponized exploits and resell them, usually to governments which use them to, among other things, exploit and track dissidents or even journalists.

Note that the price they list is the maximum payout. The vast majority of exploits will be bought for a far lower price. It is more likely to align with the price they sell it for, as they need to make a profit as ethically-challenged opportunists. As governments have ample money from tax payers, they can afford to pay non-competitive prices and ensure that Zerodium and their ilk sell preferentially to them.

The 0day industry is composed of a number of different companies, some of which are explicitly government contractors, as well as private individuals who buy and sell exploits. Generally, you give them a weaponized exploit and they pay you slowly over a period of time. If for whatever reason the exploit is patched before they have finished paying out, they will stop paying you. This provides an incentive not to use up and burn the exploit on your own. Exploit brokers often know each other and word about which vulnerabilities are in possession travel fast, which itself makes it difficult to try to sell the same vulnerability to multiple different companies. This is true of most exploit brokers.


Processing sensitive user data on a server without the possibility of data theft?

If I understood correctly, no. You're logging in and retrieving data on behalf of your customer.

The best you can do is use delegation, as Adam Luchjenbroers commented, but even then you would still have access to the user's data (albeit not their login). If this is acceptable then Adam supplied the answer. If not, uses will have to take you at your word.

If the third party service supports encryption and you do not need to actually interpret the third party data, you might implement that decryption client side and reassure customers that while you could steal their data, they are theoretically capable of verifying whether you do (by reverse engineering the client code in their possession and ensuring its current version is not transmitting the deciphered data, or your decryption key, anywhere).


How does a malware overcomes this facility

This is likely due to a lag between malware developers and your antivirus database. Kaspersky had to see and identify this as malware before it could be added to their database (then flagged by your scan). So the first time you scanned this zip file, it came back clean. But by the second time you scanned it Kaspersky had found it and updated your local antivirus about it, allowing it to be caught.


Proper way to protect against XSS, when output is directly into JS not HTML?

The correct way is to use the tools of your framework (and its template engine, if available). If you're fiddling with PHP in JS strings, you probably make life harder and more dangerous than necessary.

With plain PHP a common and safe approach is to use json_encode() as explained here. E.g.:

var foo = <?php echo json_encode($foo, JSON_HEX_QUOT|JSON_HEX_TAG|JSON_HEX_AMP|JSON_HEX_APOS); ?>

json_encode() returns the JSON representation of a value, hence it's guaranteed to evaulate to a valid object in your JS code and you can just assign it to a variable as shown. But don't omit the additional flags. Depending on the context, an attacker could otherwise use payloads like </script> to break out of the entire script tag.

The functions htmlentities() and htmlspecialchars() you referred to are used for direct HTML output and should not be employed on JS. They would also allow your string to contain line breaks, resulting in a syntax error that might have security consequences.

Talking about frameworks, Wordpress  gives you the wrapper wp_json_encode(), as recommended in this guideline. There might be equivalent functions for your own framework.

Also be aware that with json_encode() the object is safely prepared for a JS context, but any string you pass can still contain HTML tags that would be unsafe to insert via, say, document.write() or innerHTML. (Avoid these two altogether.)

This is silly, but XSS-safe:

Hello, !
<?php $name = $_GET['name']; ?>


POST request in curl return all the source code of the page

cURL will display exactly what the server provides as response, by default. In this case, the server is responding with a full HTML page. This doesn't mean that the SQL injection did not work, merely that the output even from the injected request was a full HTML page.

It is common for the response from an injection attack to appear in part of a page, perhaps a table which is drawn from a database in some way, or in a field loaded from a database query. Tools like SQLmap perform automatic comparison between "legitimate" requests and injected requests, looking for differences. It is possible to do the same manually, but the key in most injection attacks is the input string, rather than analysing the differences in the output.

Using a proxy such as Burp or ZAP allows you to edit post requests directly, and see the responses in a browser, bypassing any client side protection that may be in place. You can also extract specific parts of the page which are of interest using Burp's Intruder module's extract function - by injecting an uncommon string, it is possible to quickly zoom into the changed part of a page.


What privacy considerations are needed when choosing a payment processor?

Can someone define the nouns (payment processor, networks, institutions) and what has the most privacy guarantees?

I can define the nouns, but the short of it is - there are no privacy guarantees you can count on, except that the companies involved prefer to avoid the legal and reputational impact of data breaches.

(This answer excludes constraints imposed by the GDPR, as you've explicitly limited scope to "an online or in person purchase from a US person to a US based company")

Let's start with the parties involved -

Card Brands - Visa, Mastercard, American Express, etc. etc.  These are the central clearinghouses that facilitate transaction.  When you use a credit card, your transaction details will traverse the chain and get approved or declined by the card brand shown on the card.  The Card Brand owns your transaction details.

Acquiring Banks act as the backend for the merchant; when a credit card transaction is approved, the acquiring banks sends money down the chain to the merchant, and then collects the money to cover that from the Issuing Bank.  The Acquiring Bank owns your transaction details.

Issuing Banks issue credit cards to people ("cardholders").  They send the money for transactions to the Acquiring Bank, and then collect the money to cover that from the Cardholder.  The Issuing Bank owns your transaction details.

Payment Processors act as an aggregation point between Merchants, the Card Brands, and the Acquiring Banks.  Rather than having each merchant try to set up connections to each Card Brand and an Acquiring Bank - which is complex, hard work - the Processor provides the value-add of simplifying the process for both ends of the chain.  The Processor also owns your transaction details.  There may be multiple processors in the chain.

The Merchant is the store that charges people's cards in return for goods or services.  When you say "I'm looking for a payment processor to integrate with my app or website" you're saying "I want to be a Merchant."  The Merchant has to have a relationship with a Payment Processor (the majority), or directly with the Card Brands and an Acquiring Bank (a rarity).

Either the Merchant or the Processor may involve Third Parties, such as Fraud Detection clearinghouses.  A Fraud Detection company may, for example, work with multiple Processors, and provide the value-add that it has a more complete view of card and fraud activity than any single Processor can have.

And the data???

When someone uses a credit card, the details of their transaction traverse the chain and everyone involved gets a copy.  Those transactional details are required for the credit card process to work.  Have you ever disputed a fraudulent charge?  In order to do that, all the parties involved have to have a record of that charge.  The various Card Brands have different policies on how long information must be available to support a disputed charge, but as a rule of thumb, you can expect that 18 months is the minimum your transactional data will be available for use by each link in the chain.

In addition, each one of those entities considers the transactional data valuable in its own right.  The Card Brands, for example, have sophisticated AI that is constantly working to predict fraud more correctly, which includes knowing about past transactions.  Processors are constantly performing data analytics to provide value-add services to the Merchants, explaining trends in the data that Merchants lack the resources to delve on their own.

All their contracts say they can do this.  All their contracts include the ability to share the data with third parties that will do more of it.  When Merchants sign a contract with a Payment Processor, they are agreeing to these usages.  When cardholders open that thick envelope of fine print with a credit card in it, and sign the card, they're indicating they read and agree with all the contracts that allow this usage.  If they don't agree, their recourse is to destroy the card and not use it.

So to take this back to your original question, you

  want to reduce the risk of my purchases being logged, analyzed by
  artificial intelligence, or have copies of that transaction exposed to
  as few third parties as possible

Presumably when you say "my purchases" you mean "purchases customers make against my application/web site", which means you are a Merchant.  As such, all you can do is shop around and examine the contracts offered to you by the various Processors.  Unless you are large, you will likely be seeing standard contracts, and unless you are very large, you will not have room to negotiate those contracts.

Those contracts will basically say, they will protect the data, and can do various things with the data, including analysis and sharing with third parties.  They may list third parties, and reserve the right to change the list.  I would be surprised if you find any significant outliers there; that's not an area the Processors use to differentiate themselves, in part because they're only one link in the chain.

And as for the chain, Processor contracts cannot be used to further constrain the Card Brands and the Acquiring Banks.  The sharing of your data with those entities is an inherent and necessary requirement of the contract to process card transactions.

Bottom line: If you want privacy guarantee that your customer's purchases won't be logged, analyzed, or shared, then don't accept credit cards.


Can I safely use my own statistics to assess password strength on number of times a password is used?

As @aviv pointed out, revealing to a user that some other user also has the same password is a problem.

If you really intend to maintain such statistics, then you have another inherent problem: the "statistics engine" can only help any attacker, since it outputs a list of passwords that are in use. Even a reduced form which merely says "this password is already used by somebody else" allows the attacker to break passwords much faster, for the following reason. Good password hashing uses salts so that attackers cannot try to break 10 millions of passwords in parallel. If hashing is unsalted, then the attacker can hash a potential password once, and compare the resulting hash value against all 10 millions hashes; in that way, the attacker breaks 10 millions of passwords for the cost of breaking one. Salts prevent that. Salts are good.

Now suppose that you have a statistics engine which can tell you, when a user enters his new password, whether that password is already used by somebody else or not (even without telling how many users have chosen it). Then an attacker, who could get a copy of your database (you hash passwords precisely because you fear that scenario), can run the same "engine" on his own computers. Thus, he can "submit" a potential password to the engine, and learn whether it is used by one of the 10 millions of users or not. This allows him to prune his huge list of potential passwords down to the ones that are really worth the effort: instead of trying one billion passwords on each hashed value, he will try only the 1000 or so passwords for which he got hits from the statistics engine. You've just made the attacker's task one million times easier.

The only way out of this problem is to make sure that the attacker will never be able to access the statistics (which, in turns, implies that they won't be shown to the users themselves). This suggests the following:

When a user chooses his password, the password is hashed as usual (with bcrypt or anything similar).
The password is also asymmetrically encrypted with a given RSA public key. Asymmetric encryption is randomized, so the encrypted results don't allow for dictionary attacks. A 2048-bit RSA key is enough to encrypt data elements up to 245 bytes, which ought to be enough for passwords.
The corresponding private key is kept on an offline machine (thus presumably immune to remote attacks). At regular intervals, the "statistics officer" gather the encrypted passwords from the server, transfers them to the offline machine (with a USB drive or something similar), decrypts them, and runs statistics.


You might try a network monitoring solution like OpenNMS, Zenoss, Observium, or NeDi.

They will do more than just discover new systems and open ports.


Private CA and client server authentication

Your Option 2 is sufficient and cryptographically secure. If the CA's signature on the client's certificate validates (and the CA's signing keypair hasn't been revoked due to compromise) then yes, you can assume that the certificate is authentic - ie it was issued by your CA and hasn't been tampered with. Remember that a signature is an encrypted hash of the whole certificate, so if even a single character has been changed, then the signature will fail to validate.

However, that does not tell you anything about who sent it to you. Certificates are public, everyone could have a copy of everyone else's, so Alice could send you Bob's certificate and claim to be Bob. In order to authenticate the client, they need to prove that they have the corresponding private key. Remember that certificates only contain public keys, the matching private keys are never made into certificates, and are stored separately. This can be done by having the client send a piece of data that they signed along with the certificate.

About your Option 1: in a full public-key infrastructure in which the end-users can encrypt for each other and verify each other's signatures, everybody needs access to everybody else's public-key certificates. This is usually done by having the CA keep a copy of everybody's certificates and display them in a public place like on an HTTP site, or in a public LDAP directory.


Verification Code defeated by fake hotmail addresses

Use CAPTCHA and/or telephone number verification (SMS OTP).
It was perfectly okay to ban known disposable email providers. However, there exist(ed) services that ran on stealthier domains (coming out: I ran one of those).
Free email providers offer email addresses for free, upon simply a registration. They may require a CAPTCHA to prevent automation.
CAPTCHA, which is something you can implement yourself, protects from automated requess, and is decently effective. It blocks automated attacks.
What CAPTCHA does not is to protect from human click farms, where the attacker may hire a large number of poorly paid workers to solve the CAPTCHA, register with a free mail address, and post spam.
By using phone verification, and likely limiting the country of those mobile numbers, you are raising the bar of difficulty to the attacker.
The attacker must get a unique SIM card for every access to your website. Getting a SIM card is not that fast and easy like getting a free email address. Getting a SIM card has a fixed cost (e.g. when I change provider I always pay a non-refundable-in-credit cost around 5€ for the hardware, the card itself) and in many countries it requires proof of ID, which raises a yellow flag in a phone company if someone activates a stock of 1000 SIM cards.
Please note that this won't completely stop spammers, but will slow them down. Probably a lot, so that you will simply notice them no more.
Your mileage may vary
Please remember that antispam techniques are meant to slow down large attackers, not completely stop them
I suggest to take a look at this video that shows what a click farm is. An attacker with enough human resources can still get a decent number of SIM cards, have humans solve CAPTCHAs, register for emails and send spam.
There are similar videos. A click farm is an organization originally made for the purpose of boosting social media scores (likes, follows and engagement).
Beware!


Security and Peformance ASUS RT-AC68U router

The RT-AC68U is a pretty good consumer grade router out the box but there are a few things you need to do to tighten it up.

Disable UPnP. This is perhaps the biggest hole it has as standard. If you really need port forwarding set it up manually
Make sure you disable WPS, it is much easier to break into your WiFI with this enabled
Make sure you change not only the WiFI network password but also the router login username and password
Switch on AI Protection. While far form perfect it will give you an additional layer of browsing security as it runs its own dynamic site blacklist
Add MAC address filtering so only whitelisted devices can join your network
Make sure remote admin is switched off (if practical)
It supports WPA2 Enterprise, there is more to set up but if you want better per user encryption it is worth it

Also I am not 100% sure why you switched to a class b network space. I can see not benefit to this for a home network.

Consider running https://www.grc.com/shieldsup against your set up or even better run nmap against your gateway to find out if there are any rouge ports open.

  question: FireWall NAT Loopback, should I turn that off?

See here for details: Is NAT Loopback on my router a security problem?


Public Availability of a good Dataset in PCAP (TCPDUMP) format for IDS/IPS testing

Here is a list of some of the ones I use:

https://www.netresec.com/?page=PcapFiles
https://iscxdownloads.cs.unb.ca/iscxdownloads/CICAndMal2017/PCAPs/
https://github.com/bro/bro/tree/master/testing/btest/Traces
http://traces.cs.umass.edu/index.php/Network/Network
https://github.com/markofu/pcaps
https://ant.isi.edu/datasets/requests.html
https://mcfp.felk.cvut.cz/publicDatasets/

In some of the you need to ask for permission.


Is the IP (from) address of the email header 100% correct?

It depends, most email relays and mail servers will add headers with the sender's IP.  Yet nothing forces them to add the correct IP.  A typical email exchange would happen as:

email client -> your mail server -> destination mail server -> addressee's client

Yet, it can pretty well go:

email client -> local mail server -> remote mail server ->
  -> destination domain relay -> destination mail server -> addressee's client

Or an even more complex design.  SMTP (and its extensions) is pretty robust in terms to proxying/relaying.  And although this is not a requirement of SMTP most relays and mail servers simply keep all headers that are currently in the mail and just add their own on top.

This has a couple of interesting side effects, for example it isn't uncommon to find mail with several Received: headers.

Let's look at a couple of header examples in a rather complex mail path: The following is from a mail that uses google as a the mail server but has its own domain (i.e. the MX record for the mail domain is mail.google.com), the email then passes through a relay and then back to mail.google.com to a different gmail address.

First of all we get:

X-Received: by 10.x.x.x with SMTP id d27mr24214681oib.59.1473677977339;
        Mon, 12 Sep 2016 03:59:37 -0700 (PDT)
Received: from 1058052472880 named unknown by gmailapi.google.com with
        HTTPREST; Mon, 12 Sep 2016 06:59:36 -0400

This is from the client to gmail, note that the IP that google saw is there.  Next it goes to the relay:

Received: from mail-it0-f41.google.com ([209.x.x.x]:37883)
        by mail.fsf.org with esmtps (TLS-1.0:RSA_AES_128_CBC_SHA1:16)
        (Exim 4.69)
        (envelope-from <******@hotelquickly.com>)
        id 1bjix0-0003Rs-3K
        for ******@member.fsf.org; Tue, 13 Sep 2016 04:20:18 -0400
Received: by mail-it0-f41.google.com with SMTP id 183so31757158itf.0
        for <****@member.fsf.org>; Tue, 13 Sep 2016 01:20:17 -0700 (PDT)

So we have one of google's mail server's IP talking to the relay at mail.fsf.org

And finally back to google:

Received: from mail.fsf.org (mail.fsf.org. [208.x.x.x])
        by mx.google.com with ESMTPS id n48sd14225674qtn.126.2016.09.13.01.20.23
        for <******@gmail.com>
        (version=TLS1 cipher=AES128-SHA bits=128/128);
        Tue, 13 Sep 2016 01:20:23 -0700 (PDT)
Received-SPF: softfail (google.com: domain of transitioning ******@hotelquickly.com does not designate 208.x.x.x
        as permitted sender) client-ip=208.x.x.x;
Authentication-Results: mx.google.com;
        dkim=pass header.i=@hotelquickly.com;
        spf=softfail (google.com: domain of transitioning ******@hotelquickly.com does not designate
        208.x.x.x as permitted sender) smtp.mailfrom=******@hotelquickly.com;
        dmarc=pass (p=NONE dis=NONE) header.from=hotelquickly.com
Received: from Debian-exim by mail.fsf.org with spam-scanned (Exim 4.69)
        (envelope-from <******@hotelquickly.com>)
        id 1bjix0-0003Rx-7z
        for ******@member.fsf.org; Tue, 13 Sep 2016 04:20:22 -0400

And there is even more because I use procmail but by then it deals with local IPs only.

In summary

Most machines will add the IPs of the machines they received mail from to mail headers, yet there is nothing forcing the machine to add the correct IPs.  If you run your own mail server you can add misguiding IP addresses, yet the next hop in the mail path will add the IP it believes your mail server has to the headers (i.e. the mail server it received the mail from).


Security risks of file upload-download systems

Usually regular path on web hacking could be in a quick summarized way something like:

Find a vulnerability (scanning, fingerprinting, etc)
exploit the vulnerability
Write some kind of evil payload/shell or whatever
Access/execute that evil stuff
More and more non-important for this question steps after this.

On your system, the third point is free!. I'm not saying you are going to be hacked. But with a system like this you are putting the things a little bit easier if previous steps are completed, that's all.

Remember that only one weak point and everything is possible. Of course the most dangerous is the direct command injection. If there is no present ok, but maybe after exploiting other vulnerability could be possible to launch some kind of command in the system, for example through SQL injection. Depending of the systems, databases and versions, could be possible to execute system commands from the database.

So, beware of ALL! XSS injection, SQL injection, command injection, CSRF, RFI, etc.

And of course, check the version of ALL your components (web server, kernel, operating system, etc). If you use one component with known vulnerabilities maybe you can be hacked even with a good development in the app side.


I agree with the other answers:

It add no or little entropy compared to just using a longer password (as Steffen Ullrich points out).
If you hash the passwords separately, it makes the hashes easier to crack compared to one hash of a long password (as lengyelg points out).

But I would like to add one point related to user behaviour.

Forcing the user to pick two passwords instead of one will probably make the user pick worse passwords out of pure exhaustion. You will just encourage people to repeat the same password twice with some modification to evade any blocks you put in place to prevent this.

When you factor in the human picking the passwords I think you end up being less secure, not more. It is just annoying without any security benefits at all at best, and directly harmful to security at worst.


How to generate keystore for root signed certificate?

Assuming your filenames are accurate, so client_cert.pem and client_prv_key.pem actually contain the client cert and privatekey respectively in PEM format:

openssl pkcs12 -export -in client_cert.pem -inkey client_prv_key.pem -certfile root_cert.pem -out client.p12
# prompts for the input-key passphrase, then the output passphrase (twice)

# to specify passphrase on commandline for client_prv_key -passin pass:xyz
# similarly passphrase on commandline for the output -passout pass:xyz
# however these make the password(s) visible in ps or similar on most OSes,
# and usually visible in your shell (or CMD) history, which is often insecure

Alternatively you can provide them all on standard input in correct order:

cat client_prv_key.pem client_cert.pem root_cert.pem | openssl pkcs12 -export -out client.p12
# same options for passphrase

PS- do NOT enter an empty passphrase for the PKCS12. OpenSSL lets you create it even though it's silly, but Java can't decrypt it which defeats your purpose.


Safely creating and communicating within a fully decentralized p2p network

You asked quite a lot of different questions here. I'll try answering each.

Spoofing/sniffing:

The cure is encryption. More specifically Public Key Infrastructure. Then B can't modify (or even decrypt) data that it's relaying. The only question here is how you want to implement the key exchange (so that it can't be attacked).

Open ports:

An open port is by itself isn't a vulnerability. This really depends on the software that listens on it.

Communication:

Decentralized means that peers communicate with peers (without a central authority). Thus they need to listen to their peers for input.
(Because you can't really talk to someone who isn't listening to you.)

If you are really interested in this kind of thing check out the Tor project.


Cost of finding vulnerabilities vs developing exploits

The relative costs depend very much on what the vulnerability is.  To demonstrate this, I'm going to refer to two semi-recent vulnerabilities: the Bash "Shellshock" bug, and the glibc "Ghost" gethostbyname() bug.

Shellshock

The Shellshock bug was caused by an intricate flaw in Bash's function parsing that could be used to make Bash execute code directly out of environment variables.  The bug existed for roughly 22 years before it was publicly found.

A check for whether the bug exists looks like this:

$ env X="() { :;} ; echo vulnerable" /bin/sh -c ":"

Now, finding that bug takes either careful scrutiny of the source code, or a good session with a code fuzzer.  But as soon as it's reduced to a test case like this, exploits come easily.  For instance, if you know a target server is running a CGI shellscript:

$ nc vulnerable.example.com 80 <<__END
GET /path/to/cgi.sh HTTP/1.1
Host: localhost
User-Agent: () { x; }; printf "%s\n\n" "Content-type: text/plain"; /usr/bin/id
Accept: */*

__END

So Shellshock was hard to find, but once found, it was trivial to exploit.

Ghost

The glibc gethostbyname() vulnerability, on the other hand, is a buffer overflow vulnerability that can only happen under certain very specific conditions.  An attacker can overwrite at most 8 bytes of memory on 64-bit machines, or 4 on 32-bit machines.  The attacker has to set up a specially crafted DNS entry, then trick the target into requesting it.

This bug went unfixed for around 13 years, and when it was fixed it was not recognized as a security vulnerability.  The Qualys security researchers who did report it as a vulnerability spent a large amount of time testing individual programs for exploitability.  A large majority were not, due to the 8-byte limitation and the particulars of how gethostbyname() is typically used.  Note too that simply triggering the bug will generally cause the program to crash: more a denial of service than an exploit.

Their coup de grace, a full remote exploit of the Exim mailserver, is a meticulously engineered Rube Goldberg chain that managed to work around the limitations using specific properties of how Exim happens to organize its memory in specific situations.  It's described fully in the security advisory from Qualys, and is too complicated to go into here.

Comparison

I will admit I'm not sure which bug was harder to find.  Both sat latent in the code for a long time.  Qualys found the Ghost bug "during a code audit"; Stéphane Chazelas found the Shellshock bug by extrapolating from things he already knew about Bash behavior, and an entire series of related bugs were found very quickly once people knew where to look, some with the help of fuzzing tools.

However, the question of which was harder to exploit is much easier to answer.  I think it's clear that a very high level of engineering went into finding a way to turn the Ghost bug into a working Exim exploit.  Much more work was required than the Shellshock exploit above, which took about 10 minutes to write and test.


To get started, you're probably going to want to focus on two things:

Securing your web server
Security your website

That's really two different specialties, and I don't think I'm going to be able to dig up a single document describing both... the ardent security nerd would also point out that this assumes you're working in a secure network with decent physical security, security policies and protections in other arenas.

Securing the Web Server

Hands down, you're best information is going to be web server specific - each one has it's own foibles.  Sight unseen, I'd recommend this:

http://iase.disa.mil/stigs/app_security/web_server/general.html

Yes, it's US DoD specific, but it's got some generally not so bad guidance.

I also really like NIST stuff:

http://csrc.nist.gov/publications/nistpubs/800-44-ver2/SP800-44v2.pdf

Securing your website:

Hands down, my favorite site for secure web development is OWASP.

https://www.owasp.org/index.php/Main_Page

The information in this area is dense and there's a lot to take in no matter how you slice it.  There's no one good checklist for web app security - you have to design security in from the start and be prepared to involve security considerations in all parts of development.

What I love about OWASP is their breakdown of specific issues, for example:

https://www.owasp.org/index.php/SQL_injection

Gives you the problem, the threats, the vulnerabilities, examples, and some idea for what to do about it - pretty much everything I need, whether I'm the security analyst or the poor guy trying to fix the problem.


Mitigate resend confimartion email flooding

Rate-limit on a per-user or per-email-address basis.  Since re-sending confirmation emails is rare, setting the limit absurdly low (say, two re-sends per day) should be sufficient to keep from flooding a user.


A hash function is one-way in the following sense: given the output, the fastest method to find a matching input is to try possible inputs until a match is found. You cannot get more one-way than that. However, this does not mean that hashing names and hoping for a match does not work; in fact, people names are depressingly not varied enough, it is easy to enumerate and hash them all. Since a hash function is public (no secret element) and deterministic (if you hash twice the same data, you get twice the same output), brute force on names will work. A PC with a medium-sized GPU can hash billions of strings per second.

The same problem occurs with passwords, which are limited by the average imagination of human users. When we hash passwords, we must do it with care. The same would apply to names, possibly even more so.


If you see the first line in your examples you see that the line numbers are different. The code is first counting how many news items are creating after a set date, then getting the actual data. So it seems like the site is protected against this particular attack by using inefficient queries, since the two queries are returning different number of columns


Is there a chance USB files once opened by TV, PC etc. have any footprint, backup or residue files left on the hosting device it worked on?

"Opening" a file is not magical. This means: running some local application which will read the file contents, and do something sensible with them.

A USB flash drive is a storage device; its purpose is to store bytes and give them back when asked to. When you "open", say, a video file on a PC, the following happens:

The operating system on the PC tries to infer the file type, based on clues such as the end of the file name (the so-called "extension"): if the file name ends with .mp4, then the OS will think that "this is a video file".
The OS launches its video file reading application (e.g. Windows Media Player, or VideoLAN, or whatever was registered in the OS as "the default video file reading application"), giving it as parameter the file name.
The application will then read the file and process it, e.g. playing the video before your amazed eyes.

Now a video player will not normally save a copy of video data in other files; however, it may save some meta information. For instance, when I "open" a video file on my PC (which runs Linux), the default media player begins playing it. If I close the player, then "open" the same file again, the software asks whether I want to "resume playback" at the point I left previously. This can happen only if the player remembers that I played that file before, and where I stopped; this must be written somewhere, and, in particular, it is not written in the file itself, or (apparently) in the same directory. Basically, this means that some information (including, probably, the file name) is stored in a directory in the local hard disk. This answers your question:

Yes, it may happen that the names of files that you play may be stored somewhere on the playing device. It routinely happens for millions of PC.
No, usually, the file contents won't be saved on the playing device, because it is way too large to make this strategy viable.
However, if the video file was downloaded from "the Internet" at some point (e.g. Youtube), what the device saves might be enough to reliably identify that video later on (assuming that the place from which it was downloaded still has it).


The question doesn't make sense.
Depending on the chosen scan type, nmap sends raw TCP or UDP packets. By default, TCP packets with the SYN flag are sent, and nmap listens whether it returns a SYN; ACK (port is open) or a RST (port is closed).
TCP packets don't have HTTP headers. HTTP is a protocol that sits on top of TCP.


Messenger and Facebook photo access privacy

First, look up what the permissions mean from the developers: https://support.google.com/googleplay/answer/6014972?co=GENIE.Platform%3DAndroid&hl=en - yes, it looks like it would be possible for an app to access the entire photo store on your device.

Second, think about the impacts of your suggestion. If you have 100 GB of photos on your device, then that would mean that the app would have to transfer all those files. This would become very evident when you max out your transfer limits. It is also possible for users to see what data bandwidth an app uses. If Facebook was doing this type of transfer, it would be the top app for data use, and questions would be asked.

So, is it likely that all the photos are downloaded? No. If they tried it, the public would notice and it would make the news.

There is little benefit for Facebook to do this and huge problems if they tried it.


WHOIS

It sounds like your details are in your website's WHOIS records. You can check what is in them here. The data stored include your name, address, e-mail, and often much more.

According to ICANN's specifications:

  Until otherwise specified by a Consensus Policy, such data shall consist of the following elements as contained in Registrar's database:

    3.3.1.1 The name of the Registered Name;

    3.3.1.2 The names of the primary nameserver and secondary nameserver(s) for the Registered Name;

    3.3.1.3 The identity of Registrar (which may be provided through Registrar's website);

    3.3.1.4 The original creation date of the registration;

    3.3.1.5 The expiration date of the registration;

    3.3.1.6 The name and postal address of the Registered Name Holder;

    3.3.1.7 The name, postal address, e-mail address, voice telephone number, and (where available) fax number of the technical contact for the Registered Name; and

    3.3.1.8 The name, postal address, e-mail address, voice telephone number, and (where available) fax number of the administrative contact for the Registered Name.

Privacy

There are ways of hiding this information. Some registrars have whois privacy which will replace your details with those of a company.


The NAT software running on the Firewall system is going to block all inward traffic that doesn't match a current egress session or a port forwarding rule. If no port forwarding rules are in place, then the server behind the NAT is as good as invisible, barring compromises to the firewall.


How come I can see a full HTTPS requests via Fiddler?

Take a look at these instructions for how to make Fiddler decrypt TLS traffic. It tells you to install Fiddlers root certificate into the OS trust store.

This implies that they are doing classic TLS interception - there is one TLS connection between the browser and Fiddler, and another between Fiddler and the server. The first one uses a certificate generated on the fly by Fiddler and signed with their root certificate. The second one uses whatever certificate the server provided. As seen below:

   Browser <--- TLS Connection #1 ---> Fiddler <--- TLS Connection #2 ---> example.com
                      |                                      |
               Certificate signed                  Genuine certificate
                   by Fiddler                         for exmple.com

This works because the OS has been told to trust Fiddlers root certificate, so they can simply spin up a certificate for any domain they want on the fly and the browser will accept it.


External hard drive's hardware vulnerabilities

These are good questions to be asking. Unfortunately, the answer is pretty much “???” to both.

To the first question, individual vulnerabilities vary by drive. As with all technology, vulnerabilities exist out of mistakes in implementation or design, so there are no universal rules for knowing what something might be vulnerable to. To illustrate this, using an appsec example (I specialize in appsec, so it’s always on my mind) - just because something is a login form doesn’t mean it’s vulnerable to cred stuffing attacks. Sure, that’s a good vector for cred stuffing, but the form might be rate-limited, use sophisticated banning techniques, etc. You can’t know simply based on the fact it’s a login form.

Similarly, just because something is an external hard drive doesn’t mean that its encryption is or is not compromised. To get more information over this, you’ll need to research the drive you plan on using more. As a result, without knowing the tech and its vulnerabilities, the answer to your second point is even more unknown.

However, as a rule, the safest bet is to assume that anything left unattended in public is fully compromised. This level of paranoia/caution might be unwarranted (i.e., leaving an encrypted drive for 5 minutes while you turned your back at the library seems probably fine), but if you want maximum security, that’s the best way to go. This is especially true in a higher-risk situation - such as if the drive is old and therefore maybe less secure, if it was in an environment with more resources (the library is probably less likely to have compromise than, say, in custody with law enforcement), etc. If it really really matters, assume it’s toast and act accordingly.

I’m going to add the caveat this is just Internet Thoughts. Have common sense and don’t make decisions based off of this, knowing this answer is written from the perspective of broad generic guidelines.


In my opinion for application testing the best approach is to work on a mirror of the production site (either clone the prod. site if they're using Virtual machines or use a test environment).  The key is that the code deployed should be the same as is in production. In that kind of environment you should be able to automate away without any concerns about the data created.

If you can't do that then the impact depends on the customer and the site.  In many circumstances you can ask for test users, keep all the content under those users and then request that they are deleted at the end of testing.

If that's not possible then make sure you have the conversation about these side effects before you start testing.  What you don't want to happen is that you get into the test, someone in the business notices their site filling up with "garbage" and they shut the test down.

Of course a security idealist would argue that they should have protected against the automation of form submission on their site, but that's not going to stop an irate marketing person wondering why they have several hundred emails about new comments on their blog :o)

Also as @johndeters says it's very important to keep track of the data and in the report explicitly recommend that they clean it up.  I've had cases where someone came back to me 9 months after a test to complain that my test data had cause their upgrade to fail and I was very happy to be able to point them towards the section in my report where I told them to remove the test data.


Service TO Service Authentication & authorization

You can use the TLS/SSL protocol which can provide data confidentiality and data integrity. Combined with HTTP, this gives us HTTPS, which seems to be what you're looking for.


I guess it depends on what the server does, but it doesn't add much protection either way (or is even worse). (Also, does the average user know how to compute a hash on mobile? Sounds quite a hassle to me.)
1. Server stores hashed password
This means client sends H = hash(password) to the server, server looks up clients hashed password H' in the DB and validates client by checking H == H'. This is really bad. Now if a DB breach happens the attacker can enter the hashed password that was stored in the DB to login. It essentially removes the point of hashing in the first place and is as if the plaintext password was stored.
2. Server stores hash of hashed password
This means client sends H = hash(password) to the server, server looks up hash of clients hashed password H' in the DB and validates client by checking H' == hash(H). Now if a DB breach happens the attacker must check if hash(hash(guess)) == recovered_hash, so the cost to the attacker is about 2x what it would be with no client-side hashing (i.e. the attacker can check passwords 50% slower which isn't much of a win). The security in this case still depends on hash being a (non-invertible) function resistant to dictionary attacks, client side hashing does little to help.
Summing up, the best thing to do is have the client enter the password in plaintext, transmit the password to the server over HTTPS, and hash the password on the server with something like bcrypt or scrypt, storing only the hash on the server. The reason your construction isn't an improvement is that the "seed" if you will does not gain any entropy and is just as easy (or not easy) to guess, you are just adding an intermediate stage.


Is it secure to store passwords with 2 way encryption?

No, this is not a good practice. There are two distinct problems.

encrypting the password instead of hashing it is a bad idea and is borderline storing plain text passwords. The whole idea of slow hash functions is to thwart the exfiltration of the user database. Typically, an attacker that already has access to the database can be expected to also have access to the encryption key if the web application has access to it.

Thus, this is borderline plaintext; I almost voted to close this as a duplicate of this question, because this is almost the same and the linked answer applies almost directly, especially the bit about plaintext offenders; there is another answer about plaintext offenders as well.
sending the plain text password via plain text email is a bad idea. They could argue that there is no difference when no password reuse happens, but I doubt they would even know what that is and why it’s considered bad practice. Also, password reuse is so common that that wouldn’t be a good answer.

Additionally, as they seem to be working on the second part (even though password reset links in plain text emails are in the same ballpark, i.e. a threat that can read the password from the plain text mail can also read the link, maybe before you can), you could explain them the problem about not hashing from my answer, also feel free to link this answer directly.

Maybe even explain that encryption is one way, but can always be reversed by the inverse function of the crypto system in question, aptly named decryption. Using terms like "one way encryption" and "two way encryption" rather than "hashing" and "encryption" shows a lack of understanding.

The real problem is: them implementing a password reset does not mean they will hash (correctly) in the future; there is not much you can do about this except using a password manager and create a long, strong passphrase that is unique for this site and hope for the best.

This is especially true since they seem to want to keep the part of their system that tells staff your password (for absolutely no good reason). The implication being they keep not hashing properly - them saying staff can only see the password in that three login timeframe is not true; if the web app can access the key, so can the administrative staff. Maybe no longer the customer support staff but they shouldn’t be able to see it in the first place. That is horrifically bad design.

Depending on your location, schools as being part of the public sector have obligations to have a CISO you can contact directly, expressing your concerns. And as usual in the public sector, there ought to be an organization that is supervising the school; they should have a CISO at least, who might be quite interested in this proceeding.


Which mail clients support PGP/MIME - is there an overview?

Wikipedia has a table full of answers in its Comparison of Email Clients.  No telling when the table was last updated, but probably newer than 2006.  Yes, by all accounts, Outlook Express is still broken (surprised?  I should hope not).


What do these signatures of my PGP key mean?

If I import the public key listed in your post, I get this message:

$ gpg --import pkey2.txt
gpg: key 50268311C7AD3F62: 11 duplicate signatures removed
gpg: key 50268311C7AD3F62: 5 signatures reordered
gpg: key 50268311C7AD3F62: public key "DDoSolitary <Dxxxx@xxxxxx.com>" imported
gpg: Total number processed: 1
gpg:               imported: 1

As you can see, the extra signatures listed on your public key are labeled duplicates upon import and do not affect operation or trust of the key.

The extra signatures are most likely from key manipulations.  Each modification of your key produces a signature.

All the signatures and dates of the signatures can be extracted from the public key using the --list-packets option of gpg.

    $ gpg --list-packets pkey.txt |grep -A1 ":signature"

:signature packet: algo 22, keyid 50268311C7AD3F62
    version 4, created 1504334453, md5len 0, sigclass 0x13
--
:signature packet: algo 22, keyid 50268311C7AD3F62
    version 4, created 1508055794, md5len 0, sigclass 0x13
--
:signature packet: algo 22, keyid 50268311C7AD3F62
    version 4, created 1508055418, md5len 0, sigclass 0x13
--
:signature packet: algo 22, keyid 50268311C7AD3F62
    version 4, created 1508056216, md5len 0, sigclass 0x13
--
:signature packet: algo 22, keyid 50268311C7AD3F62
    version 4, created 1504703268, md5len 0, sigclass 0x18
--
:signature packet: algo 22, keyid 50268311C7AD3F62
    version 4, created 1504334453, md5len 0, sigclass 0x18
--
:signature packet: algo 22, keyid 50268311C7AD3F62
    version 4, created 1504703268, md5len 0, sigclass 0x18
--
:signature packet: algo 22, keyid 50268311C7AD3F62
    version 4, created 1508055418, md5len 0, sigclass 0x13
--
:signature packet: algo 22, keyid 50268311C7AD3F62
    version 4, created 1504334453, md5len 0, sigclass 0x13
--
:signature packet: algo 22, keyid 50268311C7AD3F62
    version 4, created 1508056216, md5len 0, sigclass 0x13
--
:signature packet: algo 22, keyid 50268311C7AD3F62
    version 4, created 1508055794, md5len 0, sigclass 0x13
--
:signature packet: algo 22, keyid 50268311C7AD3F62
    version 4, created 1508553430, md5len 0, sigclass 0x18
--
:signature packet: algo 22, keyid 50268311C7AD3F62
    version 4, created 1504334453, md5len 0, sigclass 0x18
--
:signature packet: algo 22, keyid 50268311C7AD3F62
    version 4, created 1504703268, md5len 0, sigclass 0x18
--
:signature packet: algo 22, keyid 50268311C7AD3F62
    version 4, created 1508055418, md5len 0, sigclass 0x13
--
:signature packet: algo 22, keyid 50268311C7AD3F62
    version 4, created 1504334453, md5len 0, sigclass 0x13
--
:signature packet: algo 22, keyid 50268311C7AD3F62
    version 4, created 1508056216, md5len 0, sigclass 0x13
--
:signature packet: algo 22, keyid 50268311C7AD3F62
    version 4, created 1508055794, md5len 0, sigclass 0x13

The dates are in UNIX EPOCH time.  These are all the signatures dates sorted and in a more readable form:

2017-09-02 06:40:53-00
2017-09-02 06:40:53-00
2017-09-02 06:40:53-00
2017-09-02 06:40:53-00
2017-09-02 06:40:53-00
2017-09-06 13:07:48-00
2017-09-06 13:07:48-00
2017-09-06 13:07:48-00
2017-10-15 08:16:58-00
2017-10-15 08:16:58-00
2017-10-15 08:16:58-00
2017-10-15 08:23:14-00
2017-10-15 08:23:14-00
2017-10-15 08:23:14-00
2017-10-15 08:30:16-00
2017-10-15 08:30:16-00
2017-10-15 08:30:16-00
2017-10-21 02:37:10-00

It's fairly easy to see that the signature times occur during the key manipulations.

$ gpg --edit-key 688E1D093C3638F588890D4450268311C7AD3F62

pub  ed25519/50268311C7AD3F62
     created: 2017-09-02  expires: never       usage: SC
     trust: unknown       validity: unknown
sub  ed25519/E758605C3A2792F6
     created: 2017-09-06  expires: never       usage: A
sub  ed25519/6DC20782F6E9E2F3
     created: 2017-10-21  expires: 2018-10-21  usage: S
sub  cv25519/1224F2883BEE2AA5
     created: 2017-09-02  expires: never       usage: E
[ unknown] (1). DDoSolitary <Dxxxxxx@xxxxxx1.com>
[ unknown] (2)  DDoSolitary <Dxxxxxx@xxxxxx2.com>
[ unknown] (3)  [jpeg image of size 29903]

So, looking at the key, there's nothing that's stands out as incorrect on the key, either as posted on the public key server or on the imported key.

As as side note: the trust level on the public key as listed in my system will not be the same as in your system.  Trust is under local control.  The trust is set as unknown by default.  However, it is unwise to blindly trust public keys listed in a keyserver.


Security risks of disabling SSID Broadcast

Take a look at the WiFi Pineapple, which is a wireless MITM impersonization device available for $100 plus shipping.  The attacker pretty much only has to power it up and configure it, and it will start offering instant MITM attacks.  If a mobile device is probing for an already-known open SSID, it will happily provide it with a working internet connection.  It requires almost no skill to operate.

The key is that a client who trusts even a single connection to any open WiFi access point anywhere has placed themselves in a vulnerable position, and this is true whether or not you are broadcasting your SSID.  The secure approach is to require credentials on your access points, and avoid being part of the problem for your clients.  Whether or not you broadcast your SSID then becomes a matter of convenience for your users, and is not a matter of security.


I think you could do it that way:

Generate your key pair, or use your existing key:

pub   rsa2048/E777DCBD 2017-06-16 [S]
Key fingerprint = 27F6 F30D EA60 B720 85EF  AF89 358C 5737 E777 DCBD
uid         [ultimate] Testing
sub   rsa2048/05662D7C 2017-06-16 []

Next, use gpg --edit keyid on that key and add the signature, etc. subkeys that you want using the addkey keyword and then save it to store this newly created subkey on the keyring. (Or only use the default encryption subkey, and do nothing...) For the sake of the example, I've added encryption, sign and auth subkeys (using the gpg --expert --edit keyid command to set my own capabilities on the subkeys) to my master key:

sec  rsa2048/E777DCBD
     created: 2017-06-16  expires: never       usage: SC
     trust: ultimate      validity: ultimate
ssb  rsa2048/05662D7C
     created: 2017-06-16  expires: never       usage: E
ssb  rsa2048/8D9F7D0A
     created: 2017-06-16  expires: never       usage: E
ssb  rsa2048/4852B177
     created: 2017-06-16  expires: never       usage: A
ssb  rsa2048/105B24A3
     created: 2017-06-16  expires: never       usage: S
[ultimate] (1). Testing

Note: when you have two encryption subkeys like here, the most recent one is used by default.
Create a backup of the whole key (secret and public) using gpg --export -a keyid > commonpubkey.asc and gpg --export-secret-key -a  keyid > masterseckey.asc. There you should take the usual PGP cautions and copy those files onto a secure offline storage.
It is also important now to backup your whole keyrings, because the next step will mess it up.
So, now you only want to take one of the subkeys, so you can: gpg --edit keyid and delete all the subkeys that you don't want to share (to do so, you have to first mark them with key n and delete them with delkey) and save:

sec  rsa2048/E777DCBD
     created: 2017-06-16  expires: never       usage: SC
     trust: ultimate      validity: ultimate
ssb  rsa2048/8D9F7D0A
     created: 2017-06-16  expires: never       usage: E
[ultimate] (1). Testing

Finally the magic gpg --export-secret-subkeys -a keyid > sharedseckey.asc will allow you to export only the remaining subkey, without any other keymaterial from the primary key!

So you can finally restore your keyrings (you'll need to restore both the secret and public one, because when you delete a subkey, you also delete your public subkey), and you're back to owning the whole key, with all subkeys and proper keymaterial.

The next step is for your colleagues:

On their machine, they need to do a gpg --import commonpubkey.asc commonseckey.asc (with those two files you've previously generated).

Et voilà, you're all ready to use gpg.

To verify that your colleagues really don't have any secret keys you don't want them to have, have a look at the output of gpg --list-secret-keys, you'll see that this key without the primary secret key is marked with a '#':

sec#  rsa2048/E777DCBD 2017-06-16 [SC]
uid         [ unknown] Testing
ssb   rsa2048/05662D7C 2017-06-16 [E]
ssb   rsa2048/8D9F7D0A 2017-06-16 [E]
ssb   rsa2048/4852B177 2017-06-16 [A]
ssb   rsa2048/105B24A3 2017-06-16 [S]

They also won't be able to sign or authenticate since they won't have those subkeys, but if you sign something with your key, the common pubkey
will still be able to verify it.

So in the end, they will not import the subkey, but the whole key, but without primary keymaterial, with only the keymaterial of the encryption subkey you'll have left there. But this is the way to go instead of trying to add this common encryption subkey to their own key, since it would then mean that when you encrypt something for them you'll have encrypted it for the whole group, since the newest encryption subkey of a public key is used by default to encrypt by gpg...

So you really don't want them to import the subkey, you want them to import the secret key, with the minimum of keymaterial.


Constant POST request spam to /cgi-bin/ViewLog.asp endpoint

You are right about the fact that he host header can easily be faked, however I assume that you've got the POST http://127.0.0.1/cgi-bin/ViewLog.asp right from yours logs - and that indicates that the requests come from (or via) your localhost indeed.
Some attacker code might try to exploit:

https://www.exploit-db.com/exploits/43884

which is aimed at (rebranded) Zyxel and Billion routers. This attack is unlikely to cause any harm to you as you're obviously not running your node.js stuff on a router, however the sole fact that something uncontrolled seems to originate from (or be forwarded by) your localhost should be investigated.


According to this and Wikipedia, the OSI Session Layer is responsible for setting up any kind of conversation/dialogue.

As ISAKMP runs on top of UDP (port 500) and it sets up a secure and authenticated channel for communication, i would say that it is part of OSI Session Layer.


Recovering locked-out accounts after losing smartphone

Contact Google and Yahoo directly, and explain the situation. They should have a way to prove your identity by proxy (presenting your ID at the wireless store where you purchased the phone, that sort of thing) that they will accept.

You aren't the only people to have been in this situation.

You can find contact information for Google by googling for "how to contact Google". There is a wikihow page that provides their customer service number. Or, you can select Google's contact us page. There is a pull down to select "Account (login)", which takes you here: https://support.google.com/accounts/?hl=en#topic=3382296


Should I encrypt sensitive form data with JavaScript on the client?

Best case scenario here is that you are wasting your time. Generally, doing client side encryption on top of HTTPS in a web application is a lot of work, but it provides no extra security.

The purpose of an extra crypto layer would be to protect against a MITM that has somehow managed to crack the TLS encryption. But an attacker that has done that could easily just modify the JS source of your extra crypto, simply turning it off. So this would only be useful against a passive MITM that has cracked TLS, and that really isn't a threat model worth investing a lot of energy protecting against.

Also note that limitations in many browsers (Opera Mini, IE9) ability to generate secure random numbers makes client side encryption tricky. You should never rely on Math.random for anything crypto related.

You should focus your energy where it has maximum impact. Instead of implementing your own crypto, make sure that you are using the good crypto you get for free  with HTTPS correctly. If you are worried about MITM-attacks, looking into HSTS and preloading would be time much better spent.


It decompresses just fine for me... I converted the Base64 string into binary, then ran gunzip on it (I am using a Linux system here). This results in another piece of PowerShell that does things which can only be considered as definitely fishy. It contains a piece of 450 bytes, that it loads into (native) RAM, and runs as code. I am way too lazy to disassemble the piece of code, but I feel pretty safe when I say that this piece of code is not safe at all.

Looks likes there is some malware in your machine, that tries to gain some privileges. It's time for some nuking, I'd say.


Are mail-servers removed from security concerns in contrast with cloud storage?

In the general case I would agree with you that cloud services are no more inherently risky than e-mail providers.  Any time you store data with a 3rd party there are security risks, especially if you're using a consumer grade service which doesn't have things like contracts in place where you can specify security/audit requirements.

I'd say that there's a slight difference between e-mail and cloud services in that the programs and servers for e-mail storage and transport are quite mature and most of them will have been through the mill of vulnerability disclosure, whereas a lot of cloud services will be newer codebases that may not have had the same level of scrutiny.  Of course that is a bit of a generalisation but may well still hold.


Sounds like you should simply assume that your local network is compromised - rather like using a public library.

You might offer to help manage her router so that you can change default passwords and add monitoring for things opening unexpected ports. You can also add some outbound firewall rules and turn off UPnP if possible.

Monitoring activity through the firewall for a while and periodically thereafter will soon give you a good idea as to whether anything nefarious is happening somewhere on the LAN.

A VPN is a good idea (assuming you can live with the overheads it brings), at least to start, you might also set up a VM with direct connection to the LAN to see if anything unexpected tries to connect to it.

UPDATE: How to monitor for open ports.

This rather depends on the router model. Most will have built-in dashboards that will show you open ports and connections so that you can view them manually. Some will have SNMP available that you can connect to an SNMP monitor tool that you would run on your VM.

Or you can run something like PortScan. Or if you want to really get technical, try nmap. You would run these on your VM. There are a great many similar tools, just search for alternatives if those don't suit.

To check the generally open ports on the WAN side of the router, you can use a tool. e.g. http://www.yougetsignal.com/tools/open-ports/ or http://www.canyouseeme.org/ though these may not spot ports that are only opened to specific remote end-points. Not to worry though because your LAN scan will find all the traffic.

As for what might constitute an unexpected connection. That is rather hard to predict. But an SMTP (email) connection to a PC on the local network might be suspect, non-standard ports being opened to remote end points, that kind of thing.


This should probably be closed as a duplicate, but given the sheer volume of duplicates, I figured I'd list them in an answer instead of a comment (to make them fit, and for readability) for posterity:

Bank cards - chips vs mag stripe security
Why are chips safer than magnetic stripes?
Why does my debit card have a stripe AND a chip?
What are the problems that Chip card technology solve? and what are the problems EMV solve? …
Why EMV cards cannot be cloned?

These, and many more interesting posts, can be found using the search interface in the upper right hand corner.


Script to make schedule tasks on Windows with Admin credentials

Rather than doing it in R, you can do it in Powershell and save the credentials as a secure string.  That solves the problem of it being human-readable, but the credentials used to create it are still available (which is not a good security practice).  More info on that is available here.    Otherwise you can do this in Powershell and use get-credential, which prompts you to enter the appropriate credentials.  (That's a lot more secure way to handle that, although it comes with the "bother" of entering your ID and password)


Common format to send encrypted data + IV?

To answer the question of how to package the ciphertext and IV, concatenation should suffice: IV || ciphertext || tag. This format is convenient because it allows streaming of encryption. You generate and send the IV, stream the encryption, then calculate and send the authentication tag (decryption output can't be streamed however as the tag must be checked first).

The downside of course is that it makes it more difficult to change the algorithm or to support multiple algorithms. You could include an unencrypted algorithm identifier, but you would have to be very careful, as an attacker could change it to whatever they like, opening the door to downgrade attacks.

But you're wanting to do this with a pre-shared key as an alternative to TLS, so there's a lot more to it than that. TLS doesn't just provide confidentiality, it also handles key management and authenticity (among other things). The problem with a pre-shared key is that you have to be very aware of the constraints. RFC 4107 is worth a read.

Nonce/IV Reuse

ChaCha20-Poly1305 and AES-GCM both use a 96-bit nonce, meaning there is a 50% chance of repeat after 248 messages when using random nonces, so the key should be changed well before that (NIST says before 232 messages for GCM). It's also possible to use a counter instead of a random IV for certain algorithms, but there are difficulties with that as well. Due to this risk, the RFCs for ChaCha20-Poly1305 and AES-GCM as used in CMS both require that an automated key management system be used.

The simplest key management system is symmetric key-encryption keys. A content-key is randomly generated for each message and used to encrypt it, and a pre-shared key is used to encrypt the content-key to send along with the message.

Salt

Your question mentions a salt, which implies that you're using a password. This is a bad idea. To derive a key from a password requires a slow KDF like Argon2, bcrypt, or PBKDF2, and a constrained device is likely not fast enough to run such a KDF with a high enough cost to be secure. It would be much better to generate the pre-shared key with at least 128 bits of entropy. Confusingly you also want to transmit the salt, which implies that the password is changing, but this contradicts your comment that you put a "key" on the device.

Authenticity

TLS provides authenticity, which is important because commonly used encryption modes like CBC and CTR are malleable. It would be best to use an AEAD mode like ChaCha20-Poly1305 or AES-GCM, or, failing that, to use an HMAC with Encrypt-Then-MAC.

Constrained Devices

On constrained devices that aren't capable of generating cryptographically secure randomness (and thus can't generate a random nonce/IV or a per-message key), a pre-shared key with a stored nonce counter might be the best you can do. You just have to be very careful to make sure that nonce reuse is prevented even when the device is power cycled, and make it as easy as possible to change the key if/when that becomes necessary.


It looks like there is a typo in the Chrome message. They repeated ".com.com" in the link.

Com.com, according to their site, is set up to take advantage of such typos:

  As an example, if a user browses to “example.com.com” the user will
  see content, information and/or advertisements based off the word
  “example.”

So, no, com.com is not a Google site, and there does not seem to be an infection. Just a typo in the Chrome message.

As a side note: never dismiss error messages quickly. They exist to give people pause before doing things that might not be safe. In this case, it's an indication that things were not as you expected.


The SSL handshake is not encrypted. The server simply sends its public key in the plain. The client knows it can trust it, because it's signed by a totally trustworthy CA. A certificate is essentially a public key with some associated data such as a domain, signed by a CA.

The client doesn't need to be able to decrypt the PreMasterSecret (from which the session key is derived), because it chose its value and still remembers it.

Take a look at how the SSL handshake works:

  Negotiation phase:

  A client sends a ClientHello message ...
  The server responds with a ServerHello message ...
  The server sends its Certificate message (depending on the selected cipher suite, this may be omitted by the server). This contains the public key
  The server sends a ServerHelloDone message, indicating it is done with handshake negotiation.
  The client responds with a ClientKeyExchange message, which may contain a PreMasterSecret, public key, or nothing. This PreMasterSecret is encrypted using the public key of the server certificate.
  The client and server then use the random numbers and ''PreMasterSecret'' to compute a common secret, called the "master secret". All other key data for this connection is derived from this master secret (and the client- and server-generated random values)

  The client now sends a ChangeCipherSpec record, essentially telling the server, "Everything I tell you from now on will be encrypted and authenticated
  Finally, the client sends an authenticated and encrypted Finished message, containing a hash and MAC over the previous handshake messages.
  The server will attempt to decrypt the client's Finished message and verify the hash and MAC. If the decryption or verification fails, the handshake is considered to have failed and the connection should be torn down.
  Finally, the server sends a ChangeCipherSpec, telling the client, "Everything I tell you from now on will be authenticated and encrypted
  The server sends its authenticated and encrypted '''Finished''' message.
  The client performs the same decryption and verification.

(Based on wikipedia Secure Sockets Layer - Simple TLS handshake)

As you can see, the encryption is only enabled after the negotiation and thus transfer of the public key completed.


A bit of theory

Technically it is possible that single certificate may have multiple chains, however it is not your case.

Multiple chains can be produced when cross-certification, when CA server have two or more certificates signed by different issuers and all these certificates are installed in the client's store to build the chain.

This situation is common for Microsoft CA (ADCS), when you renew root CA certificate and reuse the key pair. In this case, root CA have two certificates with (optionally) different settings (for example, cert validity), but with the same subject and public key information. When client builds a chain it ends with two matching root certificates. It is up to client to select the right chain.

alternatively, foreign CA can sign your intermediate (or root) CA certificate and install this certificate to user's store. When client builds a chain, it can go through different intermediate CA certificates (signed by your and foreign roots) and depending on trust settings, client may elect the best chain through some logic.

A more sophisticated scenario of cross-certification is so-called Cross-Certification Bridge, which is helpful when multiple organizations run their own private PKIs and they want to establish a mutual trust between each organization.

To your question
As far as I understand, there are clients that do not trust Comodo, while trusting your private CAs? If this is correct, then you can do cross-certification. Sign Comodo's issuing CA with your internal CA and put this certificate to all clients who do not trust Comodo. In this case, when clients will build the chain, then they will get two chains: up to untrusted Comodo root and to your trusted private root. If the chain selection on clients is good, then they will select proper and trusted chain.

Unfortunately, I don't know how to do this in OpenSSL, however I would recommend to read Microsoft's whitepaper on the subject: Planning and Implementing Cross-Certification and Qualified Subordination Using Windows Server 2003
Here you will get detailed explanation about how cross-certification works.


If you are looking for the vulnerability reports and databases, I've already answered this in this stackoverflow answer.

Copy-pasting again for easy reference:

Common Vulnerabilities and Exposures project (CVE)
http://seclists.org/fulldisclosure
http://packetstormsecurity.org/
http://www.securityfocus.com/bid/ (BugTraq)
http://www.exploit-db.com/

Various list above have feeds or subscription feature which you can subscribe to get the vulnerability information delivered right to your inbox.


how would a 12-digit password be a safe guard in this situation?

I used to exclusively use fail2ban in conjunction with blocklist.de to mitigate and report ssh brute force attacks.  As I have gotten older and more experienced I  have found that although this is effective it is not the best security model and it makes unnecessary server load.  Now that I am older and more experienced I use ssh key pairs.  This is what both Amazon and Microsoft use for their cloud services.  I recommend doing this instead of relying heavily upon passwords.  You can protect yourself with just an ssh key, and ssh key and a password or even an ssh key and a physical token like a yubikey.  I have never set up a yubikey for ssh but I understand this has been implemented recently into ssh.  If you want a good primer on generating and setting up ssh key pairs I recommend using the documentation written by github for setting up ssh key pairs. This is the most comprehensive documentation I have seen and I used it myself when I had to create a new stronger key when at a previous job because of the risks of an older encryption method.


How to stay anonymous in public wireless network?

Partly, yes...but mostly no - there are many other things that can identify you.

The Media Access Control address is used on the local network segment only. Yes, it is (supposed) to be unique to each network interface device, and sometimes can be changed/spoofed.

So to a slight extent, regularly changing your MAC address will provide you with some degree of anonymity against basic analysis on the WiFi network you're connecting to - they won't see the same MAC address twice in their WiFi controller logs. More advanced techniques (e.g. Packet inspection) will still be effective against you - see #1 and #2 below.

However, there is more to anonymity than just your MAC address. You may also want to be anonymous to the machines on the internet you're connecting to, or to any observers/agencies performing bulk surveillance:

HTTP connections are visible to the local network and internet. You might accidentally reveal identifying information by accident. For example, you might login to a poorly designed website that sends the username in cleartext. You might transmit a long-lived advertising cookie that was previously observed.
Your web browser might act in a unique way, or your machine might transmit a unique series of packets that allows your computer or browser to be identified. This is known as browser fingerprinting, or device fingerprinting.
You might physically visit the same WiFi location multiple times in an observable pattern. Even if no one sees you, the security cameras will. It would not matter how well you spoof your MAC address if there are other ways of tying you to the traffic.

To remain anonymous in the broad sense of the term, you need to consider every aspect of your actions. The WiFi network you connect to is only a small portion of what you need to consider.


Quick Security Mitigations to Impending attacks

Bit of a wide open question really as steps should be taken at multiple levels to increase the security posture of your environment. This can include hardening operating systems, secure network design, strict firewall rulesets, hardened databases, software developed with a good SDLC and security in mind throughout, etc etc.

That said if you know of an attack or one has just started the simplest way would be to block traffic from the source IP address and look to fixing the actually vulnerability as soon as possible.


How to read password from Windows credentials?

The Credential Manager stored passwords are managed by the Data Protection API and protected by DPAPI "master keys":

  The DPAPI keys used for encrypting the user's RSA keys are stored
  under %APPDATA%\Microsoft\Protect{SID} directory, where {SID} is the
  Security Identifier of that user. The DPAPI key is stored in the same
  file as the master key that protects the users private keys. It
  usually is 64 bytes of random data.

This Synactiv presentation has a good overview of DPAPI master keys:

  These masterkeys are stored in blobs, each containing:

  a GUID
  a salt
  master key structure (containing master keys)

... and includes this chart, which shows how the protections have changed with each release of Windows:

According to that presentation, shared keys can be extracted with mimikatz. Beyond the Windows platform, the dpapick project also supports offline and non-Windows use of the API, and both that project and John the Ripper include DPAPImk2john.py, a script for extracting the masterkey files for cracking purposes. hashcat also supports cracking  these DPAPI v1 and v2 masterkey files with that script.

So you should be able to use DPAmk2john.py to extract the master keys, and then crack them to access the credentials.


Is replacing bash with another shell a prudent step?

To determine definitively the degree to which this might or might not be "a prudent step", I think you would have to do some original security research on the possible replacements, which include:

Debian's dash
OpenBSD's ksh
Busybox ash
MirBSD/MirOS mksh
...and certainly others

Mark's answer suggests that at least OpenBSD's has received security scrutiny already, but I'm not sure of the extent or whether there's evidence to back this up (clearly, they did not apply any scrutiny to a cornerstone of comms security (OpenSSL) until recently when they forked it into LibreSSL). On the other hand, it's pretty clear to me that nobody had bothered to read the Bash source for security until recently, or "shellshock" would have been discovered a long time ago; the whole "function importing" thing is a huge red flag that any security researcher would scrutinize as soon as they saw it (and hopefully recommend the entire feature for removal). But for the others, it's not as clear.

What is clear, though, is that all of the above have a much smaller attack surface than Bash. In order for an attacker to take control over a program, there has to be some input channel. These can of course be non-obvious things like resource limits, system clock, etc. but they're still inputs; a program with absolutely no inputs is trivially non-vulnerable. The security design bug in Bash is that it's taking potentially untrusted inputs (the contents of arbitrary environment variables) and subjecting them to complicated processing (parsing as code). On the other hand, as far as I'm aware, none of the above-listed shells do any processing of the contents of environment variables (except individual ones with specified established meaning like LANG and LC_*, ENV, IFS, PATH, PS1, etc.) or other input; they just treat the contents as abstract data which is passed through.

So from a security design standpoint, even without auditing these alternatives, I would estimate them to be safer choices than Bash. Whether that will remain the case it not clear. Certainly Bash is getting a lot of new attention right now, which other shells are less likely to receive, so we could end up with most of the issues in Bash getting fixed while issues in other shells remain unknown. Then you have various factors to consider, like whether you're likely to targeted individually, in which case using less-mainstream software may be a liability.

Personally, I use Busybox ash most places. If nothing else, both ash and dash use about 1/5 the memory of bash and start 2-8 times faster, so they're very practical choices from a non-security standpoint as well.


This is a fool's errand.  Even if you you could obfuscate them sufficiently in the source code, it would still be child's play to hook the device up to a proxy like Fiddler or Burp Suite, and see the raw URLs plain as day when the app is running. No need to look at your source code at all, and there is absolutely nothing you can do about protecting HTTP requests, and even HTTPS requests in this case, for several reasons.

So, was suggested in the comment, you need to protect the endpoints exposed by your server via authentication and authorization. Disguising the address is simply not a viable option.


what is an internal iso 27001 technical auditor?

I think you're responsibilities is to verify on on-going basis if your company is adhering to the ISO27001. So basically your third point describes what you will be doing: reviews.

Basically you are an auditor, which indeed means you are NOT part of the IT department, but the audit department. This is to ensure independence and objectiveness as no-one of the upper management in the IT department should be able to pressure you based on performance reviews for instance.


Is using Touch ID with FileVault full disk encryption secure?

Touch ID and Filevault are not related, at least not yet in current macOS versions.

When a Mac reboots, it first loads a small unencrypted partition which presents the password dialog to unlock the "disk" (really only a volume). A key derivation function is used to derive a long passphrase from your password, which in turn unlocks the encrypted volume with the operating system and all of your applications and files.

Once the operating system boots, you may use Touch ID. It uses a similar element to iPhones called Secure Enclave - a small processor that signs cryptographic challenges. When an authentication event takes places on the system, for example screen unlock, your fingerprint matches the stored representation in the enclave and you're allowed in. As an analogy, asymmetric cryptography is being used instead of a symmetric key (a password).

Whether or not using Touch ID is secure would have to be based on a threat model, but certainly it and traditional password-authentication have their unique advantages.


Does validating a mime-type reduce or stop the need for virus scanning?

What do you mean by validating it against it's MIME-type?  Mime typing is simply a way for an e-mail to make a claim as to the purpose and type of an attachment.  It doesn't include any built in validation.  MIME-type has nothing to do with viruses or virus scanning and a virus could still be sent through a Word document macro for example. So no, nothing to do with MIME type precludes the need to virus scan e-mail attachments.


The point of a unique salt is this: the work that the attacker does to crack one user's salted password cannot be reused against another user.  So it addresses this point of your question:

  Even if you had to recalculate the hash for every user, that is still only one calculation for each user. And if applying a salt is O(1), shouldn't undoing every user cost O(n)?

Think of it this way.  Suppose you have n users, and the attacker is going to perform m password guesses.  If you don't have salts, then the attacker can check each of their m password guesses against all n users—which means that they get to perform m × n user/password pair guesses.

With unique salts, on the other hand, each of the attacker's m password guesses is only applicable to one user.  So they can only perform m user/password pair guesses.  This is a big win for the defender.

And this analysis assumes that the attacker knows all of the salts.  So the answer to your question, strictly speaking, is that salts can be public because they offer a big security improvement even if the attacker knows them.  (In contrast with keys—e.g., ciphers offer no security against an attacker who knows the key.)

We can tweak your question a bit and ask: why not keep salts secret?  Well, that's actually a thing; the popular name for "secret salts" is  pepper, and there are people who use them—they offer further security beyond what public salts do.  But many folks just don't think that the extra security is worth the time and hassle of keeping additional secrets.  Others are actually skeptical that it adds more security in practice:

http://blog.ircmaxell.com/2012/04/properly-salting-passwords-case-against.html
https://stackoverflow.com/questions/16891729/best-practices-salting-peppering-passwords

Others however are more positive about peppers.  Notably, the Password Hashing Competition's call for submissions specifically allowed for algorithms to accept a "secret key" (pepper) in addition to a salt.

But one thing is for sure—using peppers is much more complex than using public salts, because of the extra measures needed to make sure that a compromise of the password database doesn't also compromise the peppers.


Are there any known MD5 hash collisions of messages of different length?

Does the theory of finding an MD5 hash collision (exploiting the weakness in the algorithm) prescribe that the lengths of the messages be the same? In other words, would the theory be unusable to find a collision of two messages of different lengths?

MD5 includes the length of the message in the final block. This means that from an attackers point of view, for reasonable sized messages*, a difference in message length* is effectively equivalent to a distinct chosen suffix.
There have been attacks on MD5 that have allowed for distinct chosen prefixes, but afaik none that allow for distinct chosen suffixes.
* Technically you could find a collision between two messages whose size differed by 2^64 bits, but actually storing and transmitting the larger of the two messages may pose a challenge.


Difference between key encipherment and data encipherment?

Key encipherment means that the key in the certificate is used to encrypt another cryptographic key (which is not part of the application data). This is used within TLS in the RSA key exchange, where the pre-master secret (from which the symmetric encryption key is derived) is generated by the client, then encrypted with the servers public key and send to the server and decrypted there with the servers private key.

Data encipherment means that the key in the certificate is used to encrypt application data. This is not used in TLS. But certificates are not only used for TLS (for example also in S/MIME, VPN, signing of documents ...) so there might be use cases where this is needed.


Use john to crack passwords with X alphanums and Y symbols

The logic of masks does not allow for this. If you want only a certain number of a certain character set in more than one position, then you have to permute the options yourself, either by what you have done or by creating a wordlist.


A certificate serves to identify a connection endpoint, like a device or user.  Any of the client computers can verify a server's certificate was signed by the company's root certificate, which can assure them that they're connecting to a genuine trustworthy endpoint.  So each server needs only one certificate.

Similarly, the (optional) client certificate is issued to a machine or user.  The server can validate that the client is who he says he is; plus, the server can read a value from the Distinguished Name field on the certificate to ascertain the client's identity.

So when using certificates for authentication, you don't need to establish one pair of certs between each communicating pair of endpoints.  You just need one cert for each server and (optionally) one for each client.