Why/when would a CA make a generated public key available to the public at large? I'm confused about this section in my CEH security exam training material:

  A CA is able to perform a number of roles in addition to the validation process outlined here.  Some actions that a CA is called on to perform include the following:

  Generation of the Key Pair - When a CA goes through the process of
  creating a certificate, a key pair that is made up of a public and
  private key is generated.  The public key is made available to the
  public at large whereas the private key is given to the party
  requesting the digital certificate.

Can someone please explain this to me and perhaps include a practical example?  I understand that a certificate authority would need to publish/distribute their public keys for root certs, but under what circumstances would I want a CA to distribute the public key for MY individual signed cert?

Strictly viewed, it's not correct.  It's not the CA that generates the public and private keys, but the requester.  There's no reason the CA should have access to the private key.  The private key should be kept private (though it's needed to establish every new secure connection, so it must be also on-line).

There's no point in keeping the public key hidden.  By publishing the public keys, the CAs can increase the confidence level in the certificate itself, by giving another path to verify the identity.  The purpose of public key cryptography is that there's no practical way to compute/impersonate the private key with only the public key accessible.

And, despite that, public keys/certificates are not published in all cases - for example, in the case of client certificates, assigned for user email addresses/persons, it might even be dangerous to publish all issued certificates.

On the other hand, publishing server certificates has the advantage that every other entity running servers can verify and confirm that noone else has issued a certificate to someone else for their name - this is actually a subject of a recent initiative called Certificate Transparency.

See - https://www.certificate-transparency.org/known-logs

Update: I agree that the quoted description is quite vague, and somewhat confusing.

Perhaps the poster misunderstood the same way, that the CA publishes any public key that it issues a certificate for.  But it's not what it's about.  What it appears to say, that when the CA initially creates its own certificate, with the public and private keys, it is then sharing that specific initial public key (well, not really, but the whole CA certificate) with the world "at large".  This is the whole the point of a CA certificate - so that everyone finding a certificate issued by this specific CA will be able to verify it with a public key already available, in a way that no third parties can maliciously (at least not easily) interfere with.


inetsim installation perlipq/libipq error I am trying to install inetsim on Ubuntu 14.04 where I have been following instructions from several blogs and the inetsim.org documentation itself at:
inetsim download and installation

Now one of the blogs mentions the way to install several of the dependencies of inetsim.

Everything from the above blog works fine, except for the part where you are supposed to install a dependency called Perlipq.

When i try to make perlipq as suggested in the blog above, i get an error that says

  IPQueue.xs:11:20: fatal error: libipq.h: No such file or directory

some further research suggested that Makefile was trying to locate libipq.h in /usr/local/include as suggested by line number 155 in my Makefile as :

  INC = -I/usr/local/include

I checked the location and figured out that the libipq.h header file is not present at that location. Nor does locate/find for libipq.h returns any results.

Through some further digging in I came across the change logs of IPTABLES and figured out that libipq.h references have been removed from IPTABLES beyond a certain version. In my case :

  iptables --version is iptables v1.4.21

which falls in the above category.

Now one way to fix this could be to remove the iptables I have and install an older version that still supports libipq.h, but am afraid doing so might
break the other stuff that might be dependent on the latest versions of iptables.

Is there some other way to get inetsim working on my Ubuntu 14.04? Or do I have to take the approach of installing an older iptables itself ?

Kali Linux has inbuilt inetsim you can try that..!
or else check it out https://techanarchy.net/2013/08/installing-and-configuring-inetsim/ it has step by step process


Attacking VPN server's outgoing connections Due to increasing concerns about privacy online I recently started researching technologies that are supposed to hide our activities from unwanted watchers.
I've been learning about workings of VPN servers and there's one concept I cannot quite get my head around.
As I understand the way VPN works (correct me if I'm wrong tho) is that the VPN client encrypts any data sent from the user to the world, thus preventing anyone from learning about that user's activity. The packet is sent through a public network to the VPN server, where it is decrypted and then sent to its destination.
What bothers me is the route from the VPN server to the destination. Since it is decrypted and travels through a public network it's not protected. Moreover, it seems perfectly reasonable to assume that any data leaving VPN servers are by default sensitive so it occurs to me that for an attacker seeking such data, listening at the getaway from the VPN server is like hitting a jackpot.
Is there something I'm missing here?

You are not missing anything. Protections and controls have a scope. A VPN's scope is from the client to the VPN server. Beyond that is outside of the VPN's scope.
But protections also happen in layers. Most, if not all, web traffic is sent using TLS. So, it is not as simple as "listening" to the traffic that the VPN sends on, since that traffic will also be encrypted.
If the traffic is not encrypted, however, then it is as simple as listening to the VPN's output.


How to analyze malware to find out where keylogger sent data? Recently my antivirus detected some keyloggers that infected my network. I have those malware files and I want to analyze them.

How can I do this analysis? It is important to find out details like where those logs are sent. At least I need a clue to investigate.

If you are not an expert in this skill, there are still some automated malware analysis tools that you can use to get some information from the files.

Sandboxes can be a very useful tool, and Cuckoo is probably the one that is best known. You can either set it up yourself, or find online Cuckoo sandboxes. Cuckoo will tell you what DNS queries are made and what HTTP connections are attempted as well as many other details.

You can replicate a part of what a sandbox does by running the malware in a closed VM and monitoring the network connections and any changes it makes to the filesystem. This only gives you a tiny amount of info (and is very risky to run unknown malware in your own VM if you do not know what you are doing), but it is also an option.

If you do have some skills, then you might want to run a debugger and see exactly how it runs, but this is an advanced skill.

There are, of course, numerous commercial tools available.


MX record Man In The Middle Attack? In theory it should be possible to change a MX record of site A (aspmx.exampleA.com) to site B (aspmx.exampleB.com) and send the mail back to site A (aspmx.exampleA.com)
So you can intercept all the mails, without having to worry about SSl/HTTPS and cracking the users passwords.

EDIT:
What about servers with TLS enabled?
See http://tools.ietf.org/html/rfc3207#section-6

A man-in-the-middle attack can be launched by deleting the "250
   STARTTLS" response from the server.  (...) In
   order to defend against such attacks both clients and servers MUST be
   able to be configured to require successful TLS negotiation of an
   appropriate cipher suite for selected hosts before messages can be
   successfully transferred.

DNS poisoning and MiTM are not necessarily the same thing. You can poison DNS without then fowarding the traffic on to the real source. E-mail is a stateless transaction, you would just need to receive the transmission, in most cases there is no immediate reply. Now, of course if you forward the message to the real recipient they will respond and you can then capture the next reply back to the poisoned target.

In a basic setup this should work. Obviously this won't defeat messages encrypted with PGP or equivalent since those are at a higher layer. I need to check back after some research, but I think that there are some peer-to-peer trust protocols or similar which would detect that a rogue was in place, but those are not basic/standard.

Edit:
Actually this could work both on the SMTP and POP/IMAP side. Obviously doesn't work if IP addresses are used without name resolution.


Why can't I send raw HTTP requests with arbitrary headers from a web application? I am trying to write a script to test a Suricata network signature.  To do this I need to simulate the same user agent as the malware.  I tried the following.

$.ajaxSetup({
    beforeSend: function(request) {
        request.setRequestHeader("User-Agent","XAgent/1.");
    }
});
$.ajax({
    url: "localhost",
    type: "GET"
});

But I got the error Refused to set unsafe header "User-Agent".

I have also tried WebSocket and socket.io.js but each of those do not allow me to actually send a raw HTTP request.

What I want is a tool like telnet in the browser so I can just send the raw HTTP request and define all the fields, but this seems not possible. Is there any way I can do this?

But I got the error Refused to set unsafe header "User-Agent"

Browser behavior varies here. Firefox allows unprivileged web applications to issue a custom User-Agent header to the same domain (or to other origins via CORS), but Google Chrome doesn't.

User-Agent is not in Mozilla's list of forbidden headers:

  Note: The User-Agent header is no longer forbidden, as per spec — see forbidden header name list (this was implemented in Firefox 43,) so can now be set in a Fetch Headers object, via XHR setRequestHeader(), etc.

But you will never be able to send the header cross-domain without permission. That's also why XSS flaws based on altered user-agent strings are usually not exploitable.

Beyond that, an unprivileged web application can't issue actual "raw HTTP requests" for obvious security reasons. Even if you write a browser plugin, chances are that you have to go through the respective higher-level APIs and can't entirely control HTTP requests on a byte level, as you could using netcat.


Is there a reason why IMEI is stored in EEPROM? I've heard that IMEI numbers are stored on an EEPROM, essentially letting them be erased and rewritten.
Thieves misuse it to make stolen phones untraceable.
Is there a reason why manufacturers don't use one-time programmable non-volatile memory for this?

In some countries (I think China is one), regulations require the phone's IMEI to change each time it's repaired.


Is there any point in passphrase-protecting an SSH private key that is used by a service account? On my Linux machine (Alice), I am setting up a service account with an rsync cron job that will synchronise some files with a remote host (Bob). Obviously, I would want to make the rsync secure by using SSH with a keypair.

So my question is: is there any point in passphrase-protecting the private key?

If I passphrase-protect the private key, obviously I wouldn't put the passphrase in the cron job script. I would need to set up a persistent background ssh-agent process so that the cron job can use it. But then doesn't it somewhat defeat the purpose of the passphrase-protection? If someone can break into Alice, can't they just use the persistent ssh-agent to do all the bad things they need to?

The only (slight) advantage, I believe, is that even if the intruder can copy the private key, they would not be able to use it elsewhere because they don't know the password.

In these sorts of scenarios, I leave the private key unencrypted (except maybe sitting on a disk that is encrypted), with read permissions restricted to root and the cronjob run as root, but do my best to restrict the usefulness of this key to the very specific task (versus allowing it to login to the remote machine with full access).

This can be done fairly straightforward by configuring ~/.ssh/authorized_keys to automatically run a specific command when logged in using the specific key.

command="/usr/bin/remote_commands_to_run.sh",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty
ssh-rsa AAAAB3Nz...DfE= root@some_client

See this answer for more.  You should be careful that the commands in /usr/bin/remote_commands_to_run.sh do not allow an attacker to escape to a shell.  It may also make sense to create a limited account on the remote machine with say a shell of /bin/false that runs the remote command (e.g., put the public key associated with the private key in that limited user's authorized_keys).

As always after setting it up, try and test to make sure the key doesn't have more privileges than you expect (e.g., that sftp doesn't work, that rsync doesn't work, etc).

On further notice, I see your particular task is to rsync data (presumably in specific directories).  This is probably doable with limited accounts (e.g., chroot'd to only be able to see those specific directories with no access to a shell).  I believe tools like rssh can be used to do this; consult superuser or serverfault for best advice.

Granted what I do in this scenario is mount the specific directories as a network drive (e.g., through NFS / sshfs) and then do the rsync to that mounted directory in a cron script.


Understanding HTTP GET attack requests I have captured some web attacks. I am trying to understand what purpose does each attack request achieve.

GET /site/public/timing?<!+XSS="><img+src=xx:x+onerror=alert(14721850.00337)//"> HTTP/1.1" 200 6718

GET /site/public/timing?<sVg/OnLOaD=prompt(14721850.00307)> HTTP/1.1" 200 6718

GET /site/race/registrationAuth/1761?x"+onmousemove=" HTTP/1.1" 302 -

GET /site/registration/create/3269?execution=e1s1'+(select*from(select(sleep(3)))tw)+' HTTP/1.1" 302 -

GET /site/registration/create/3269?execution=e1s1%20waitfor delay'0:0:3'

This looks like automated attacks to test for different injection vulnerabilities. As far as I can tell, there is no actual mallicious payload here. The attacks are instead design to detect if the site is vulnerable. I asume that sites will be attacked with real payloads if the tests are positive.

The first three are XSS attacks. The first two will try to pop up a prompt with the number 14721850.00337 in it. If you multiply that number by 100 you get a Unix timestamp that was three days ago. I guess it is used for timing the attack somehow.

Some common tricks to fool filters are employed - mIxeD CAsE, prompt instead of alert, / instead of space.

The forth and the fifth tests for SQL injection. It tries to make the database server "sleep" for a while. That makes it easy to check if the attack was successful. If it takes a couple of seconds to get a reply, it worked. I think the forth one is aimed at MySQL and the fifth at MSSQL, but they might work on other systems as well.

So do you need to be worried? Not really. This kind of automated attacks are common against any internet facing servers. It does not need to mean that someone is actively attacking you specifically, or that you are vulnerable. But still, it is probably a good idea to test if the attacks work or not, because if they did you can be sure that they have been exploited by now.


Using iptables to set up a killswitch for openvpn: DNS requests are blocked but they shouldn't I bought a subscription to a VPN service and I am using the openvpn 2.5.1 client to connect to it. I am using Ubuntu 20.10.
I now want to emulate the "kill switch" feature of most proprietary VPN client.
That is, I want to block any connection that is not tunneled through the VPN. Said otherwise, if the VPN connection drops for some reason (eg. server unreachable), I want all internet connection to be blocked.
To achieve this result, I am following this tutorial.
I have come up with the following iptables rules:
*filter

# Drop all packets
-P INPUT DROP
-P FORWARD DROP
-P OUTPUT DROP

# Allow incoming packets only for related and established connections
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

# Allow loopback and tunnel interface
-A OUTPUT -o lo -j ACCEPT
-A OUTPUT -o tun0 -p icmp -j ACCEPT

# Allow local LAN
-A OUTPUT -d 192.168.1.0/24 -j ACCEPT

# Allow VPN's DNS servers
# Gli indirizzi del server DNS di NordVPN sono 103.86.96.100 e 103.86.99.100
-A OUTPUT -d <DNS_SERVER_1> -j ACCEPT
-A OUTPUT -d <DNS_SERVER_2> -j ACCEPT

# Allow the VPN itself (both protocol/port and interface)
# We use TCP/443
#-A OUTPUT -p udp -m udp --dport 1194 -j ACCEPT
#-A OUTPUT -p tcp -m tcp --dport 443 -j ACCEPT
-A OUTPUT -p tcp --dport 443 -j ACCEPT
-A OUTPUT -o tun0 -j ACCEPT

COMMIT

and I am importing it with sudo iptables-restore < ./vpn_iptables_killswitch_rules.ipv4.
After the import I am able to connect to the VPN successfully. That is, the openvpn client establishes the connection successfully.
However, I am unable to resolve domain name in IP addresses. In fact, ping google.com returns a temporary failure in name resolution, while traceroute 8.8.8.8 works without problems.
This should not happen since I have whitelisted the DNS servers on my rules.
A nmcli connection show <SSID> shows that the connection is using the DNS servers provided by my VPN provided and is ignoring the DNS servers provided by DHCP.
What I am doing wrong here?

Run the command tcpdump port domain and then try to reach a hostname while connected to your VPN to see if the packages are actually sent to the expected DNS servers, and if you can see any response coming back in order to figure out what is going on.
If the DNS requests are sent to the wrong DNS servers, you can change your setup. Or even better, since you are already using iptables for this, you can add some iptables rules to the nat table that redirects all outgoing DNS packets addressed to any DNS server outside of your LAN to your VPN DNS servers:
*nat
# Redirect all outgoing DNS requests to a specific server (both TCP and UDP)
-A PREROUTING -s 192.168.1.0/24 ! -d 192.168.1.0/24 -p tcp -m tcp --dport 53 -m state --state NEW -j DNAT --to-destination <DNS_SERVER_1>
-A PREROUTING -s 192.168.1.0/24 ! -d 192.168.1.0/24 -p udp -m tcp --dport 53 -m state --state NEW -j DNAT --to-destination <DNS_SERVER_1>

(Be aware that these are rules for the nat table, so be sure to separate your filter rules from your nat rules by having your filter rules below the *filter line and your nat rules below the *nat line.)
You can also use iptables to log packages to see what are actually matched by your rules in order to troubleshoot.
Sending and receiving of UDP packets seems to be working since you are able to do traceroute which defaults to UDP mode on linux, so it seems kind of strange that DNS doesn't work.
Your last rule "-A OUTPUT -p tcp --dport 443 -j ACCEPT" seems to be accepting ALL outgoing https connections on ANY interfaces. This will also allow https requests to be sent outside of the VPN tunnel. You should probably put in the address of your VPN server in this rule.


Is it possible to spoof a paired Bluetooth device? I'm thinking about creating a physical lock, where the keys would simply be paired Bluetooth devices, but I'm concerned about the potential (in)security. I would like to have a Bluetooth 2.1 module constantly scan for paired devices and when one is found and successfully connected to (by the module), open the lock. Neither the 'lock' module or 'key' device will broadcast their Bluetooth address (since the module should already know the key's address from pairing), hopefully keeping the existence of them unknown to any 'bystanders'.

So my question is: using a long pairing PIN (e.g 20 digits) and assuming the pairing process isn't intercepted, would it still be possible to 'spoof' a paired device and therefore allow the module to connect to it?

It has been in the past with various devices. It entirely depends on the BT version and implementation on the device. Modern versions of BT (4.x) have good feature sets to prevent these things from happening but most implementations will downgrade to an insecure setting to facilitate connection with an older device.

There are also implementation errors that lead to a new device leaking the pairing key that is supposed to secure the connection from spoofing.

READ THIS:
https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-121r2.pdf

THEN READ THIS:
https://null-byte.wonderhowto.com/how-to/hack-bluetooth-part-1-terms-technologies-security-0163977/


How do Message Authentication Codes provide source authentication I have heard that Message Authentication Codes give source authentication.
But I am a little confused. Message authentication makes a hash code by hashing data or message and when the packet arrives at the destination, it checks the MAC code with the hash code it got from hashing packet message.
At this point, many websites say that by doing this, it can give the source authentication because of a secret key.
But hashing doesn't need a secret key.
So how does it work?

The client would take a piece of data, typically the body of your message itself and use a hashing algorithm that takes two inputs - your input and a long random secret pre-shared key, normally the key would be concatenated to the message before hashing to make it harder to determine the key from a block of cipher text.

This creates a HMAC - hashed message authentication code, it can be created with many hashing algorithms such as md5 or the SHA family. The HMAC offers a guarantee of data integrity as the hash would would not match if any part of the message has been modified.

The client performs an operation in the form MAC = Hash(key, data) and sends this with the message.

On receipt, the server which knows the key is able to perform the same operation and verify it came from the claimed source. To be more accurate, it only proves it came from a source that knows the secret key.

The HMAC security is dependant on the algorithm chosen and key length, common attacks are brute force attempts on the key.

Further info

RFC 2104 HMAC

Wiki - Message authentication code

Security Stackexchange question


Are GUIDs safe for one-time tokens? I see a lot of sites use GUIDs for password resets, unsubscribe requests and other forms of unique identification.

Presumably they are appealing because they are easy to generate, unique, non-sequential and seem random.

But are they safe enough for these purposes?

It seems to me that given a GUID, predicting subsequent GUIDs may be possible since (as far as I know) they're not intended to be cryptographically secure...or are they?

Note:

I'm not talking about sites that use a random blob of gobbledygook encoded in base64.
I'm talking about sites like this that appear to be using a raw guid:
http://example.com/forgotPassword/?id=b4684ce3-ca5b-477f-8f4d-e05884a83d3c

The UUID specification details several "versions" which are methods for generating the UUID. Most are aimed at ensuring uniqueness (that's the main point of UUID) by using, e.g., the current date. This is efficient but means that while the generated UUID are unique, they are also predictable, which makes them inadequate for some security usages.

The "version 4" UUID generation method (in section 4.4), however, is supposed to use a cryptographically strong random number generator. 6 of the 128 bits are fixed to a conventional value (to indicate that this is a version 4 UUID, namely), so this leaves 122 bits from the RNG.

If the underlying RNG is secure (e.g. /dev/urandom on a Linux/MacOS/*BSD system, or CryptGenRandom() on Windows) then given many generated UUID, an attacker is not supposed to be able to predict the next one with success probability higher than 2-122, which is adequately small for most purposes, including launch codes for nuclear missiles.

122 random bits ensure uniqueness with high probability. If you generate many version 4 UUID and accumulate them, you may expect to encounter your first collision after about 261 UUID -- that's about 2 billions of billions; simply storing that number of UUID would use more than 30 millions of terabytes. If you consider "only" 1012 such UUID (one thousand of billions, storable over 16 terabytes), then risks of having two identical UUID among these are about 9.4*10-14, i.e. about 700 thousands times less probable than winning millions of dollars at the lottery.

Therefore, UUID are appropriate for security purposes if (and only if) they are "version 4" UUID generated with a secure RNG.


SSL Server Certificate optional In RFC 6101 (Of SSLv3), In page 22 ServerHello Certificate is marked with * which means optional or situation-dependent messages that are not always sent. It means that Certificate is optional in server hello message. What are the cases when the server certificate is not sent in SSLv3 Handshake?

If you are using anonymous ciphers (ADH etc) you don't exchange certificates.
But in this case you need another way of identifying your peer, because identification is no longer done with the certificate. And without identification of the peer you are open to man-in-the-middle attacks.


NMAP not reporting open ports with -sS scan Is there any reason why an -sS scan (or most others) would not report an open port that doesn't have a service attached to it?

For example, in a regular -sS scan, the following port will be missing, but when I scan that port specifically, it appears as open.

PORT     STATE SERVICE

2509/tcp open  unknown

Most likely, it's because port 2509 isn't in the 1000 most frequently used TCP ports.  Unless you specify specific ports or a port ratio or top port number, nmap scans only the 1000 most commonly seen ports.  See the nmap user guide for details on port selection.


Security of SHA256 and Bitcoins The Bitcoin network use SHA256 as a core component to it's design.  I'm no expert on cryptography, but it seems to me it usually is only a matter of time before security vulnerabilities are discovered (MD5 for example as well as many others).

What are the chances similar vulnerabilities are discovered in SHA256 at some stage, and how damaging would it be to the Bitcoin network?

The only known way to ascertain the security of a cryptographic algorithm is to leave it under close scrutiny of hundreds of cryptographers for several years, and see what comes out. So the right perspective here is historical.

MD5 was published in 1992; it was actually designed the year before (1991). In 1993, first weaknesses were spotted, then bigger weaknesses in 1996 (collisions on the compression function, found by Dobbertin). It took 8 years for these weaknesses to be turned into actual collisions by Wang, in 2004. Seven years later, in 2011, we can create MD5 collisions at will (and much more efficiently than with Wang's original method), but preimage and second-preimage resistances of MD5 are still as good as ever.

From this we can infer that when weaknesses are found in hash function, they do not appear overnight: we have quite some time to react. Also, the first MD5 weaknesses were discovered only one year after its publication, and that was in the early 1990s when the public research in cryptography involved much fewer people than nowadays.

Let's see what this gives for SHA-256: first published in 2001; ten years later (2011), we still have no clue whatsoever on the slightest hint of a weakness. This would be suggestive that SHA-256 is indeed robust, and collisions for SHA-256 are not just right around the corner. Also, I have not looked in full details at the Bitcoin protocol, but it seems that collisions are not a real danger for Bitcoin -- it rather relies on preimage resistance, for which not only SHA-256 is rock solid, but even MD5 would still be reliable.

However, it is dangerous to make statistics on a single measure. In 2007, it was estimated that there was a relatively high risk that the attacks on MD5 could be transported to SHA-1 and then SHA-256/512 -- this prompted NIST to organize the SHA-3 competition. It turned out that attacks on SHA-1 have somehow stopped progressing, and there is no attack on SHA-2. Whether this is because SHA-2 is really robust, or because all the cryptographers are busy trying to break the SHA-3 candidates, is not known (but my opinion is the former: SHA-2 is a secure hash algorithm).


How to defend vs. Image Remote File Inclusion, e.g. RFI using a .gif file (Apache/PHP)? To defend versus Remote File Inclusion where attackers try to abuse image files, I usually recommend to never use include to include image files into PHP code.

Sometimes though, the avoidance of image includes may be not possible at all (for whatever reasons, doesn't matter).

In such case I usually pickup the images somewhere in the upload process and convert them to another image format (sometimes im combination with a lossy compression) to hopefully destroy any malicious code possibly contained in the original image.

This works, but I'm not quite satisfied with it. Mainly because of the additional server load such processing produces and possible image quality decreases that sometimes may happen.

Are the any smarter ways or best practices on that?

EDIT

To clarify: I'm talking about a siutation where the attacker injects PHP code into an image file to get the injected code executed on server side after uploading the image. Forums for example allow users to upload avatars (small image files) for personalisation.

Your PHP code should never evaluate a data file. It may print out the content of an image file, but include()ing an image file is an insane use of code. I strongly assume that what you're doing is using PHP code to serve images, and because PHP doesn't evaluate anything that isn't between "<? ?>" tags, this works in most instances. That is definitely the wrong approach.

You may be looking for the readfile() function.

Other than that, don't write include statements that take external input.


When given a SSL-Private-Key, how would one generate a valid SSL-Cert from this? in the hybris of leaked ssl-private-key everybody cries for new certs and new keys.

my question: how would one generate a new and valid ssl-cert with the private key from my website? our certs are usually domain-validated, meaning that ssladmin@domain.com usually gets a ntoficiation on issued cert-requests

p.s. i'm not talking about how agencies would do this, but a hacker that possibly obtained the private key

Web sites hand out the public certificate, that's the way TLS authentication (identity) works (excepting aNULL ciphers). If, for example, you were able to use Heartbleed to steal a site's key, the required certificate would already have been given out by the server during the handshake (for every correctly set up TLS connection).

No creation of a certificate is required. Were it required or desired, then a CSR can be generated from the key material. Then that CSR could be signed by a commercial CA, your last line of defence is the weakest verification policy of the most lenient CA. Not very comforting ;-)

Clients which implement certificate pinning should not, in theory, fall for such a bogus certificate.

You could also sign it with your own private CA, the resulting certificate would be valid in a technical sense, just not accepted by clients (which verify CA chains, and won't recognise your private one).


Which smartfone OS/vendor is the safes when it comes to exploits? If I understand correctly, when an exploit for Android is found I have to manually patch my system because even though Google fixes the bug, vendors like Samsung, Huawei don't recompile their systems after every change of Android core.

So what is the best way to protect myself from exploits?
Here are the options I can think of:

choose a smartphone for which there's an open-source system that's often updated - like CyanogenMod, and flash that system to the phone
forget Android and get smartphone with Ubuntu Phone
perhaps there are some vendors that update their systems very often? maybe OnePlus?

Answers regarding iPhones and Windows Phone are welcome for completion sake but I'm not interested neither in Microsoft nor Apple products. Only open source stuff.

The answers given in this earlier question might perhaps be of some help to you?

Mobile Phone security - help finding security/privacy review/discussion

In addition to Ubuntu Phone, as stated in an answer to that question, there is also Sailfish OS, but, unfortunately, both have a very small market share, and hence few apps, and hence a small market share..


How does ASLR work if in the assembly code the addresses are the same Let's say I have this piece of code that changes the 10 address to the value 20 and the following one to 30
mov ebx,10
mov [ebx],20
add ebx,1
mov ebx,30

How can the address change each time it is executed?
is it require change that the compiler do to allow it

The addresses are translated
Unless you are working on firmware or maybe hypervisor code, the addresses in your assembly are not physical but virtual. The lower ring levels (kernel, hypervisor, firmware) have Page Tables in their own memory space to map the virtual addresses into physical ones (or at least corresponding to their own address space when the hardware supports nested paging) and the Memory Management Unit keeps a cache of those mappings to make memory accesses as fast as possible.
When ASLR is active, every time you run an executable the kernel allocates the stack, the heap and instructions into new randomly generated memory offsets so the attackers have no way to be certain about the exact address the overflown buffer is at. Thus, they cannot feed it to the Instruction Pointer when RET is executed and, if by chance they managed to reach it, with W^X they still would have to find out the right instruction addresses of the functions they want to return to to avoid segmentation faults. The same applies to object pointers on the heap.
Attackers usually use NOP slides and Heap Sprays in order to deal with address uncertainty, or either try to to access them on the Global Offset Table or find other leaks instead.

How can the address change each time it is executed? is it require
change that the compiler do to allow it

It needs to be a Position Independent Executable: the addresses need to be defined as offsets rather than absolute inside the executable and linked library files, and that is done by the compiler. When the Dynamic Linker loads them and allocates them what it's basically doing is generating a different random base to multiply against those offsets.


Scanning A Subnet Externaly Scenario:

I have a router with a public IP address provided by the ISP and two hosts behind the router with a private IP (192.168.x.x).

Is it possible to perform a port or vulnerability scan of a private subnet behind  public IP address?

Usually, such networks are behind Network Address Translation (NAT). NAT waits for an internal system to send out a packet to a WAN address, then stores the route in a table. It alters the packet to have the router's WAN IP as the source address, and sends it to the internet. When the response comes back, it matches the incoming packet to a route. This is especially easy to demonstrate with TCP, where a connection has a sequence number that can be used to match the packets to the conversation.

The problem with accessing internal ports is that if a packet doesn't explicitly match a route that the router knows about, it is dropped. Note that port forwarding rules create their own routes, so the router knows to route incoming TCP / UDP packets to a particular internal IP. As such, it's quite difficult to enumerate internal IPs from the internet, let alone do port scans.

The only way I know of to do this is to exploit the router. Some of them have badly set up UPnP daemons, allowing you to trick the router into setting up port forwarding rules. There are some other ways to do this, usually via the router's control panel.

Aaron "finux" Finnon (@f1nux on Twitter) did a great talk about pwning UPnP at BSidesLondon earlier this year: http://www.securitytube.net/video/4140


My old job has massive security exploits in their product, but they dont care A company I used to work for developed a Point Of Sale system that also has an eCommerce portion. While working there, I discovered massive flaws with their security model.

Simply put, there is 0 server side validation.
Any user, logged in or not, can do things like edit prices, fake transactions, mess with time sheets, etc all from the comfort of their home.

I reported this several times verbally, but it was mostly ignored as not a priority.

They have since expanded their clientele, and now serve quite a few clients.
I have verified that the exploits still work just as they did several years ago.

I have no interest in saving face for this company, they treated me and many others very poorly and abusively, forcing overtime without pay and similar transgressions.

What is the best way to report this security issue? Do I send an email to their clients? Or should I publicly disclose the attack and let the internet handle it?

edit:
I would like to add that the exploits are not complicated or obscure, anyone who opens devtools in their browser would be able to figure out that they can just edit the data on the fly

edit 2:
After reading all of these responses, I will be sending the company an anonymous email with a POC. I will also be giving them a 60 day period to address the issue before I report it to their customers

A quick word of caution
You sound very invested in trying to do something.  If I can be frank, it sounds like you are at least partially motivated by your frustrations with the way the company treated you as an employee.  This can be understandable, but doesn't necessarily lead to good decisions.  In particular one option you mention (contacting customers and informing them of the issues) is the sort of thing that, rightly or not, and regardless of how it might end, can result in lawsuits filed against you by your former employer.  So tread lightly.  All things considered that is probably a very bad idea.
Understanding the (flawed) business perspective
Unfortunately the situation you describe is not uncommon in the software industry.  From the perspective of many companies, bring up security issues is asking them to spend real money (in terms of developer salaries) to fix a problem that they cannot see for a benefit that may not be needed for a long time (it may in fact be a while before someone tries to hack them).  It's a hidden benefit with an upfront cost, and that is something that short-sighted thinking can easily ignore for a long time.  After all, from their perspective, the things you are making a big deal about have never actually caused problems but will definitely take a lot of time and effort to fix, so why should they fix it? (I'm not saying they are right, I'm just explaining the thought process).
It's important to understand that this is obviously the approach that your ex-employer is taking.  You have informed them of the issue and they have decided to ignore it.  There is no reason to think that any (legal) action you can take as an outsider is going to change that, especially since you failed to make any changes as an insider.  Of course we know that with bad security practices, someone finding and exploiting a weakness is inevitable, especially if they start to see any real kinds of success (i.e. having a large customer base).  As long as none of their customers take the time to delve into potential security concerns proactively (and most don't, because they don't know enough to look properly even if they do care), situations like this can go on for a surprisingly long time before it causes real problems.  In the worst-case scenario this leads to situations like the equifax security breach.  For smaller companies this can result in complete bankruptcy.
Reality
So what do you do about it?  If management knows about the problem but refuses to change, there probably isn't anything you can do to force them to change.  You can try things you mentioned like reporting this to their customers, but their customers may not take you seriously.  For all they know you are simply a disgruntled ex-employee trying to cause trouble for your previous employer.  If they didn't know enough to look into these things before starting to use the platform, then there is no reason to think they know enough to take your claims seriously now.  More likely than not you'll just end up being ignored or sued.
(per @forest's answer) You certainly should submit a CVE.  You could also try submitting bugs through a third-party bug bounty program.  There are some that have popped up in recent years and exist to try to act as a neutral arbiter between "independent security researchers" (aka you) and sites that otherwise don't have bug reporting programs (aka your former employer).  Of course such programs work only if the company you are reporting bugs to actually listen.  You already know that your former employer won't.  However, having published and ignored vulnerabilities through standard channels will help their customers in the long run when they do get hacked. This will change the situation from simple incompetence to outright negligence, which comes with much stiffer financial penalties in civil court (FYI: IANAL).
Any further (legal) options will vary depending on your jurisdiction, aka in Europe you may find some venues for legal action through the GDPR.  In many places though you probably don't have any options that can immediately bring legal trouble to them.  Most likely that won't happen until they get hacked and their customers sue.  Having a published CVE will help their customers when that happens.  In the meantime, it sounds like you are thinking about posting something publicly "on the internet". What would be your goal there?  Realistically, your attempts to do that will probably just be ignored by the internet.  It's possible however that they may end up hacked much sooner as a result though.  Without going through more standard channels though, you probably dramatically increase your own risk of legal repercussions.  Therefore, I would personally keep to official channels.
Again, I may be misreading you, but I think your question is largely coming from the place of someone who is angry at a past-employer and is, to some extent, looking to cause trouble.  That's a good way to get yourself in legal trouble or at least give yourself a bad name when trying to find jobs in the future.  I think you should try to stop looking at this from the perspective of an ex-employee and focus more on the perspective of a neutral-security-researcher.


Why do Chrome extensions need access to 'all my data' and 'browsing activity'? Often I install simple add-ons for Google Chrome. However, they almost all need what appears to me as excessive access to my data. For instance, I install an extension that allows me to click on any part of a page and it gives me the color of the clicked object. This is what this extension requires:

Why does it need my data on all websites? (Also, what does this imply?). And why does it need to know my browsing activity? I see this for almost all extensions, and it looks highly suspicious.

I install a plugin that allows me to click on any part of a page and it gives me the color of the clicked object

For this to work, the plugin needs to register a click event handler and it needs to interact with the document object model of the current HTML page.

HTML was originally designed to share scientific documents. And while we build complex web applications today, it remains document orientated at its core.

The browsers cannot know what parts of a HTML page contain sensitive information. For example if you login to StackExchange, it will display your username in the top navigation bar. On your bank website there will be information on your money transfer and account balance. There is no structural difference between those informations and a text saying "Your answer".

Chrome does offer some protection against malicious plugins by preventing them from arbitrarily accessing your harddisk. But a plugin that needs to interact with webpages, can do anything to them. All other browsers have the same issues, they just don't go into detail.


How does TLS work (RSA, Diffie-Hellman, PFS)? I know "How TLS works" has been discussed numerous times here and crypto, but I am still somewhat confused and would like to summarize what I know so far 1 in this giant blob of text with the hope that one day this becomes helpful.

There are two popular TLS key-exchange methods: RSA and DH. In either case, the typical TLS Handshake looks like this:

Client sends a ClientHello message which contains the maximum TLS version that it supports and a list of cipher suite in the order of preferences. In addition, a random 28-byte value called ClientHello.random is also transferred.
The server replies with a ServerHello message with the best cipher suite and version it can support, along with its own random 28-byte value called ServerHello.random and a digital certificate.
The client verifies the server's digital certificate against its trusted CA store. Then the client creates a pre_master_secret, encrypts it with the server's public key extracted from the server's digital certificate, and sends that back to the server. This is known as the ClientKeyExchange.
The server decrypts the message using its private key, and then generates a master secret.

The way master_secret is generated in TLS 1.2 is as follows 2:

  master_secret = PRF(pre_master_secret, "master secret",
                      ClientHello.random + ServerHello.random)
                      [0..47];

Afterward, the client also sends ChangeCipherSpec record (6 bytes) to the server, indicating it wants to use symmetric encryption as well as Finished message.
The server responds back with ChangeCipherSpec as well as Finished message.
From this point onward, all traffic will be communicated over TLS and are encrypted.

Question 1:
What is "master secret" in the derivation? In other words, what is the actual value?

Question 2:
How does client encrypt its message? What key/secret is used?  I am not sure the role of master_secret?

Question 3:
In 3 (RFC section 7.3), it says the following, what are those "random values" and what are their purpose?

  Generate a master secret from the premaster secret and exchanged    random values.

Question 4:
I often read the term "session key". What is it? Is it the master_secret?

RSA

I know that RSA can be used in step 3 above for integrity control, meaning using public-private key asymmetric encryption so pre_master_secret is not readable in plaintext.

DH
The major weakness of using RSA is that using server private key. An attacker can record all traffic and decrypt traffic if the server's private key is compromised. So to provide forward secrecy 4, DH can be used.

DH works on the principle of discrete algorithm. The mathematical properties allow both sides (the client and the server) to generate its own secret (a, b respectively), and derive to the same shared secret given p, G and g^X mod p (where x is a and b respectively) over the public channel (the world can read them) 5.

Question 4:
I believe all subsequent traffic will be encrypted using the shared secret, correct?

Perfect Forward Secrecy (PFS)

To ensure no one can read prior logged traffic, PFS was introduced. Basically, instead of using a long-lived shared key, client and server generate short-lived session keys which are discarded from memory.

Question 5:
What are the short-lived keys? X (a, and b respectively) of client and server's?

PSF and RSA

My understanding is RSA is used for authentication (what 'server' sends is coming from 'server', not MiTM). There's HMAC for integrity checks, which is generated during key exchange.

Question 6:
Is that right?

I might be omitting other relevant questions/details. But any response is appreciated.

Frankly I don't expect this to be terribly useful. Be that as it may ....

As a preliminary matter almost everything you say is for TLS up to version 1.2 only. TLS version 1.3, which makes fairly major changes in the protocol, was released last year (after a long delay) and is now in the process of spreading; based on historical experience it is likely that TLS<=1.2 will be pretty much gone in something like 3 years. To be fair, most of the resources you can easily find online, notably including the Ursine Epics at #20803, pre-date 1.3.

  In either case, the typical TLS Handshake looks like this:

Not either, what you describe covers only RSA; DH is different. More below.

  ... In addition, a random 28-byte value called ClientHello.random ....
  ... ServerHello message ... along with its own random 28-byte value called ServerHello.random and a digital certificate.

The fields named .random are actually 32 bytes, split into a 4-byte timestamp (which is not random if your computer's clock is even vaguely correct, as it should be) and 28-bytes of actual random data. The value used in key derivation etc. is the 32-byte value.

Strictly speaking the server certificate is not in the ServerHello message, it is in a separate message. However, both these messages, plus ServerKeyExchange when applicable and ServerHelloDone always, can be part of one record and are usually part of one TCP-level transmission. More substantively, if the server cert requires one or more intermediate or 'chain' cert(s) to be verified, which is almost always the case nowadays, that(those) chain cert(s) should be included as well; there are many Qs on several Stacks about "browsers consider my server connection secure but $other_sw gives $some_error" and this is often due to not correctly configuring a chain cert. (The A often varies depending on the server software involved.)

  The client verifies the server's digital certificate against its trusted CA store. Then the client creates a pre_master_secret, encrypts it with the server's public key extracted from the server's digital certificate, and sends that back to the server. This is known as the ClientKeyExchange
  The server decrypts [premaster] using its private key, and then generates a master secret

The client verifies the server cert, (usually) via its chain, against the client's truststore, AND verifies that the server cert matches the name (or possibly address) of the server the client wants to connect to. (If we want to connect to HonestBank.com, and trying to connect gets us a cert that was issued by a trusted CA to WeAreCrooks.com, we don't want to send our bank info on that connection.)

Both the server and the client derive master_secret from premaster and the 2 random's.

If the server requests client authentication, also called client certificate or 'two-way' or 'mutual' authentication, the client actually sends Certificate before ClientKeyExchange and CertVerify after. This is all explained in 5246, but is rarely used.

  Afterward, the client also sends ChangeCipherSpec record (6 bytes) to the server, indicating it wants to use symmetric encryption ....
  From this point onward, all traffic will be communicated over TLS and are encrypted.

After CCS all traffic is encrypted and authenticated; both are important. The methods vary: older ciphersuites use a (pure) cipher to encrypt and a (separate) HMAC to authenticate (HMAC = Hash-based Message Authentication Code); 1.2 also had new (in 2008) authenticated ciphers, officially called AEAD = Authenticated Encryption with Additional Data, which do both encryption and authentication in one combined operation; compare section 6.2.3.3 to the immediately preceding sections.

  Question 1: What is "master secret" in the derivation? In other words, what is the actual value?

It's different for every session, and nobody except the two endpoints (client and server) should know it, hence 'secret'. (Although sometimes debugging features let you extract it; there are several Qs on those.) Its value is computed using the formula you posted, from 8.1. In case the 'search' function on your browser is broken and some flaw in your display makes the table of contents invisible, PRF abbreviates Pseudo(R)andom Function and is explained in section 5.

  Question 2: How does client encrypt its message? What key/secret is used? I am not sure the role of master_secret?

The master_secret is used to derive multiple working keys, or more exactly secrets; see section 6.3. The client uses the 'client_write_key' to encrypt, and the server uses it to decrypt. For ciphersuites that use IVs, which in 1.2 is only some AEAD ones, they also use the the client_write_IV. For ciphersuites that use HMAC, which is the non-AEAD ones, the client uses client_write_MAC to generate the HMAC, and the server uses it to verify. See Are session keys just the symmetric keys? or cross https://crypto.stackexchange.com/questions/1139/what-is-the-purpose-of-four-different-secrets-shared-by-client-and-server-in-ssl .

  Question 3: In 3 (RFC section 7.3), it says the following, what are those "random values" and what are their purpose?
  - Generate a master secret from the premaster secret and exchanged random values.

This is exactly the formula you posted in your 3 from section 8.1. The ClientHello.random sent to the server, and ServerHello.random sent to the client, are exchanged random values, and are combined with the (shared) premaster_secret to generate the (also-shared) master secret.

  Question 4: I often read the term "session key". What is it? Is it the master_secret?

It can be either the master_secret, or the derived working keys/secrets (plural), or both. In particular, session resumption (aka re-use) in TLS<=1.2 is done by saving the session-id (in ServerHello) and the corresponding security parameters including the master-secret, and then using them on a subsequent or even concurrent connection.

  RSA

  I know that RSA can be used in step 3 above for integrity control, meaning using public-private key asymmetric encryption so pre_master_secret is not readable in plaintext.

'Plain' RSA keyexchange does use RSA encryption, which is a type of asymmetric encryption aka public-key encryption, so that pre_master_secret is not readable. Although asymmetric or public-key cryptography does use public and private keys, we don't normally say 'public-private key'. I have no clue what you mean by 'integrity control'; RSA encryption does not much resist an adversary manipulating the ciphertext, which allowed an attack by Bleichenbacher that remains an issue. The (only) protection on a plain-RSA handshake is the PRF values in the Finished messages, which functions as a kind of MAC (as long as at least one endpoint is honest and correct).

  DH The major weakness of using RSA is that using server private key. An attacker can record all traffic and decrypt traffic if the server's private key is compromised. So to provide forward secrecy 4, DH can be used.
  DH works on the principle of discrete algorithm. The mathematical properties allow both sides (the client and the server) to generate its own secret (a, b respectively), and derive to the same shared secret given p, G and g^X mod p (where x is a and b respectively) over the public channel (the world can read them) 5.

That's discrete logarithm. More exactly, Diffie-Hellman ephemeral provides forward secrecy; it is the 'ephemeral' that is critical. 1.2 (and earlier) also defines static (non-ephemeral) DH keyexchanges, but these are practically never used and serve mainly to cause confusion. (They are deleted entirely in 1.3.) There are technically two variants: the original DH-ephmeral using integers, designated DHE in TLS; and the elliptic-curve version, designated ECDHE. Although the same principles apply to both, the actual code (and data) to implement them is quite different.

  Question 4: I believe all subsequent traffic will be encrypted using the shared secret, correct

Not directly. The secret generated by [EC]DHE agreement is used as the premaster-secret, in the same fashion as above: first derived to the master secret, then to the working keys/secrets. Compare sections 8.1.1 and 8.1.2, immediately after the excerpt you posted.

  Perfect Forward Secrecy (PFS)
  To ensure no one can read prior logged traffic, PFS was introduced. Basically, instead of using a long-lived shared key, client and server generate short-lived session keys which are discarded from memory.
  Question 5: What are the short-lived keys? X (a, and b respectively) of client and server's?

Yes. Except that although generic DH is often described in terms of a/A and b/B (canonically, Alice and Bob), the TLS specs use different notation. For integer-DHE in 5246, the public keys for server and client respectively are dh_Ys and dh_Yc; (corrected!) the corresponding private keys presumably are Xs and Xc, but are not shown. For ECDHE in 4492, the server public key is simply named public while the client one is named ecdh_Yc (even though in ECC generally we use X,Y for coordinates of a point, and call the privatekey (integer) d and the publickey (point) Q) and again the private keys are not shown.

  PSF and RSA
  My understanding is RSA is used for authentication (what 'server' sends is coming from 'server', not MiTM). There's HMAC for integrity checks, which is generated during key exchange.
  Question 6: Is that right?

(That's PFS. Or just FS.) I'm not at all sure what you're saying, so to mostly repeat what I said before:

for RSA keyexchange, the premaster is encrypted by RSA with the server's publickey in its certificate, and the only integrity check on the handshake is Finished, which uses PRF, which is based on but different from HMAC
for [EC]DHE keyexchange, the keyexchange parameters are signed with the server's publickey in its certificate. That key and thus the signature may be RSA (in either case), or it may be DSA (also called DSS for historical reasons) or ECDSA depending on the keyexchange
regardless of the keyexchange, depending on the cipher either HMAC or AEAD (but not both) is used to authenticate the data traffic


Why is nmap being so noisy with "-A" option? Just a quick question. I run http server and when i scan it with classic "nmap -T5 -O -v [IP]" i see absolutely nothing in log. But when i do "-A" instead of "-O" i see lots of things that are OBVIOUSLY coming from nmap scan. In the user agent is "Nmap Scripting Engine" and few other things are there. Why isn't it at least try to hide itself ?

It's obvious that doing "-A" is gonna be noisy but it could be at least 'a bit less' noisy.
Also, how do i change those defaults ?
Thanks

The decision whether to try to avoid detection by default or to provide unique traffic signatures is a tricky one. In the long run, it becomes an arms race, where the attacker must always keep up with the defender or risk detection. Nmap has historically offered means of evading specific types of detection, but has not turned them on by default: packet fragmentation, source port spoofing, connection throttling, Idle/zombie scanning, etc. The capability is there, but the default is to be honest about what we are attempting to do.

NSE scripts have similar capabilities: you can slow them down with --max-parallelism and related options. You can change the user agent string with the http.useragent. You can send your traffic through a proxy with --proxies. But by default, Nmap is very honest about being Nmap.

The trouble is that if any of these behaviors were made the default, they would cease to be stealthy. When Nmap was first written, its default TCP scan mode, half-open SYN scan, was the stealthiest thing around, because the TCP handshake was never finished, so there was nothing in the server's application logs about a closed connection. But as this scan mode became so much more popular, network IDS capability soon expanded to catch it, since the "SYN, SYN-ACK, RST" behavior is so very unusual and noticeable. But Nmap didn't change its default simply to avoid IDS. Instead, it kept the reliable method that had the least impact on ignorant systems—you might be surprised to know how many daemons will crash if you simply connect and hang up (-sT).

I've written a more thorough analysis of Nmap's defaults, how they can be detected, and how to change them to avoid detection. It's on my blog in a two-part article called "They see me scannin'". I have a feeling you would find it interesting.


Can accounts still be logged onto if Password Caching is not enabled? This is kind of an extension of my other question.. If you were not storing any of the passwords locally (caching disabled), would a user some how still be able to circumvent around this security measure? I am not quite sure how this all ties into the registry / SAM & other files.. So would someome still be able to use a program like ONTPRE and add in an administrative password to the SAM file or log in by some other means? (Through sniffing or other means.. I assume the other means would generally be easier, if they exist)

In a nutshell, this is my question: The question is it still possible to reset the SAM file or some other method to get onto the computer without accessing the domain?

is [it] still possible to reset the SAM file or some other method to get onto the computer without accessing the domain?

Of course. If you have physical access to the machine, you always have the ability to reset the local security database with known password values, thus giving yourself access. This is due to the very design of the PC architecture and cannot be changed, no matter what operating system you are using.

But this would be local access to all data stored on the local machine - you would not have any online domain resources available. You would still need valid domain account credentials for that.

The risk for accessing the "data at rest" (i.e. data which is persistent even when the operating system is not running) can be alleviated by using encryption mechanisms which are tied to user input or removable keys (like smartcards) - for example with EFS.


Can I use this webservice safely? I'm trying to integrate with my host using their REST webservices.

However, the authentication for the webservice only requires two things:

An API token which I can retreive from their portal once
My username to access their portal

Both need to be sent in the headers, under the names 'ApiKey' and 'PortalUsername'. It must be sent over HTTPS.

How safe is this really? I'm not too technical but I do know a few things about webservices and authentication. HTTPS is not the solution to protect against every kind of interception. I once integrated with some other webservice and they required the API key to be hashed and digested via some path, that seemed safer to me.

I'm eager to learn more on this subject and know wether I can safely use this webservice (with access to quite sensitive information), or not based on how they setup the authentication.

Everything that goes through HTTPS is enough as long they use a certificate trusted by your browser. This means it was forged by 3rd-party acknowledged authority of certification.

If they use their own certificate. You may reconsider that statement.


Is it possible to force a new SSL session Using tools like Wireshark, it's possible to decrypt SSL traffic as the client by logging the (pre)master secret, without using a mitm attack or the server's private key. But I found that for some websites, the ssl session id is being stored by the server (evidence: the ServerHello message contains the re-used session id), and (I believe) this prevents a full ssl handshake from occurring, which prevents wireshark from successfully decrypting. Admittedly I am not totally sure that this is the problem, but the evidence is quite convincing.
Is there any way for the client to prevent re-use of ssl session ids?

In SSL, the client and server may engage in an abbreviated handshake only if both client and server remember the "handshake parameters" (in particular the negotiated pre-master secret). The "session ID" is how the client and server advertise their remembrance: the client sends in its ClientHello a copy of the previous session ID, and the server sends it back to acknowledge that this session is going to be reused.

You cannot remove the session ID from the client message without breaking the subsequent cryptographic operations, because the Finished messages at the end of the handshake are computed as hash values over all previously exchanged handshake messages, which includes the ClientHello. Thus, the client and the server MUST "see" the same ClientHello. (However, see below.)

Therefore, to prevent an abbreviated handshake, you have to enforce forgetfulness. SSL servers can be configured not to remember sessions beyond some inactivity timeout; for instance, see Apache mod_ssl directives SSLSessionCache and SSLSessionCacheTimeout. Shutting down and restarting the server may also flush the cache, as long as the cache is kept in RAM (with multi-frontend servers, the cache may be shared with a common database or something similar, in which case cache purging can be more complex).

On the client side, SSL sessions are normally kept in RAM. Internet Explorer, for instance, internally consists of several process that talk to each other, and you have to kill them all to make it forget a SSL session (in practice, this happens only when you have closed all the IE windows).

Your quote, however, is weird:

  Using tools like Wireshark, it's possible to decrypt SSL traffic as the client by logging the (pre)master secret, without using a mitm attack or the server's private key.

Wireshark cannot do that. Wireshark inspects data packets in transit, and from these packets, you cannot infer the pre-master secret without having a copy of the server's private key. That's the whole point of SSL: to prevent eavesdroppers from being able to make sense of the exchanged data. To be able to break into the SSL session from a "network interceptor" point of view, you have to do a Man-in-the-Middle attack, with double impersonation; and this may happen only if you bypass the certificate security model: you have to convince the human user (SSL client) to install a fake rogue root CA key, or to "click through" the red and scary warning that the browser displays when it notices that the alleged server certificate is funky.

If you can "log the pre-master secret" then this means that you have privilege access to the RAM contents of either the client or the server; that access would not use Wireshark, but a debugger. What Wireshark offers is the possibility to use some captured pre-master secret (e.g. see this page); with this information, Wireshark will be able to decrypt the data. However, it does not (and cannot) do the capture itself: this requires direct access to the client or server RAM, or a modified client or server to leak the information. To make an analogy, you cannot do brain surgery over the phone.

If you indeed have a debugger-like access to the client (or the server), then you should be able to manipulate the client (respectively server) to make it "forget" a previous SSL session, thus forcing a new full handshake.


Safely creating script tag and attaching to DOM I have a need to to create and attach a script tag to DOM at runtime.
The script is a remote webpack bundle (via a Module Federation plugin). I would like to be able to change the URL to that hosted remote bundle at runtime (for A/B testing and debugging purposes).  This is my own code, and I trust it as far as not having malicious code in it.
The code would look something like this
var myScript = document.createElement('script');
myScript.src = 'https://my.remote.domain.com/path/to/myScript.js';
document.head.appendChild(myScript);
... other code to make the webpack bundle run

My questions are:

Is it safe to allow passing the https://my.remote.domain.com/path/to/myScript.js as a URL query param. Ex: https://my.site.com?scriptUrl=https://my.remote.domain.com/path/to/myScript.js? The code would look like this:
var urlParams = new URLSearchParams(window.location.search);
var myScript = document.createElement('script');
myScript.src = urlParams.get('post');
document.head.appendChild(myScript);

It feels like this could open up a hole for an attacker to pass in a URL to an attacker script and execute it on my page. Am I on the right track here, or is this not a concern?

What are the ways to secure the code above and allow the dynamic script attachments?

If this code is an issue, what are the specific attacks that could happen here?

Is it safe to allow passing the [script URL] as a URL query param?

No, this is very dangerous. See question #3.

If this code is an issue, what are the specific attacks that could happen here?

Since an attacker could pass any script into the URL, and have it run in your sites origin, you are basically opening yourself up to XSS attacks. That is very bad.

What are the ways to secure the code above and allow the dynamic script attachments?

You need to somehow make sure that only scripts that you trust can be used. I can see two possible ways to do that:

Whitelisting. Instead of having the script URL as a query param, either have the value A or B for your A/B-testing. Then you keep a dictionary in your code of what URL A and B refers to. This way, only scripts you have allowed in the dictionary can be used.
Enforce that the script only comes from domains you trust. You will need to be careful about file upload functionality and open redirects with this solution though.

If not to impractical, I would go with the whitelist.
It might be tempting to use CSP for this, but I would strongly advice as using that as your only line of defense here. Browser support is not perfect, and it is easy to loose the header with a small server configuration mishap.


Are smartphone apps theoretically capable of detecting what a user is looking at? For instance, would it be possible for an app to determine what pixel range on a smartphone display a user is looking at by analysing their eyes with the front facing camera? If so, with what kind of precision?

It would be very discomforting to know that apps could collect data in the background on how your eyes respond to displaying certain advertisements.

Eye Tracking for Everyone. 2176-2184. 10.1109/CVPR.2016.239. (2016) - Krafka, Khosla, Kellnhofer et al

  Our model achieves a prediction error of 1.71cm and
  2.53cm  without  calibration  on  mobile  phones  and  tablets
  respectively. With calibration, this is reduced to 1.34cm and
  2.12cm.

So yes - it is possible. This particular study was performed using iOS and achieved a read rate at 10–15fps.

There are several companies selling products with similar technology - UMoove for example. It would not surprise me if a higher precision than in the mentioned paper could be achieved.

If you are paticularly concerned a number of smartphone camera covers are available - here is one example.


Ensuring File is Write-Only For Specific Program So in this scenario there are several programs that exist that can open and read a binary file, but my program must be the final arbitrator in creating and writing to these files.

In this program I am writing, I would like to detect if the file has been tampered with outside of my program's control. It is important that the other programs can still read the file, but that my program can refuse to open it on grounds it was edited outwith my program.

My solution is to distribute the file(s) with a list of checksums. If the corresponding checksum from the checksum list doesn't match the accompanying file, or the list does not exist, then that file is rejected.

My solution would be to calculate the hash using the following method:

sha256(HIDDEN_SALT + FILE_CONTENTS)

The HIDDEN_SALT being the key property that ensures outside sources couldn't spoof the hash.

In this situation, as the salt is applied to multiple files (1000+), would it be easy to crack by working backwards from the checksum(s) it was applied to? (Consider they have both the checksum and the file.) Would there be a better solution to this problem?

There are actually two problems to solve for:

that the file has not been tampered with since creation by a trusted party
that a trusted party actually created the file in the first place

This is exactly the scenario that open source software distributors- RedHat, Ubuntu, MySQL, etc, etc- face in ensuring the integrity of the binary packages they release.

Their software, running on a consumer's computer, has to ensure both that a package the user has asked to install has not been tampered with by an attacker, and that it originated with a trusted source.

Hashing will help with the former, but public key cryptography is needed for the latter. The distributor- you and/or your delegates- publish their public key, and then cryptographically sign the distributed software using their private key.

The way this is usually done is by creating a cryptographic hash of the file contents- which can just be of the contents, and does not have to include a salt or anything- and then sign the hash, and distribute the signature, the hash and the file, along with the public key.

Then anyone can use the public key and the signature to verify that the hash originated with the holder of the private key, and then that the hash matches the file.

Take a look at "package signing" documentation from RedHat or Ubuntu using PGP keys for more information.


Extract data from burp response and use in another request I have a scenario where I need to call /api/login to get a token and then use said token in subsequent calls until it expires.

Is there a way to automate the call and token extraction and then to auto add it to the next requests?

I have tried making a macro and I can define a regex to get the token - but I can't seem to make it auto add to my next requests in the repeater.

I made a session rule to call the macro on each request, and it does get called but the token isn't being added

Burp does not have a native functionality for that. You'll need to either write an Extension or use an existing one. I'd recommend Token Extractor.

Otherwise, you could also use Python for that. Proxy through localhost if you want to see the traffic in Burp. Do a regex parse (or json, if your data allows) and get the required token to use in the next request.


Firefox vs. Chrome extensions: Which are---ceteris paribus---more secure? As an 'extension' to earlier questions on this topic (for instance here and here),

What makes Firefox extensions (appear) more secure than Chrome extensions?
Do they just appear to be more secure, or are they really?

See also:

Why do Chrome extensions need access to 'all my data' and 'browsing activity'?
Worst case scenario, what can a Chrome extension do with "Your data on all websites" and "Your tabs and browsing activity"?
Are official browser add-ons really safe?
http://lifehacker.com/5990769/why-do-chrome-extensions-need-to-access-all-my-data
https://support.google.com/chrome_webstore/answer/186213?hl=en-GB

Your assumption is incorrect, Firefox addons are not inherently more secure than Chrome extensions (though in terms of security, I attach more value to Firefox's official addon gallery (AMO) than the Chrome Web Store because all addons on AMO are manually reviewed).

Addons in Firefox are trusted by design; they can do anything that is allowed by the Firefox process.
The worst-case scenario for installing a malicious Firefox addon is a re-install of your operating system to clean up the mess.

The Chrome extension APIs are very constrained, because the Chrome browser does not fully trust extensions (unlike Firefox). Chromium extensions can typically not access any resources outside Chrome's sandbox without the user's approval. The worst-case scenario in Chrome is less severe than Firefox' (and also applicable to Firefox): All of your web browsing activities can be considered compromised.

Since most of us increasingly spend more time in the web browser than in native applications (e.g. internet banking, e-mail), it is a stupid act to install extensions that you cannot trust. Considering this view, then Chrome extensions could be more secure because you have to consent to the every newly requested permissions upon installation/update. Firefox has not implemented any addon permission warnings, so when you install a Firefox addon, you should always mentally add a "This addon could access all data on your computer and the websites you visit" warning to the installation dialog.

On the other hand, because Firefox addons are more powerful, they can also integrate security features in a much better way than Chrome extensions. For example, to date it is not possible to create a NoScript equivalent in Chrome because of the limited extension API.

If you want to know more about Chrome's extension security design, I suggest to read:

Barth, Adam, et al. "Protecting Browsers from Extension Vulnerabilities." NDSS. 2010., available at http://www.adambarth.com/papers/2010/barth-felt-saxena-boodman.pdf
The security foundations/design of the Chrome extension platform.
Carlini, Nicholas, Adrienne Porter Felt, and David Wagner. "An Evaluation of the Google Chrome Extension Security Architecture." USENIX Security Symposium. 2012., available at https://www.usenix.org/system/files/conference/usenixsecurity12/sec12-final177_0.pdf
An evaluation of the effectiveness of the Chrome extension security architecture.
CRX API Security Checklist
Security checklist for developers of Chrome extension APIs.


Arp poisoning doesn’t work with HTTPS navigation I’m trying to do an ARP poisoning attack in my LAN. I use Ettercap and I place my attacker computer between my routers and target Windows computer.
Despite the target ARP table changing, when I use this computer to visit an HTTPS website, the browser (Chrome) stops the connection. Is there a method to make an ARP poisoning attack and allow the HTTPS navigation in the victim’s computer?

With an ARP spoof, you intercept only the Ethernet layer. To have a ManInTheMiddle position, you need to spoof both client and server and enabling forwarding.
Without that, when your victim computer tries to browse tinder.com, the IP layer is built with the IP of tinder.com and sent to your evil computer (if you spoof the gateway address or the tinder.com address).
If your evil computer does not have a web service listening on port 80, tinder.com will appear unreachable for your victim computer.
Maybe you also intercepted the DNS request and the victim computer never received the Ip to use to contact tinder.com
If you configured your ettercap attack to forward the captured packet to the real destination, you should be able to see the cleartext traffic. I remember that ettercap tries to spoof the SSL layer by default. If yes, your problem could come from the fact that ettercap displays an untrusted certificate to the victim and the tinder.com is configured with HSTS to avoid this.
If you disable the ssl interception (with -S) the victim should be able to reach tinder.com but you will not see the content of the HTTPS layer (you will only see the Ethernet, IP, and TCP layers)


How hard is it to suspend and track a twitter account seeking ransom from govt. organizations? Am asking in reference to this Twitter account - DarkOverlord .
It threatened to post decryption keys, for the documents pasted on pastebin (archived), and few minutes ago it did post them.

Couldn't this account be closed and tracked as well ?
Also, it gives an email-ID for correspondence, are VPN's so secure that this mail account origins couldn't be traced?

Could it be closed? Absolutely. Twitter can close any account they would like and the account owner can do nothing about it short of moving their content elsewhere. However, tracking the account creator is a different matter. If they used an anonymity network like Tor, and sandboxed the browser or otherwise protected themselves from the potential of browser exploits, then they would be almost impossible to track. Actually tracking them would either require using very valuable 0days (which may not even be enough, depending on the level of precautions taken), or by hoping the hacker slips up his OPSEC.


Is it bad to give www-data user access to Postgres client certificate? I have a first server running a Postgres database.
I have allowed remote access to this database, using certificates for authentification (following this tutorial).

Now I have a second server running a Django app on Apache. The user used by Apache is www-data, and I have understood it is good practice to give this user very restricted permissions.

In order to let django access its remote DB, I have added a .postgresql folder in the user www-data home folder (ie /var/wwww) containing all the needed files - root.crt (trusted root certificate), postgresql.crt (client certificate) and postgresql.key (client private key).

Of course if 'someone' gets access to those files, he will be able to connect to my DB and read/write on it.

Is this setup dangerous? If yes, can it be avoided?

Ultimately, the web server needs access to its database.  Using file permissions for a limited user is a tried and true way to do this, so I don't think you are doing anything inappropriate here.  There are some tools - like Hashicorp's Vault, KeyWhiz, etc - that provide some alternatives to this, but even those ultimately need SOME way of identifying the valid user (though they give you some alternatives, like allowing one and only one request from a given EC2 instance which may be more secure in some instances, especially in a situation where you are autoscaling and if an instance fails it is torn down.)


What exactly is an evil maid attack? Today I received the message below, after booting my PC:

What exactly is an "Evil Maid"-attack? What are possible ways to get attacked by this? What does it mean in this case, that my disk "may have been tampered"?

"Evil maid" attacks can be anything that is done to a machine via physical access while it is turned off, even though it's encrypted. The name comes from the idea that an attacker could infiltrate or pay off the cleaning staff wherever you're staying to compromise your laptop while you're out.

For an encrypted device, the most likely evil maid attacks are some form of keylogger, either physical or software. Physical loggers are all-but-impossible to detect in software but can be found by physical examination (although depending on the available space inside a laptop and how easy it is to disconnect the keyboard, it could possibly be done in a way you wouldn't see without disassembling it again yourself...). Software keyloggers are a major threat, though.

The VeraCrypt bootloader (which takes your password and decrypts the encrypted volume or at least the next required part for booting) must itself be decrypted - if it were encrypted, it couldn't execute without something else to decrypt it - so it's a prime target for a place to put a software keylogger. Take an encrypted hard disk, attach it to another machine, replace the VeraCrypt bootloader with a compromised one that secretly saves the password, and put it back in the victim's machine. Next time the victim boots the computer, their hard disk password is logged for later retrieval. VeraCrypt attempts to detect this by computing a cryptographic fingerprint of the bootloader to see if it's been tampered with, although a skilled attacker could thwart this as well unless the machine is using a TPM or similar that checks the bootloader against a key which the attacker can't overwrite.

So, it looks like that verification of the bootloader failed. That's why VeraCrypt is telling you to change you hard disk password immediately. The attacker probably can't steal the password as you enter it - it would be a multi-stage attack instead, first compromising the bootloader and then retrieving the password later after you've had a chance to enter it - but if they get to the machine again after you unlock it once but before you change the password then they could steal that password (and all data on the disk).

Of course, it's possible that something totally innocuous happened and there is no attacker. Anything that messes with hard disks at a low level, like partitioning tools, backup/restore tools, OS installers/updaters, or a few other things could potentially invalidate the bootloader fingerprint without actually doing anything malicious. Changing the password is still probably a good idea anyway, though... when was the last time you did that?


A special file as the second factor in 2FA Can a file e.g. containing some random key that is generated by my web site/app be acceptable as the second factor (the first factor being a normal password) in 2FA?

Someone said so to me in the past, but i doubt about it, because i think a key in a file is not very different from a password in that it is stored on the hard drive and can possibly be read and copied by adversaries.

I don't know much about hardware security tokens (e.g. USB dongles), but seems to me they can atleast be protected against copying/cloning via ordinary and easy means.

As described this scenario wouldn't satisfy any requirements I have for MFA from a vendor or anything, but it is, strictly speaking, a Something you Have. You're perilously close to re-inventing the wheel here, since SSL client certificates and PKI already exist.

A client certificate installed on a device/browser is a valid factor for authentication and combined with a username/password is suitably multi-factor. There are implementation issues with PKI that make it difficult, so it isn't common in many environments.

If you'd like something more whiz-bang there are several options for doing multi-factor with mobile devices that generate codes (TOTP - Authy/Google Authenticator/etc.) or do really fancy push notification and/or one-time passcode generation (Duo Security's service and mobile app).


Blocking direct access to ajax source I made a calculator. I want to prevent that people can use the ajax.php to get the results without accessing the webpage. They have to use the API for that purpose.

I suppose I can add a key (timestamp encoded with symmetrical encryption) inside the webpage. When I will call the ajax.php, I will send also the key. On the server side, I will decode the key, extract the timestamp and check if the key was not generated too long ago.

I can also generate a one time key inside the database.

Do you have better ideas?

You can also use the CSRF token functionality available in most of the web development framework. CSRF is stored in web page's header metadata and must be sent across for each ajax request. CSRF is assigned when a full page is rendered by the server.

If CSRF token doesn't exist in ajax request, or if CSRF is wrong then just don't process the request. This way you make sure the user was logged in the actual interface.
Perform session validation of each ajax request.

You can use these within an API call too.

Still it will help to protect from some web intercept proxies or custom made html web client scripts.


Would a VPN with changing configs do the same job as different VMs I was reading this post: Can a website still detect my real MAC address even if I use different virtual machine each time I visit the website?

So my question:  Would using a VPN and swapping out configuration files that change your IP address mask the same as using different VMs?

They protects against different things. VPN protects you from network fingerprinting, while single-use VM protects you from desktop/browser fingerprinting. To be truly anonymous you may need to use both (among other things), depending on your threat model.

If you want to/are fine with establishing anonymous identity (i.e. others can establish multiple related acts are done by the same person, but they can't track it back to you as a person), then you may be fine with just using a VPN, and but hashing to replace VMs.

But if you want to prevent people from linking your actions together, then you may need both.


How do VPNs protect identity without assuming legal blame? I understand that a VPN is basically a computer (or a collection of computers) that encrypts and proxies your internet traffic. One popular use of a VPN is to pirate software or movies. The VPN keeps you anonymous because the VPN itself is effectively downloading the content and then sending it encrypted to you (so no one can see the communication between you and the VPN). But now the VPN is downloading the illegal content! How can VPN providers do this without taking legal responsibility for having downloaded the pirated material? Is it simply because the VPN servers are in a country where the laws are different?

There are variants of safe harbour policy for digital copyrights laws in many countries. For example, in the US, digital copyright is ruled by the Digital Millennium Copyright Act (DMCA). DMCA Section 512 limits the liability of some entities from copyright infringement:

Transitory digital network communications
System caching
Information residing on systems or networks at the direction of users
Information location tools

VPN providers and ISPs squarely falls under transitory digital network communications limitations, so they're not considered liable for copyright infringement done by their customers.


Is SSH security necessary on an internal network? I am running a VM on a spare laptop we have in the office, and have set it up for SSH access from my workstation. Currently I am using user/password authentication, but I would like to remove the password with the NOPASSWD flag to make it easier to run sudo commands and for my coworkers to use it. My office network doesn't allow SSH access from outside the local network. Is this a bad idea?

In general any authentication with empty password is a bad idea.

  My office network doesn't allow SSH access from outside the local network.

SSH is not the only way an attacker might gain access to the internal network. A password means enhanced security and security means that when a system fails it fails gracefully or at least as gracefully as possible. If the VM is not designed to be a honeypot then in any other case a password is necessary.

  I would like to remove the password with the NOPASSWD flag to make it easier to run sudo commands and for my coworkers to use it.

Then consider using ssh-keys for authentication, it's considered more secure way to login and you don't have to type the password every time.


Secure way of masking out sensitive information in screenshots? As a guy working in security/pentest, I regularly take screenshots of exposed passwords/sensitive information. Whenever I report these, I mask parts or complete info as in the sample given below

I often wonder, is it possible for someone to 'reverse engineer' these pics and recover the original information? If so, what should be the correct way of masking such kind of info?

I am using shutter for taking screenshots and using accompanied edit tool to add the black stroke.

EDIT:

As pointed out by some of you, my question is different from this since:

I am not asking about MS paint/black strokes. The image is just an
example to better explain the question
I have clearly asked for the correct/most secure way of producing
photographic evidence.

Yes, it can be recovered.

As long as shutter does not use layer (it almost certainly does not) and as long as the black is really all black (it must not be transparent), it is enough.

The picture that you provided uses some amount of transparency, see here:

All I had to do is use the Fill tool in MS Paint. If I used some algorithm that would take the jpg compression into account, I could probably get better results.

Solution:

Use an editor that does not make the block transparent. Make sure layers are not used. Make sure change history is not stored (to allow undo) in the file. I believe MS paint + bitmap format satisfy all the requirements. Most editors combined with bitmap (BMP) format without compression should satisfy these requirements, but I can not confirm this.

Remove the data. You can do so in many editors by selecting it and pressing delete or Ctrl + X. Then apply redaction graphics, whether black box, or anything else.

DO NOT use JPEG (jpg) or other lossy formats anywhere from capture until redaction. JPEG may leave artifacts that may convey information about the deleted pixels. This may also apply to other lossy formats, use lossless formats if possible. Using any format after the image is redacted is fine.

As lossless formats may also retain some information, if they are not completely re-encoded after the edit, it is recommended to either only use pure bitmap format with no compression before redacting, or to change the format after redacting.

Double check:

You can double check no compression is used in BMP format by checking the file size. The size should be larger than color_depth / 8 * width * height (resolution in pixels, color depth usually 24). Note that this check will not reveal transparency and artifacts caused by lossy compression, so make extra sure you did not use lossy format at any point.

It may also be useful to post a specific question about your proposed setup here, so you can see additional opinions and recommendations. It is hard to give definitive answer, that would work in general for all platforms, formats and editors as they all have their specific caveats.


Can network admin detect port scan on a certain host? Say, my laptop is connected to a large WiFi network with many other users. If a port scan is run on my IP by somebody else on the same WiFi network, is it possible for the network admin (or anybody) to find out, or is it only my system that can detect this?

Any one with sufficient access to the network traffic can detect this, All you need is the capability to see all traffic and how its routed (to what TCP Port and IP).

The trick is to detect the 'unusual' signature of a port scan over the rest of the traffic (like sequential trying of random ports / specific well-known ports). This can be quite tricky for automation but is usually easy to do for a human.

As to specific who can detect it that is :

everyone in between you and the attacker that can see enough network traffic.
the attacker. obviously
you (the target) since your machine receives all these requests.

As a rule of thumb: Nothing within a network is anonymous / untraceable from within the confines of that network.


What is blocking me from accessing my locked Android's file system on my unencrypted phone? My Android is unencrypted and running Android 5.1. However when I restart the phone I cannot access the file system using Android File Transfer using a trusted computer unless I enter my password.

Given that my device has not enabled whole disk encryption what is stopping me from gaining access?
Do I still need to encrypt my files?
If the files are accessible how can I access them in order to test if the encryption I intend to add works?

Note: Encryption is not on by default on Android phones so I'm taking these measures to encrypt at the file level to secure the data for users who have not enabled whole disk encryption on their phone.

When you attempt to access files through a USB port against a running phone, that access is mediated by the running OS.  That OS has rules about what can grant access - and one of those rules is (generally) that the phone must be unlocked (and often specifically configured to allow USB file access).

This is because the phone OS is designed to protect the legitimate owner and their files - they have no reason to allow the OS to provide the files unless you can demonstrate you are legitimate (by unlocking / configuring).

Encryption is intended for a separate purpose - for preventing you from doing an end-run around the OS.  The data is, after all, stored on NVRAM (or equivalent - not an expert on phone storage) - for someone willing to directly access this memory using hardware (or for someone able to root the phone without clearing the storage), the OS protection is nullified (since it either isn't running (hardware attack) or is under control of the attacker (root))

File System Encryption prevents access to the data even if the user can access the raw bytes on the file system - without the key, that access becomes useless.


Motion sensors - detectable when using mobile network? Motion sensors are a good way to be informed of any activity at our house. Many modern motion sensors have the possibility to notify the user using an internet connection using Wi-Fi or even mobile data network by inserting a SIM card (where no wifi exists).

This is a good way to secure our home. But from another perspective, an abuser can use this device to get information about someone also.

Let's say we have a scenario in which we suspect that someone put such device to our home. We have the option to find that device physically by searching for it around the house, which can take hours of searching.

Is such a device possible to locate using any method or device which could locate it?

If the device uses cellular

You can use devices like this or these to find the device.

If the device uses WiFi

Some enterprise AP configuration tools can locate a device in a building, so this could be used to detect the bug

If the device uses other RF

You would need to use a receiver or SDR to detect and move towards the strongest signal. These devices can be used to detect the other options too, but the more specialised devices may get better results.

Issues

These answers rely on the device transmitting when you are looking for it. If it only pulses quickly, or batches up data together, then it will be harder to find.