How am I ever going to be able to "vet" 120,000+ lines of Composer PHP code not written by me? I depend on PHP CLI for all kinds of personal and (hopefully, soon) professional/mission-critical "business logic". (This could be any other language and the exact same problem would still stand; I'm just stating what I personally use for the sake of context.)

To the furthest possible extent, I always code everything on my own. Only when absolutely necessary do I, reluctantly, resort to using a third-party library. For some things, this is simply necessary. For example, e-mail parsing and other very complicated stuff like that.

For managing such third-party libraries, I use PHP Composer. It's a library manager for PHP. It is able to download libraries, and their dependencies, and update them with commands similar to other "package managers". In a practical sense, this is much nicer than manually keeping track of this and manually downloading ZIP files and unpacking them and dealing with all sorts of problems. It at least saves a lot of practical headaches.

However, the most fundamental security problem still persists: I have no idea what this "installed" code contains, nor do I know what is added/changed with every update. One of the libraries' authors could have easily been compromised one day when my Composer fetches updates, causing my PHP CLI scripts to suddenly send my Bitcoin wallet.dat to some remote server, install a RAT/trojan on my machine, or even worse. In fact, it could already have happened, and I would be none the wiser. I simply have no idea. I logically cannot have any idea.

My own code base is about 15,000 lines in total. It takes me over a year to painstakingly go through that code base. And that's code that I have written and which I know intimately...

My "Composer" directory tree currently is at over 120,000 lines of code. And that's for the minimal number of crucial PHP libraries that I need. I use very few, but they have various dependencies and tend to overall be very bloated/inflated compared to my own code.

How am I ever supposed to "vet" all this?! It's simply not going to happen. I "zone out" very shortly after even attempting. I don't even know how I'm going to make it through another "vet round" of my own code -- let alone this 10x larger one, coded by other people.

When people say that it's a "must" to "vet third-party code", what exactly do they mean? I also agree that it's a "must", but then there's the pesky reality. I will simply never have the time and energy to do this. Also, I obviously don't have the money to pay somebody else to do it.

I spent countless hours trying to learn about Docker and see if there were some way I could "encapsulate" these untrusted third-party libraries somehow, but it's a losing battle. I found it utterly impossible to get that going, or have any of my many questions in regards to it answered. I don't even think it's possible in the way that I imagine it.

You can't vet individual lines of code. You'll just die trying to do that.

At some point, you have to trust someone else. In 1984, Ken Thompson, one of the co-inventors of much of Unix, wrote a short article on the limitations of trusts. At some point, you do have to trust other people, you have to trust that whoever wrote your text editor isn't automatically hiding some Trojan code that the PHP interpreter will execute into some Bitcoin stealing malware.

You have to do a cost-benefit analysis to prioritize what you vet.

For the most part, you should do the best you could to vet the authors of the code, the project's internal security practices, and how the code reaches you. Actually reviewing the code is expensive and hard, so they should be reserved for the parts that you consider most important for your project.

Is the library a popular library that's used by lots of people with a respectable company or a well-known project lead behind it? Does the project have proper project management processes in place? Does the library have a good past history of security issues, and how did they handle them? Does it have tests to cover all the behaviors it needs to handle? Does it pass its own tests? Then the risk of the library being compromised without anyone noticing is reduced.

Take a few sample files for deeper vetting. Do you see anything concerning there? If the few files you took have major issues, you probably can infer that the rest of the codebase has similar problems; if they look good, then it raises your confidence that the rest of the codebase is similarly well written. Note that in very large codebases, there will be different areas of the code with varying levels of code quality.

Does your package manager repository check for the package signature? Is there a pre-vetting system required to register a package into the repository or is it an open registration repository? Do you receive the library in the form of source code or as a precompiled binary? These affect how much you can trust the library, the risk factors, and how you can further improve trust.

You also have to consider the application and the execution environment that the application will run on. Is this for a national security code? Is this code handling part of an eCommerce or banking handling credit card numbers? Is this code running as a superuser? Is this code life/safety-critical? Do you have compensating controls to isolate and run code with different privileges (e.g. containers, VMs, user permissions)? Is this code for a weekend side project? How you answer those questions should let you define a budget on how much you can invest in vetting code, and therefore how to prioritize which libraries need vetting, at what level, and which ones are fine with lower trust.


How to enumerate PGP keyring e-mails related to a specific ccTLD I would like to enumerate all possible PGP e-mails related to North Korean top-level domain .kp
Few years ago I would have used theHarvester but I recently discovered that this useful tool doesn't support PGP keyring search any more :(
If I use the official MIT PGP Key Server search and I try to search for .it:

the search starts but it then ends up in an error:

Which makes sense, there are maybe too many entries.
But for North Korea it might maybe be possible because they had only 1024 routable IP addresses, the list of website is not that long, and I'm quite sure some might have used PGP. Or maybe not?
Anyway, do you know any tool alternative to theHarvester to dig into e-mails by top-level domain?

A keyserver can't let users query something that will return arbitrarily large amounts of data. If this were possible it would make them extremely vulnerable to Denial of Service.
If you want to find all keys containing emails in a certain domain, you'll have to download the SKS keydumps (for example, from here) and trawl through them.
The keydumps total around 11 GB, so this is no small project.


How does the same origin policy apply to two different windows of a browser? One particular scenario I am interested in: If I a open one chrome browser window with --disable-web-security option, then SOP is disabled. Will this window be able to access data on websites opened in other window?

The --disable-web-security option applies to the entire Chrome application, not to a particular window.

If you enable this option when starting Chrome, then any site you visit can access data from sites you visited earlier.

For example if I was looking at my Online Banking account, and then closed that tab and navigated to a malicious site, that malicious site would then have the ability to access my Online Banking account, transfer funds, etc. Note that this risk survives even after the tab is closed, and continues so long as you are still 'signed in' to the Online Banking account.

Also, it can survive restarts. So if

I sign into Gmail.
I restart the browser, now enabling --disable-web-security
I visit a malicious site.

Chances are, you are still signed into Gmail, because it often will not sign you out as an Online Banking site would. In that case the malicious site can access your Google account and control all services, as long as you have not yet Signed Out.

  Will this window be able to access data on websites opened in other window??

--disable-web-security applies to the whole browser, not just one window.
The 'other window' need not be open to for 'this window' to access data from that site.

So if you must use --disable-web-security, you should first Clear Private Data, especially Cookies & Saved Passwords. Then get your testing done, and then use the proper Exit feature at your earliest opportunity to get back into a safe browsing environment. Make sure your browser is fully Exited (not even a tray icon) before you assume you are back into a safe browsing session.


Steps for building a good File Encryption on Android I want to be able to build a good file encryption on android for a cloud app.
The files used are mainly pdfs, pngs, docs, audio, video, etc (all sorts of files).
Edit:
My constrains are that it should be fast enough, so the users of the cloud app don't wait forever to upload/download. Also, the encryption must be secure, thus, if the device gets stolen nobody can get a hand on the data.
I know a bit about encryption, I read about AES (understood the math behind, not so complicated), but I am not sure how to properly do a good file encryption, from the usability point of view.
I do hope you get my problem, if not, please tell what is not clear.

Therefore, my question is:

Which steps should I follow (things to read, etc) in order to have a proper encryption which is also fast enough?

Example: I saw that Spotify is doing a sort of encryption of their files which seems to be fast enough.

Known fact: All I know so far, is that the storage of the key used for encryption is safe enough, so that is not a problem.

According to the comments, I would suggest doing simple AES on the server side. Eg, when a customer wants to open a document, he request decryption, and then the document are sent to server, decrypted and then sent back. The decrypted document are then never stored on flash or memory, only in RAM. Once the user are inactive a certain period or switches to a Another app, delete the document from RAM.

Note that the Communication channel need to be secure (eg protected by VPN or similiar means), since cleartext data will be transmitted over the wire.

However, if you could afford (eg theres enough disk space so you does not need to leech on the device's disk space), you could instead store Everything at customers disk or your disk, unencrypted, and use a simple server-sided authentication process to make sure stolen devices cannot request documents off the server. That would mean no documents ever are stored on the device.

Then you simply send the documents for immediate vieweing in the app.


What drives FIPS140-2 compliance? I guess this questions is directed at Product Managers, but I would be very interested to learn what people in the know think.

What are the most common factors that drives FIPS140-2 compliance for hardware modules? Why do you want/need to implement FIPS140 compliant security in your products?

Here are a few reasons I can think of:

Market Demand:
Customer (government) demand
Customer (individuals) demand
Customer (corporate) demand
Market acceptance/expectancy
Marketing
Corporate IT Security Policy/Security Posture; (e.g. less likely to get hacked if using FIPS140 compliant modules)
To mitigate against present/future attacks; co mitigate past security holes
Bragging rights; just because you can; to prove you can do it
Because it is easy to bring into compliance
Because it is inexpensive to bring into compliance

It is driven entirely by US Government demand.

FIPS stands for Federal Information Processing Standards which are US Government standards.

No one else cares about it except them.  Others claim to care about it only because it is widely adopted by the "big boys".  Look around at FIPS 140-2 requirements in other nations.  There are virtually none except for those who wish to sell electronics or software to the North American market.  Other countries do levy cryptographic requirements for localized procurement and the extent of those requirements may borrow heavily from prior work putting FIPS together, but they are different and largely non-interchangeable programs.


Is packet sniffing a risk on VPS? If so, what precautions should I ask my host if they take? I'm wondering if packet sniffing is a risk on my VPS.

Is it always a risk or are their precautions a host vendor can take? e.g. a switched network etc.

If I wanted to ask my host what precautions they take, what questions should I ask?

EDIT

Thanks for everyone's feedback.

I suppose what I was wondering if there are any specific packet sniffing risks associated with a typical VPS setup. But I suppose 'typical' makes this question to general, so instead what would help a VPS setup less prone to packet sniffing (other than the things I can do myself e.g. https / ssh) - perhaps the way the VPS network is configured? The host is using kvm.

Every VPS is different therefore No one can possibly answer this question except for you.

Run tcpdump or Wireshark on your system.  Look for non-broadcast (like TCP) traffic that contains a MAC address that isn't yours. SIMPLE!


Is it possible to extract and re-use the API of a mobile banking app I am aware of the basic techniques used to reverse engineer and extract meaningful information in order to re-use an applications API.

I am not fully adept with the advanced techniques however.

Is it reasonably possible to extract the API information in order to re-use it. In general I ask as I am aware all apps are created differently.

I'm thinking the biggest problem would be if there is any advanced obfuscation of code.

That said, is it possible that reading code isn't required. If I could just MITM the SSL connection then I could figure out the API. I would need to change the SSL cert in app if exists. This would change the APK signature though.

Yes, MITM is the way to go.

Reverse-Engineer possible Certificate-Pinning

You will first have to change the certificate. I assume any reasonable banking app has it pinned.
Create your own certificate and change it. Remove the signature from the APK file so that Android will install it.

Redirect to your MITM Server

You will have to redirect requests from the app to your MITM server. You can change the hostname in the app but the easier way is to sniff what servers the app is connecting to and set up your own DNS (can be done with any non-cheap home router or Windows tools).

Root CA

If your certificate is not signed by a trusted Android root CA you will have to add the signing certificate to your phones root CAs. I dont know how to do this in particular but Google will help.

Set up the MITM server

I'd suggest that your MITM server keeps track of all the traffic, appears to be the simplest way to me. But you could also use a network sniffer (e.g. Wireshark) und decipher the traffic with your own certificate.

If Android installs the App (enable installing from untrusted sources) everthing should work.


Doesn't this make 2FA irrelevant and possibly dangerous? Article.  One sentence summary: they use a smartphone as a listening device to compromise computer security.

I know our bank doesn't allow us to take cell phones in at all, even if they're completely off.  If a cell phone can be used to hack a computer like in this article, doesn't that make 2 factor authentication dangerous?

No. The attack-vector described in the article requires the PC to be compromised in some way and requires someone to intercept the radio-traffic sent from your PC.

So first of all: unless your working on some really complex stuff that would be worth spying on it with great effort, you're pretty unlikely to be attacked this way anyways. 2FA via smartphone usually requires you to be online, while the described attack is used for air-gapped machines. Not exactly the setup an every-day user would use. In addition this attack isn't exactly powerful due to the low amount of data that can be transferred, so in general: no sane-minded attacker will ever use this to compromise your machine.

How does the attack even work

The attack, which the developers named GSMem, works by using data-buses as transmitters. There have previously been comparable attacks using video-cards as radio-transmitters, etc.. This kind of attack can be used against air-gapped machines, which obviously can't transfer data via WiFi or any other network. While the video-card type of attack worked via radio-signals and could be received by a smartphones FM-receiver, GSMem uses a different wavelength that can be received via normal base-band chips that are deployed in older phones as well. By using multichannel-buses, the amplitude of the signal can be increased to the point where the receiver can actually read the data. But there are high limitations on this approach even with specific hardware, the connection only had a range of 30m and was restricted to ~1000bit/s. This would be sufficient to steal cryptographic keys, or comparable small data, but won't do for anything larger.

This attack needs two things to work: a compromised machine that you want to intrude. Here the GSMem-malware is deployed to make the PC emit a strong enough signal. And of course a receiver, which would be a phone with a modified baseband-firmware that would function as a receiver. The computer sends the signal to the phone and the phone hands it on to where-ever it is configured to hand it on.

Does that make 2FA useless?

Probably. If someone has access to your PC, he can always just wait for you to authenticate and eavesdrop or modify your actions as long as you're logged in. There's no difference between this and the PC being compromised with any other malware from this POV. But 2FA might keep the bad guy from accessing anything that needs to be authenticated as long as you don't authenticate it on your own.

Does that make 2FA dangerous?

Why should it? As long as your smartphone isn't compromised as well, it at least keeps the bad guy from using services protected by 2FA as long as you don't log in. So in fact it would be an additional layer of security, that would help at least a bit. And if the your smartphone is compromised as well, it just renders 2FA useless. But 2FA won't introduce any new attack-vectors here.

TL;DR

The described attack is - cooked down - spyware that works for air-gapped machines. It won't interfere with 2FA in any different way than usual spyware. In addition it's highly unlikely, that this attack will be used against the average user, due to it's complexity, high limitations and specific intended application.


How can HTTP Parameter Pollution be exploited? In HTTP Parameter Pollution, I know theory how it work; you inject multiple HTTP parameters with the same name to trigger bugs in the server, but I can't understand how one can exploit this.

When I send some request using this technique and for example I know that the server is using last occurrence of parameter, how this technique can be useful, because no matter what, server this uses last occurrence, so it doesn't matter what were other occurrences right? Or when server does concatenate parameters with same name, some server script will get concatenated result.

HTTP parameter pollution is an attack which exploits a different interpretation between parameter validation and parameter use. For example in case of foo=innocent&foo=some-sql-injection the validation inside a Web Application Firewall or inside the web application itself might extract the first value innocent as the parameter value and do some security checks on it. The actual use of this parameter inside the web application might though take the second value some-sql-injection instead.

The attack pattern of exploiting interpretation differences is not unique to HTTP parameter pollution. It can be for example done with HTTP header fields (which Content-length header is the correct when two are given?) as in HTTP cache poisoning or HTTP smuggling or with XML signatures in XML signature wrapping attacks.


concrete real-life examples where grsecurity prevented an exploit From theoretical point of view, grsecurity kernel patch looks like a great hardening tool. Most importantly, PaX seems like a good idea.

Do these theoretical advantages have indeed practical effect in preventing malware attack/exploits/rootkits ?

There were several critical security problems recently (Shellshock, Heartbleed, Turla, ... to name just a few)

Can somebody please point to a concrete exploit which grsecurity would have prevented ?

I'm root@anapnea.net, and grsec is the reason I can sleep at night.

As an example of an exploit blocked by grsec, you can look at almost any of the recent kernel vulnerabilities. Stock exploits simply don't work against a grsec kernel.

As an example of a vulnerability blocked by grsec, and in particular UDEREF, you have the recent x86_32 local root.

Grsec (minus rbac) won't do anything against "logic" problems (shellshock, hearthbleed, the LD_* problems) but, together with an hardened toolchain, it makes some classes of bugs much more difficult to exploit.

What it does help against are kernel bugs.

Then you have RBAC. Rbac is a MAC (mandatory access control) system, like SELinux or AppArmor, which offers further protection, even from root.

Plus, grsec will allow you to enable some nifty restrictions which are really useful if you can bear them. The main ones are:

socket restrictions (no listen/no connect/nothing)
/proc visibility restriction (per user)
stronger chroots
restrict runnable programs to only root-approved ones

The point is: grsec will make your system safer, but bugs will be found even on the safest system.


Why does broad based malware use XOR obfuscation? As per title...

The malware can easily be de-obfuscated if it was obfuscated using common and known XOR techniques. Why don't malware authors use any other method that make it much harder to automatically de-obfuscate and identify malicious embedded code?

My thoughts:
Malware authors actually do use other encryption methods,
Simplicity of implementing the algorithm,
Size limitations on the code,
Pre-packed options (metasploit) offer these techniques,
There is no need - XOR techniques bypass most of the detection mechanisms already

Even so, malware is already incredibly complex, why isn't obfuscation always complex too?

Well there are more complex obfuscation techniques. For instance metamorphic and polymorphic malware.

There is an article on searchsecurity by Margareth Rose detailing how this works:

  Metamorphic and polymorphic malware are two categories of malicious
  software programs (malware) that have the ability to change their code
  as they propagate.

  Metamorphic malware is rewritten with each iteration so that each
  succeeding version of the code is different from the preceding one.
  The code changes makes it difficult for signature-based antivirus
  software programs to recognize that different iterations are the same
  malicious program.

  In spite of the permanent changes to code, each iteration of
  metamorphic malware functions the same way. The longer the malware
  stays in a computer, the more iterations it produces and the more
  sophisticated the iterations are, making it increasingly hard for
  antivirus applications to detect, quarantine and disinfect.

  Polymorphic malware also makes changes to code to avoid detection. It
  has two parts, but one part remains the same with each iteration,
  which makes the malware a little easier to identify.

  For example, a polymorphic virus might have a virus decryption routine
  (VDR) and an encrypted virus program body (EVB). When an infected
  application launches, the VDR decrypts the encrypted virus body back
  to its original form so the virus can perform its intended function.
  Once executed, the virus is re-encrypted and added to another
  vulnerable host application. Because the virus body is not altered, it
  provides a kind of complex signature that can be detected by
  sophisticated antivirus programs.

  In another example, a new key might be randomly generated with each
  copy to change the appearance of the encrypted virus body -- but the
  virus decryption routine woud remain constant. In either scenario, it
  is the static part of the code that makes it possible for an
  anti-virus program to identify the presence of malware.

  Metamorphic malware is considered to be more difficult to write than
  polymorphic malware. The author may use may use multiple
  transformation techniques, including register renaming, code
  permutation, code expansion, code shrinking and garbage code
  insertion. Consequently, advanced techniques such as generic
  decryption scanning, negative heuristic analysis, emulation and access
  to virtualization technologies are required for detection.

Now if you are trying to keep things small, XOR is an excellent way of writing your virus. XOR is an instruction which your processor understands, so this means that to implement a XOR encoder you will need only a few lines of assembly code (if you are writing the virus in assembly) opposed to vast amounts of code when you want to implement either of the above mentioned techniques.


How do virtual machines help security? Can using a VM help to protect my passwords or other data?

I'm asking this because I'm using a PC that others have access to, so there is possibly of some malware. I don't want to login to my web accounts, assuming malware can "steal" passwords, packets, etc. So, I wonder whether using a VM for accessing web accounts makes more sense.

An additional question: does a VM help web anonymity? I'm guessing the server will still see the host's address, right?

I don't know much about how stuff like TCP/IP works, so please inform me if my assumptions are wrong. I think VM's are usually used the other way, i.e. doing the "dangerous" stuff in VM, to keep the host clean. But in my case, I do not have an option not to share the host, so I wonder what can I do to make it safer.

The VM approach helps in that "what happens in the VM stays in the VM".

You would do your things in the VM and then reinstantiate it every time (there was, perhaps there still is, a Windows helper app called SteadyState that allowed a sort-of-VM to continuously be recreated while still being kept updated).

The essential thing is that the VM have no access except to the Web. Your personal documents, etc., would stay outside the VM. If you need to download a file, you would download it in a "shared folder" and then run a virus scan from the guest operating system once the VM is down.

This way, in theory, no virus or malware can infect the guest operating system unbeknownst to the user.

Your sequence of operation would be:

keep the "pristine" virtual machine somewhere (say C:/VM/Pristine).
copy the VM into C:/VM/Dirty
fire up Dirty and use it to your heart's content
copy any files you need from Dirty to C:/VM/SuspiciousFiles
power down Dirty
delete it. Any virus, malware, etc. is thereby destroyed.
virus-scan the contents of SuspiciousFiles

Occasionally, you would fire up Pristine, run Windows Update or zypper update or apt-get update or whatever, then power it down again, ready to be cloned into a new Dirty.

As for anonymity, your IP address would be still visible, but you could configure your VM to be a "stock" machine with all default options, in no way different from millions of others. Even frameworks such as Panopticlick would be hard pressed to reliably recognize it.

UPDATE: reverse protection

A similar level of isolation, but lower, exists between the outer machine and the VM (i.e. you want to use a clean VM on a possibly dirty machine). What happens is that a virus or malware in the outer machine has full access to the files constituting the virtual machine, but normally and as far as I know, it would be too complex for a malware to try to influence the virtual machine through the virtualization layer. Even so, protection against the hosting machine is less reliable, and the VM's very existence may be jeopardized (for example by some ransomware such as Cryptolocker, deciding to encrypt all large files in the hypothesis that they must be valuable).

In such cases it is best to employ a different approach, that of a portable machine on a bootable USB device. There are many such; this is one. The idea is that when you need to perform some sensitive operation, you reboot the machine and boot it from the USB device. The machine will restart (should restart, if the USB application is well built) with no connection to the outside except basic networking, and no knowledge it even has a hard disk. Everything you do will only exist on the USB which will then be put back in your pocket when you finish. There's still a remote risk of so-called bluepilling rootkits having rogue access to the USB while it's plugged in, but as far as I know it's (for now at least) more theoretical than realistic.


Why verify a file / firmware downloaded online against a checksum? When ever there is a file / firmware to download online and they provide a checksum to check the file against, i always confirm the check sum of the downloaded file matches the checksum posted online.

But it has often crossed my mind, if a malicious 3rd party has FTP access and are able to swap out the file / firmware with a malicious build, they would surely have the technical know how and access to go into the html / php etc webpage file and update the check sum to match their now malicious build, thus rendering the check sum worthless.

Have i missed the point here ?

Checksums are provided to detect corruption during file transfer, not to detect man in the middle attacks.


How to extract the hash from SMB I have a running PC which I need to test against.It is running Windows with SMB enabled. When capturing using Wireshark while connecting with an incorrect password I see packets being sent to / from the IP address. Could the hash be extracted from it?

I am running Linux

Smb uses two main authentication schemes:

ntlm - a challenge response protocol sending a salted hash
kerberos - a centralized authentication protocol using a salted hash as the encryption mechanism.

In both cases you cant extract a plain nt hash from the traffic, but you can bruteforce it as the challenge is supplied.


Preventing an unauthorized mobile application from calling REST back end services In my scenario, I have an Android mobile application which calls my back end REST services. I want prevent other application to call my back end REST services.
If I use certificate and shared key as well as obfuscating my application's code, attackers can beautifier my source code and do reverse engineering.

This is near impossible to achieve (short of white-listing IP addresses). Given someone has enough time and resources, they will almost certainly be able to send calls to your API outside of your client. In general, you want to assume attackers know everything about your API and design your API to be resilient to attacks (Ex. good authentication practices with correct limiting, etc.), otherwise, you would likely be fighting a losing battle.

A great example I like to bring up is the Pokemon Go API (link here). Hackers were quickly able to create 3rd party clients that would send erroneous location data and allow users to catch Pokemon from all over the world. Despite API updates, hackers continuously re-reversed engineered the necessary steps to send valid API calls.


Private CA and client server authentication I have a private application - that has one server and many clients and I want to use two way authentication(AKA client auth)
I use Tomcat server and java keytool to create TLS certificates.

If I create one certificate authority, and create on certificate (signed by the CA) for the server, and pre-install it for all clients - then they can authenticate server (although I didn't really had to sign it with the CA, but anyway)/

If I create a client certificate, sign it with the CA, and then install it on the client:

Does the server need to hold the client certificate(and the CA) or just the CA in the truststore?
And by that I mean - Is there any place that I need to "Remember" the client certificate and check against an existing file(that will of course - match one to one with the certificate the client sent)
Or is it enough to just check the signature of the CA on the certificate, causing that just one copy of the client certificate exists in my system, and cryptographically - it is secured to assume that if the signature of the CA exists(and validates) on the certificate a client sent me - it is the client

Your Option 2 is sufficient and cryptographically secure. If the CA's signature on the client's certificate validates (and the CA's signing keypair hasn't been revoked due to compromise) then yes, you can assume that the certificate is authentic - ie it was issued by your CA and hasn't been tampered with. Remember that a signature is an encrypted hash of the whole certificate, so if even a single character has been changed, then the signature will fail to validate.

However, that does not tell you anything about who sent it to you. Certificates are public, everyone could have a copy of everyone else's, so Alice could send you Bob's certificate and claim to be Bob. In order to authenticate the client, they need to prove that they have the corresponding private key. Remember that certificates only contain public keys, the matching private keys are never made into certificates, and are stored separately. This can be done by having the client send a piece of data that they signed along with the certificate.

About your Option 1: in a full public-key infrastructure in which the end-users can encrypt for each other and verify each other's signatures, everybody needs access to everybody else's public-key certificates. This is usually done by having the CA keep a copy of everybody's certificates and display them in a public place like on an HTTP site, or in a public LDAP directory.


Statistical Anomaly Detection with machine learning applications to IDS as the topic says I am interested in learning more about anomaly detection in particular I am asking for good starting points and references both from a theoretical point of view and applied one. For the second one (applied) I am interested in tools to use and good "traffic packets sets" which I can use to start digging this area of information security.
You can assume I really don't know much about IDS and ML but I do have a good enough background in statistics and math.

Thanks in advance.

This Wikipedia article is a bit of a stub, but has a couple of references to start with.

Theoretical issues with ML based IDSs are that they can be fooled in a number of ways, for example by forcing them to mis-train as in this article. Think of this attack as what spammers do to confuse Bayesian filters.

A huge practical issue is that this approach is extremely slow for modern network speeds. The amount of processing that has to happen for every packet is probably going to grind such an IDS to a halt on a few Mbps.

Traffic packet sets are hard to find. There is this very old one http://nsl.cs.unb.ca/NSL-KDD/ that used to be popular. There is also http://www.netresec.com/?page=PcapFiles.


Gentoo Hardened vs other distros I was wondering if the hardened profile from Gentoo was really more secure than any other distro (like Debian, RHEL, Arch ...). For those who don't know, Gentoo hardened allows a system to be built system-wide with specific hardening GCC options (pie, ssp, relro, ...) and other few things (grsec/selinux ...).

For example, I know Arch Linux does not build all binaries with those GCC hardening flags, so does it imply some sort of concern about security?

I know OpenVPN is built without PIE and partial relro. Does this mean that if an exploit is found against OpenVPN, an Arch installation may be less secure than a Gentoo one?

TL;DR: is it a real advantage using Gentoo Hardened over any other distro in terms of security of binaries?

Its all in the source! Gentoo hardened is an security driven distro the hardened profile really packs a great deal into making it really secure.
But is it worth the compile? A big question among the linux forums.
Lets look at Gentoo hardened profile in terms of security:
while it adds some security
it's so little that it's not worth it in most cases. It provides more security
on a binary distro because everyone has the same binaries and an attacker
don't need to guess where a specific piece of code may get loaded but by
running a source distro your address space is already pretty unique. The only
case where it provides some security is when an attacker is trying to guess an
address for an exploit, making the wrong guess will likely crash the process
and it will be reloaded on a new address. Do you have valuable enough data for
an attacker to go through that hassle in order to get it? If you do then you
should use a hardened profile, but physical security and disk encryption is
more important because if it's worth that much it'll be easier to just rob
you.
Be aware that there's no hardened desktop profile so that alone will make it
somewhat harder if plan to use it on a desktop.
Another reason is if you want to use something like SELinux (which doesn't
require a hardened profile) that gives you very fine grained control about
access control but it's also very restrictive. I think it's only worth it for
large networks with many users and different levels of access to sensitive
data.
I needed some of SELinux features but settled for using AppArmor in an unusual
way to accomplish them because SELinux is too much trouble. All AppArmor
really does is provide process isolation or sandboxing. If an attacker gains
access through an exploint he will only be able to access the files that the
exploited service has access to. I use it with a catch all profile that
prevents execution from all world writeable and home directories, and access
to ssh/pgp keys, keyrings, etc. This works nice for servers and desktops and
is not too restrictive. And if I need to execute code from my home dir for
development I can launch an unrestricted shell via sudo. I can leave my laptop
unlocked with the wallet open (I use the kwallet pam module) and it will be
really hard for you to get anything like ssh keys or passwords (I also have
patches for kwallet so it requires a password to show saved passwords), but
the programs that need them have access to them.
But what makes it hardened? lets look at some of those items as well:

PaX is a kernel patch that protects us from stack and heap overflows.
PaX does this by using ASLR (address space layout randomization),
which uses random memory locations in memory. Each shellcode must use
an address to jump to embedded in it in order to gain code execution
and, because the address of the buffer in memory is randomized, this
is much harder to achieve. PaX adds an additional layer of protection
by keeping the data used by the program in a non-executable memory
region, which means an attacker won’t be able to execute the code it
managed to write into memory. In order to use PaX, we have to use a
PaX-enabled kernel, such as hardened-sources.
PIE/PIC (position-independent code): Normally, an executable has a
fixed base address where they are loaded. This is also the address
that is added to the RVAs in order to calculate the address of the
functions inside the executable. If the executable is compiled with
PIE support, it can be loaded anywhere in memory, while it must be
loaded at a fixed address if compiled with no PIE support. The PIE
needs to be enabled if we want to use PaX to take advantage of ASLR.
RELRO (relocation read-only): When we run the executable, the loaded
program needs to write into some sections that don’t need to be
marked as writable after the application was started. Such sections
are .ctors, .dtors, .jcr, .dynamic, and .got [4]. If we mark those
sections as read-only, an attacker won’t be able to use certain
attacks that might be used when trying to gain code execution, such
as overwriting entries in a GOT table.
SSP (stack-smashing protector) is used in user-mode; it protects
against stack overflows by placing a canary on the stack. When an
attacker wants to overflow the return EIP address on the stack, he
must also overflow the randomly chosen canary. When that happens, the
system can detect that the canary has been overwritten, in which case
the application is terminated, thus not allowing an attacker to jump
to an arbitrary location in memory and execute code from there.
RBAC (role-based access control): Note that RBAC is not the same as
RSBAC, which we’ll present later on. The RBAC is an access control
that can be used by SELinux, grsecurity, etc. By default, the creator
of a file has total control over the file, while the RBAC forces the
root user to have control of the file, regardless of who created it.
Therefore all users on the system must follow the RBAC rules set by
administrator of the system.

Additionally, we can also use the following access control systems, which are used to control access between processes and objects. Normally, we have to choose one of the systems outlined below, because only one of the access control systems can be used at a time. Access control systems include the following:

SELinux (security-enhanced Linux)
AppArmor (application armor)
Grsecurity, which contains various patches that can be applied to the
kernel to increase the security of a whole system. If we would like
to enable grsecurity in the kernel, we must use a grsecurity-enabled
kernel, which is hardened-sources.
RSBAC (rule set-based access control): We must use rsbac-sources
kernel to build a kernel with rsbac support.

It all comes down to the big question as stated earlier? Worth the Compile? Really comes down to how or what you are securing and is the effort worth while? Or will you be able to truly secure what you do not want prying eyes to see?


Is there a diagram based language for modelling security? A lot of us will be familiar with UML for software engineering. Its uses are plentiful.

Is there such a thing as a set of conventionally used diagrams for modelling any security scenario (speaking as a complete beginner to the field of security).

For example, if I wanted to sketch out all the different ways a client's password database could be breached, given that she has situations like the file being stored in the cloud, 2FA with a physical key, application on her phone, etc.

The idea would be to have a clear, pictographical overview of the situation, and ideally be able to easily see flaws or how to best optimise the security given the person's lifestyle.

I have not (yet) found such (markup) language for modelling.

This was the closest I found, but it's not really suitable:
Secure Tropos

"Secure Tropos is a security-aware software systems development methodology"


Disguise System as VM Malware sometimes uses anti-vm and anti-sandbox behavior to act like harmless software.

Wouldn't it be a good prevention strategy to disguise the host machine as a VM so that potential malware would never break out?

I am not talking about "faking" virtualization which I think isn't possible, I'm talking about having a VM-typical MAC address, running an empty process in the background that doesn't do anything but has the name of a VM-typical process (like vboxtray.exe) or having empty driver dlls named like VM-drivers which would indicate to potential malware that the current system is a VM.

Could emulating a virtual environment stop some malware from infecting the system?
Yes. Also, creating fake VM artifacts is not the only way to stop malware from executing. You could also create fake mutex objects or certain registry keys that malware oftentimes creates in order to avoid re-infecting the same machine. And there are a lot of other ways, depending on the malware's internal workings.
Would it be a good prevention strategy?
Not really. It would be a false sense of security. Assuming the malware even has any anti-VM measures in place at all and cares to halt execution if a VM is detected (which many don't), there are still thousands of ways malware can check if it's inside a VM or not, and every malware differs. There are hundreds of millions of malware programs out there, all of which work differently, and to spend your time on all of them would not only be a massive undertaking but would still likely prove to be an unreliable defense against malware.
So no, it would not be a good prevention strategy. Your time and effort would be better spent elsewhere, such as focusing your threat model around preventing actual execution of the malware at all by using good security practices, enforcing code signing, hardening your system, reducing attack vectors, keeping everything updated, not downloading anything suspicious, etc., and you should be ok, and have no need for this prevention strategy.


User Account Guessing I'm doing some research on user account guessing, and wanted to ask a question about some "bad practices".

My thought is generally this is a trade-off between great user functionality/UX and security. For example, I could create a user webservice that checks if an e-mail exists, and when a user is attempting to register, I can (as a convenience) let them know that this account already exists (instead of having to go through the entire process only to find out at the end the account is taken).

In addition for a login form, telling the user 'this username/password is invalid' instead of 'that username does not exist' will help reduce the chances of an attack.

At the end of the day (for registration), my thought was the aforementioned webservice is a bad idea, but there needs to be some way to ensure the user knows that name is taken/available.

What are some techniques people do to secure this? I thought that rate-limiting service requests from a server perspective is a good idea. Is this common? For example I have some sort of means of blasting away at that webservice using a dictionary attack. After enough time, I could have a whole list of e-mail addresses that are valid and now I only need to guess passwords instead of both. But if the service is limited such that after N requests, the response slows down, or asks for a CAPTCHA, it would be harder to pull this off (now I need to use unique IP addresses, etc).

I wanted to see if my thinking was right here or if anyone could add on to this.

By the way, I noticed Microsoft does not do this! When you go to live.com, and you type in some made-up username, it tells you that username does not exist. When you try a valid username but bad password, it tells you that password is wrong! Isn't this a bad practice? What are some reasons they allow this? Because it's a popular service and getting a live e-mail address is pretty easy? Or maybe that coupled with lots of server security?

Rate limiting is probably the only practical way to mitigate these attacks for most public-facing applications. And you're correct in that it's a security/UX tradeoff.

Given the login form example alone, "your username or password is invalid" is a more secure (exposes less information) way to present info, but most public services have other endpoints to check for usernames anyway.

The attacker can, for example:

check for a "user profile url" if implemented in the app (eg. /users/profile/NAME)
or hit the "create a new account" endpoint and get "username/email exists" info from there.
or create a new account and hit a (for example) "send a message to user NAME" and get info from that response.
etc, etc

If you add rate limiting then these attacks will take a lot longer (to the point of being impractical — depending on your user base size and rate limiting details), but in theory it's still almost always possible.

It's possible to leak zero information about accounts, but that will make the experience for actual users much more confusing.

Imagine an existing user tries to create a new account — you have to accept all the info, show an "Account created" page, detect it's a duplicate, and email the user that "you tried to create an account, but already had one".


What is the purpose of JSON Web Token (JWS)'s issued at "iat" field? RFC 7519 specifies an optional "iat" field, indicating when a token was issued. The RFC provides a terse commentary:

  This claim can be used to determine the age of the JWT.

What is the purpose of the "iat" field? For example, why would one want to determine the age of a JWT? Were there specific purposes in mind when the spec was created?

Edit: Notably, x509 does not have an analogous field, though it may frequently be that Not Before is very close to "Issued At." Another way of phrasing this question is: Why does JWT deliberately distinguish "Issued At" vs. "Not Before?"

What is the purpose of the "iat" field? For example, why would one want to determine the age of a JWT? Were there specific purposes in mind when the spec was created?

"iat" can help the service that issued the JWT to make decisions on its own instead of depending on the issuer for a fixed expiration time (suggested in the comment by Joe above).
A JWT issuer could also set both an expiration "exp" time as well as an issued at "iat" time - the service receiving the token could decide that the expiration time is much too long, and discard it after a shorter interval which it can compute with "iat".


How dangerous are SWEET32 and LUCKY13 vulnerabilities? I wonder how vulnerable are Sweet32 and Lucky13? I mean, how often do the attackers use these vulnerabilities to gain access?

Lucky13 and Sweet32 are both attacks on SSL/TLS, i.e. these attacks can be used to intercept the encrypted connection between the client and the server.  In the case of a server that is vulnerably to Lucky13, an active attacker may be able to launch a MITM attack by exploiting this vulnerability.  The same applies to Sweet32, but even a passive attacker may be able to decrypt the encrypted data if they are able to collect enough data.  However, neither attack can be used to gain access to the server.


Firefox switches charset encoding, brings up request to install extensin. Is this a "browser virus"? I use Devuan ASCII (~= Debian Stretch) on my desktop, with Firefox and Chromium as my browsers. As I came back to my computer today after a few hours of non-use, and loaded a new browser frame, suddenly I got an overlay with a suggestion to install an extension:

The Text Encoding entry on the menu is now grayed-out, which it shouldn't be. The same thing happens on other websites, including, say, DuckDuckGo.

I have not done anything meaningful with my browser or my machine over the past several hours (before which I did not experience this behavior). And still - this seems very much like a kind of a virus, or a virus "bootstrapper" which wants to get me to install the next phase part of the virus, with more privileges.

Am I misinterpreting the situation?
Is this a known virus/malware script/whatever you call it?
How could it have gotten onto my system?
Other than shutting down, how can I mitigate my exposure?
How can I remove it?

Three "diagnostic" procedures to try (perhaps not the best thing, safety-wise) are:

Check the same sites with your other browser.
Disable all your extensions, and restart the browser normally.
Restart the browser from the command-line with firefox -ProfileManager, create a new profile, and have the browser start running with the new profile.

If you don't experience the same phenomenon when doing any of the above, it's most likely malicious code in one of the extensions you've installed.

Note that a perfectly benign extension can theoretically become malicious by virtue of an auto-update, if someone had exploited a vulnerability in the extension author's system, at Mozilla, or somewhere along the way. Of course - you might have installed an extension which seemed benign but start acting maliciously only now.


How is hibernation supported, on machines with UEFI Secure Boot? I've been learning about UEFI Secure Boot, which attempts to prevent "bootkits" by locking down the boot process so that only signed bootloaders and kernels can be loaded.

Hibernation seems like a major attack vector.  Hibernation (also known as suspend-to-disk) involves writing the entire state of memory, including all of kernel memory, to disk.  After this state is written to disk, you can shut off power.  When power is restored, you resume by reading this state from disk and writing it into memory.  What prevents an attacker from modifying the hibernated state of memory on disk, e.g., modifying kernel memory to inject malware into the kernel?  When the user resumes, it seems like this will load into memory a now-infected OS kernel.

Similarly, evil malware could presumably create a crafted hibernation file (containing a modified version of a real hibernation file, modified to insert malware into the running kernel) and then force a reboot.   Upon reboot I presume this will load the hibernated image into memory -- which would seem to violate the security goals of UEFI Secure Boot.

I suspect I've gotten myself confused or twisted up somehow.  Can anyone clear this up?  Is hibernation compatible with UEFI Secure Boot?  Does it pose any security risks to the goals of UEFI Secure Boot, and if so, how are these risks being addressed in deployed uses of UEFI Secure Boot?

Is hibernation compatible with UEFI Secure Boot?

For that you need to look at section 27 of the UEFI spec. In this specification, you will see a full explanation of the protocols in use. I'll briefly summarise them - this is a rough interpretation and might have loose edges (if you want a definitive interpreration, find one of the UEFI people :) ):

The EFI_AUTHENTICATION_INFO_PROTOCOL section defines a number of methods for getting and setting authentication information for a number of different implementations.
Signing UEFI drivers (support applications for the UEFI firmware). Since UEFI executables are based on the Windows PE image file format, you can think of this as comparable to authenticode for EFI modules.
EFI_HASH_PROTOCOL provides the programming API for hashing.
Then the juicy parts: firmware/OS key exchange. This is responsible for enrolling the platform key. EFI refers to two modes:

Setup mode
User mode
The former mode refers to a setup without an enrolled key, the latter to one with an enrolled key.

The enrolled key is the private key root from which image signatures will be trusted.
passing public keys to the OS firmware section is exactly what it sounds like - a protocol for notifying the firmware of a key it should use to talk to the operating system.
UEFI Image Validation refers to the process of checking a UEFI Image (bootloader etc) against a signature database.
The signature database is a list of valid UEFI images which have been signed with the platform key. An image must match a signature in the signature database to boot.

There is a reason for covering these EFI spec details is to point out that the UEFI Secure Boot spec covers to what extent UEFI secure boot extends - that is, it aims to achieve two things:

The UEFI Loader will only load EFI Images whose signatures are present in the signature database if UEFI secure boot is enabled and a platform key is enrolled (this is done by the platform owner).
The UEFI spec provides a means to communicate information between the operating system and the UEFI firmware to interrogate the signature database and EFI variables.

The important point here is that there is no requirement for an EFI Image, once loaded, or an OS Kernel once loaded to enforce signing on anything it subsequently loads.

For example;

Fedora will enforce signing all the way down for the kernel and respective modules - see this status update and this
Have a read of this article on the UEFI shim that came out of the Fedora EFI boot process. This shim can optionally have a UEFI variable set to disable signing checks on later parts of the boot process - like the kernel. In more detail
Ubuntu itself will not be signing its kernels:

  Therefore, we will only be requiring authentication of boot loader
  binaries.  Ubuntu will not require signed kernel images or kernel
  modules.

So, ok, let's get back to UEFI:

  Similarly, evil malware could presumably create a crafted hibernation file (containing a modified version of a real hibernation file, modified to insert malware into the running kernel) and then force a reboot. Upon reboot I presume this will load the hibernated image into memory -- which would seem to violate the security goals of UEFI Secure Boot.

Yes. This is a perfectly good attack vector for current operating systems and has its origins in the pagefile attack.

Both of these attacks work because the kernel (used to, in the case of the Windows kernel) blindly loaded data from disk back into RAM. Correct modification lets you alter drivers, install your own etc and off you go, ring 0 and one compromised system.

In a UEFI secure boot environment, the EFI firmware guarantees that your bootloader will not have been tampered with up to the point of shutting down the EFI firmware and proceeding to the operating system. That's about it. UEFI secure boot doesn't guarantee the loaded operating system is unmodified, or that the loaded operating system has loaded altered images from boot. That is up to the operating system to check and enforce.

Now some personal thoughts:

  Does it pose any security risks to the goals of UEFI Secure Boot

I suppose it does, in the sense that what you expect from UEFI Secure Boot is a sane state kernel. That's not what you're guaranteed to get, but it's what you expect.

  how are these risks being addressed in deployed uses of UEFI Secure Boot?

I think more accurately these risks are being mitigated across the board by code signing initiatives such as AuthentiCode. 64-bit Windows, for example, will not load unsigned driver code unless you perform a special dance involving F8 and boot menus. Most Microsoft images are signed and - I don't know but would guess - ntkrnlmp.exe is signed and validated by the Microsoft EFI bootloader.

As you can see, distributions such as Fedora have certainly adopted this approach as security of the system as a whole means little if you have simply got a verified bootloader.

The next stage of this process I suspect is to close attack vectors such as hibernate. For example, the page file attack was fixed by Microsoft. It is trivial to fix if you consider that you simply need a signed checksum which you can produce using, for example, one of the keys stored in the EFI firmware and verify in boot, either in the bootloader or the kernel.


Android shared library integrity protection Is there a way to somehow 'protect' a native shared library (.so) for the Android platform against binary changing? E.g. someone could overwrite a JMP instruction with a NOP after reverse engineering the application, and distribute that library to rooted devices.

Is there anything someone can do?

What I'm looking for here is ideas about implementing a series of checks (e.g. encryption, checksumming etc). Of course since the platform does not look like it offers support for this (correct me if I'm wrong) it would have to be all 'client-side'. Thus the whole thing is a bit futile, but at least will hinder reverse engineering some.

No.  There's no way to do this securely, in a way that will resist serious dedicated attack: the owner of the phone will be able to modify the copy  of the library that is stored on their phone.  Once rooted, it's their phone, after all.

You're basically asking for software that is resistant to reverse engineering.  We have extensive experience (based upon copy protection, digital rights management, and other schemes) showing that while you can make reverse engineering a little harder, you can't prevent it.

Why do you want to prevent the owner of the phone from modifying their own code?  Maybe there are other solutions to your problem.


How was my email intercepted? I've recently sent an email to a non-Protonmail user, using a code phrase. Someone on the recipient's home network was able to intercept the email and responded to me in detail as to the content of my original email. I don't know much about network security, but I would love  to know how this person managed it? All I know is that the recipient was using the WiFi at home. Is that a weak link?

*To be noted that Proton Mail is an email service that allows the user to encrypt their inbox with their own strong password and also encrypt the sent email with a new password that needs to be communicated to the receiver in order to decrypt the received email.

Email is insecure.  It is completely insecure.  Between you and the recipient it likely passed through dozens of servers and for each one it was passed as pure plain text.  Without specific details and logs it is impossible to say where a copy was lifted but regardless you should always ASSUME that anyone can read anything you put in an email.

If that concerns you then you should be encrypting the contents of your email using a tool like GPG.


Is there any benefit to requiring authorisation to access static assets in a web application? I work on a web application which has an array of static assets like button images and css files, every time a request comes in for one of these assets forms security has to check that the request is valid.

I understand that it's important to protect user's data and the source code but is there any benefit to checking the user's authorisation when they are requesting files that any signed in user would be able to access?

Static assets like JS, CSS and Images those are already in public domain need not to be authenticated requests. By making it authenticated you are putting extra load on the server and restrictions on using CDN and Cache servers which will negatively impact the performance.

If these files are dynamically generated to serve differently for each signed in user or it contains user specific details like a JS data or a profile picture then it should be requested with authentication. Also, You can remove the authencation here by making the file names for such assets hard to guess.


Is appending a salt before hash generation enough I have what I think is a securely generated salt/hashing mechanism for HMAC authentication based on SHA256 in place. However, I'm using the salt in, from what I can see, a fairly standard way of just appending to the rest of the token before the hash generation like so;

// conceptual, not actual code...
var saltedToken = token + salt;
var hash = HashGenerator.Get(saltedToken);

But is appending the salt enough to be secure or should I be doing something more with it, e.g. applying the salt as an XOR operation on the rest of the token, etc. This answer implies that putting the salt at the start may be insecure in some hashing algorithms but I'm left unsure.

Is a straight forward "append salt" approach enough? Or do I need to scramble it further before hashing?

You do not need to salt in HMAC, per my best understanding. Salting prevents rainbow tables of common hashed things - in the case of passwords, this is because there are lots of people with the same password, way too often. In an HMAC algorithm, you are hashing a message and secret key, concatenated in a specific way. This is not something there will be a rainbow table for, so salting is not necessary. The answer you see has to do with password hashing, not HMAC.

https://tools.ietf.org/html/rfc2104 is probably how you want to do this - but really, implementing this sort of thing yourself is rarely what you want to do. Use libraries for cryptographic functions, choosing ones that have been studied and reviewed by cryptographers.

More detailed questions are probably best asked in crypto.stackexchange.com, rather than here

EDIT to clarify: Note that for a straight hash, you have HASH(message) - message has only one part.  When you talk about prepending a salt, then it's HASH(CONCATENATE(hash,message)).  When an RFC2104 HMAC is in place, you have HMAC(key, message).  PBKDF2, defined in RFC2898 PKCS #5, uses HMAC with salt, message, where some additional math applies so that the output of the each HMAC iteration feeds into the next HMAC round.


What is the difference between Key Escrow and a Recovery Agent? From what I understand Key Escrow is where a copy of they key is stored in a safe place so they can be recovered later.

But in the context of PKI, a Recovery Agent can also recover private keys.

How are they different? Does a Recovery Agent use a Key Escrow?

A "Key Escrow" is used in cases where a third-party needs access to encrypted data, as defined by law (so if you get a court order to decrypt data), while a "Recovery Agent" is someone who is permitted to decrypt another user's data in case of emergency and has a key that can accomplish the decryption.

So in effect, the key escrow is someone who holds the keys for each user, while a recovery agent has a master key.

Edit: "Recovery Agent" is actually defined by Windows too, so you can access EFS-protected files of other users. See: http://searchitchannel.techtarget.com/definition/data-recovery-agent-DRA for details.


What are the security implications of allowing all incoming connections in a firewall on a typical Windows server? Say I have a Windows Server (let's go with Windows Server 2012, but I'm thinking in general). It's running IIS. We have our standard website, accessible via port 80 (HTTP) and 443 (HTTPS). And let's throw in an FTP server (ports 21 and 22) for administration. Standard user authorization on that (no anonymous FTP). Otherwise everything is in the default settings and we have whatever applications that Windows includes and runs by default.

What's the implications now if we allowed all incoming connections in the firewall?

If I understand correctly, this is almost akin to disabling the firewall -- it turns it into a blacklist, but by default there isn't any block rules, so it's like the firewall wasn't there. I can't think of anything that would be listening on a port that the firewall blocks access to, so can't see what the difference is (and trust that Windows can handle data sent to closed ports appropriately).

EDIT: Actually, I decided to try this on a server that I have access to (hopefully this wasn't a completely stupid idea). I ran an nmap port scan on that server and found the following ports are open (some I have descriptions for but don't know what they are):

21 (FTP)
80 (HTTP)
3389 (ssl/ms-wbt-server?)
5985 (Microsoft HTTPAPI httpd 2.0 (SSDP/UPnP))
8172 (Microsoft IIS httpd 8.5)
47001 (Microsoft HTTPAPI httpd 2.0 (SSDP/UPnP))
49152-49164 (RPC)

Each port that's open to connections from the internet is a possible attack vector, opening just one port ie TCP/80 gives the hordes of botnets the ability to connect and try to exploit any vulnerability with IIS to gain access or possibly break your server.

Opening every port multiplies this risk greatly, more ports and services, more chance that one of them is misconfigured or vulnerable and it only takes one vulnerability or an easily guessed password for your server to be compromised.

Is every single service running on that server patched and hardened? Does every service have protection against brute force attacks? Do you know what every open port is doing? If the answer to any of those questions is no, then the implication is that your server will eventually be hacked. The internet is not a forgiving place.


Destroying sensitive partition I've got multiple partitions on a computer, one for basic, day to day user stuff, and one for pentesting and security. The pentesting partition is encrypted. Could anyone give me, and everyone else, a way to destroy one partition, completely and totally, without wiping the other partition, and making it quick, seamless, and unnoticeable as possible. Think of an enemy operative has a hold of my computer, and I need sensitive information and tools to be erased, but my basic operating system is fine, to prevent suspicion.

The simplest is to have it set up as an encrypted partition.

To wipe, simply lose the passphrase.

That easy.

Sure, you have to use a framework that doesn't leak the passphrase or store it anywhere accessible, but this way has been tried and tested, and it works.


Should I restrict access to the username and passwords in a web app? I'm starting to write my first serious web application and am thinking about how to store username and password information. There are plenty of articles detailing how storing plain text passwords is a very bad idea, and salted encrypted passwords are the way to go. However, I can find very little about restricting access to the user data itself.

I'm thinking about a situation where my web app is compromised and an attacker can read any information from my database. Even if I store usernames with encrypted passwords, the attacker would be able to lift all the usernames from the database (but at least they wouldn't get the passwords). However, I could restrict access so even my app didn't have access to this data. I was thinking of preventing my database user from viewing the table with usernames and passwords. You could then give access either via a stored procedure which takes the username and password - returning true if a username with the password is found, or presenting a view of hashed usernames and passwords, so even if compromised the real usernames can't be found.

Given I'm new to this, and I've failed to find any articles recommending anything similar, I'm guessing I've missed something simple that makes the above ideas insecure.

Thanks.

From a security standpoint, the threat is what happens when the underlying storage is exposed in any way, not just a sql injected select * from users; statement. Do you trust your administrators? Even the one you just had to let go? Who else has access to the box? Is it a virtual machine at a third party company? In the cloud? That is why getting hashing right is often emphasized first and foremost, so that user passwords have some defence even against an authorised user.

That said, I think controlling access via views would be beneficial, such that you can only execute pseudo-instructions "insert user", "delete user", "does user exist" and "are these credentials valid" from the perspective of your web app. This prevents enumeration except by repeated calls to the database with a known user identifier, which is better than allowing the attacker to grab the entire database. You're reducing the ease with which an attacker can grab that table in the case of any potential compromise/unforeseen vulnerability in your code or framework. The only question is compatibility with existing frameworks, although many provide the ability to extend or implement custom authentication backends.

On the subject of hashing, use PBKDF2 or bcrypt or an equivalent slow hash key derivation function with salt, not just a hash with salt. A decent article on the subject will explain this stage (sometimes they recommend repeating a hash for several thousand iterations; it's essentially the same idea).


Best practices for sending tax data to and from a tax professional I started doing business with my tax preparers when we all lived in the same city. Recently I moved to a different state. I requested that they send my "Tax Organizer" which contains some personal data:

Home address
Date of birth
Phone number
Email address
Bank name, but not account number
Name of employer
Some previous tax totals

I emailed them from the email address that I have on file with them, and gave them my name, address, and date of birth. Based on this, they emailed me a PDF file. The title of the file contained all but the last digit of my social security number. The file is password protected with my house number, which was included in my email to them.

I feel that I should have a discussion with my tax preparers about how they're handling my data, but I'm not sure what to tell them to do instead. What would a better way be for us to exchange data? Is there a way that doesn't involve me demanding they sign up for expensive new services?

In this case, using some form of encryption is the best.  However, the problem with encryption when talking to non-technology professionals is usually two fold:

Exchanging the password is often done insecurely, e.g. "send a password in a separate e-mail."  Password exchange, if using password protection, should be done on the phone or offline.  Also, as you've noted, often passwords are easy to guess.
Difficulty to use:  Often the user experience of using encryption tools (such as PGP) is so prohibitive, either for the person sending it or the recipient, that security measures are turned off or weakened in order to make it easier.

PDF encryption when using modern PDF software is actually pretty good and uses AES by default, at least in Adobe's implementation.  However, if you use a weak password, then those benefits are largely undone.

However, there are some good solutions out there that are easy to use:

E-mail encryption plugins:  There are various easy-to-use software out there for securing e-mails.  An example of this is Virtru.
Self-hosted File sharing services like ownCloud:  You can setup an instance of ownCloud to securely share files with a link and expiration dates can be set for files.
Hosted file sharing services like box.com:  You can share files encrypted with passwords and expiration dates.

I'd recommend suggesting these options to your tax preparation company, so they can avoid potentially costly and burdensome security incidents due to their current practices.


"Memory-Hard" vs. "Memory-Bound" Functions? One of the approaches in order to prevent Sybil or DoS attacks is CPU-bound PoW. However, because of the influence of Moore’s law, the memory-based approaches are suggested.
As actually there are two different terms: (1) memory-hard functions and (2) memory-bound functions;
Does it mean that we have two different memory-based techniques?
And if yes, What is the difference between these two terms?

To quote Wikipedia:

In cryptography, a memory hard function (MHF) is a function that costs significant amount of memory to evaluate. It is different from memory bound functions, the latter incurs cost by slowing down computation through memory latency. MHFs find their use as a form of proof-of-work.

In other words, Memory-Hard Functions require a specific amount of memory to evaluate them. Memory-Bound Functions do not specifically require a certain amount of memory, but their computational speed is dependent primarily on the memory available, in contrast to Compute-Bound Functions, where the computational speed is dependent on how fast you can calculate.

Does it mean that we have two different memory-based techniques?

Not really. Memory-Hard is a more desirable property when it comes to cryptography than Memory-Bound.

Why go for Memory-Hard Functions in the first place?

The reason for this being is that for any given algorithm, given a motivated enough actor, they can build a specialized ASIC that computes this algorithm fast and efficiently, in comparison to a CPU or a GPU. Memory, however, scales rather poorly still. Having an 8 GB large cache for an ASIC to ensure maximum performance is still not feasible, but the calculations can still be done on off-the-shelf hardware.
In the best possible case, you want an attacker with specialized, purpose-built hardware to have as little advantage over an off-the-shelf computer as possible. In other words, the only way for an attacker to attack you "faster" is to spend more money, not to spend the money "smarter".


padding problem in sha-1 hashed password For my own understanding of Websphere LTPA keys, I'm running this Java code, which works correctly when using the author's Base64 encoded, SHA1 hashed password:

String ltpa3DESKey = "IpGJOdpSxV3J8yWAuB+UiUlYCK7sAzSfENE5MLT2q+s=";

but fails when using mine:

String ltpa3DESKey = "ExGKSrGQPGN+q0GE1GlFhK7BB8bLB0mEp9ec1Vzxv8A\=";

Error:

Invalid escape sequence (valid ones are  \b  \t  \n  \f  \r  \"  \'  \\ )

But if I escape it:

String ltpa3DESKey = "ExGKSrGQPGN+q0GE1GlFhK7BB8bLB0mEp9ec1Vzxv8A\=";

I get:

 javax.crypto.BadPaddingException: Given final block not properly padded
    at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:991)
    at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:847)
    at com.sun.crypto.provider.DESedeCipher.engineDoFinal(DESedeCipher.java:294)
    at javax.crypto.Cipher.doFinal(Cipher.java:2165)
    at com.me.ltpa.LtpaToken.decrypt(LtpaToken.java:79)
    at com.me.ltpa.LtpaToken.getSecretKey(LtpaToken.java:62)
    at com.me.ltpa.LtpaToken.decryptLtpaToken(LtpaToken.java:110)
    at com.me.ltpa.LtpaToken.getInstance(LtpaToken.java:118)
    at com.me.ltpa.LtpaToken.main(LtpaToken.java:139)

I've read the basics of the Base64 Wikipedia article, but apparently backslashes aren't part of the Base64 alphabet. Any thoughts on how to proceed?

There are different variants of base64 encoding; while none I know of should produce a \, you might be better off just to use a URL-compatible version (which is done in java like the first google hit says).

Obiter Dictum:

There is usually no good reason to use 3DES and not AES. 3DES has long been deprecated.
Sha1 is not a good key derivation function, please use one that is designed for this purpose, like PBKDF2.


Is it possible to detect if a proxy is intercepting and sniffing or modifying data I was watching a video on burp suite and proxies which started me thinking. What if someone setup public proxies that anyone could use and also ran a tool like Burp suite to sniff and/or modify data.

Is this a serious security concern where any proxy you do not have control over should be avoided or can you detect this MITM based attack know that the proxy is safe to use.

The moment you decide on using a proxy outside of your control, the is always going to be the risk your data may be monitored. This should be no surprise as you are forgetting what a proxy is, and what it does. A proxy relays data from your system, to another, and vice versa. The questions become the following:

1) Q) Can my data be intercepted?

A) This shouldn't be a question. Your data is already going through the proxy

2) Q) Is my data encrypted?

A) This depends. Consider the following. I create a company called PrivacyProxies123.com and charge a fee. I pay for a certificate (SSL), and the cert is authenticated, and published to all browsers, etc.

1) You --> "I need to get to Facebook" --> PrivateProxies123.com

2) PrivateProxies123.com --> alright but accept my cert for this session --> You

3) You <--> transparent connection your browser acknowledge/accepted cert <--> PrivateProxies123.com

4) You <--> open FaceBook <--> Private Proxies decrypts (using their cert) <--> passes data to FB

The likelihood of this happening is low, not because it cannot be done, but the amount of disk space needed to store data, then the computing power necessary to parse data out of packets is a huge undertaking. So using the same concept above picture the following:

CriminalMaintainedProxy:

You --> I need to get to FB --> publicProxy

publicProxy (with a stolen cert) <--> absolutely I will send data to FB for you <--> You

You --> here is my user/pass <--> publicProxy (MiTM + ngrep "user\|pass")

When you use public proxies, or any proxy outside of your control, you run the risk of losing your data. The LIKELIHOOD however of this happening is altogether different. You need look solely at the history of Wikileaks (it started via TOR proxy sniffing) to understand this.


Anonymity on Facebook - how do they suggest people I should know? I want to know how Facebook discovers the people who you know in real life or who know you.

I tried the following to see if Facebook can still discover my acquaintances in real life and suggest them to me as a friend.

I connected using a VPN (an anonymous VPN, not one of those free VPN services). I have confirmed that it does not leak my actual IP address.
I cleared the cookies in my browser (specific to sites like Facebook, Google, and Yahoo) and started a fresh instance of Browsing Session. Anyway, cookies specific to Facebook only should matter, and I cleared them all.
I registered an email account with an email service provider who does not require a mobile number for registration. I used an email ID name which had no resemblance to my real name. I did not mention anything related to my geographical location while registering the email address. Please note that this was a fresh email address, and I have never used it to send an email or receive an email.
Now, I registered on Facebook, using a name which does not resemble my real name in any way.

However, Facebook requires phone number verification before you complete registration on Facebook. This is the only place where I specified my real phone number to receive their security code.

Once, I completed the verification. The moment I logged in, I could see Facebook giving me a list of suggestions of people I may know. It was surprising indeed, since this list was extremely accurate. It included people I knew in the past as well my current acquaintances. It makes me rather suspicious.

The only way I see that Facebook was able to identify the people who may know me or I may know them was using my phone number. So my assumption is:

They appear to have a deal with the telecommunication providers in different countries. Once you disclose your phone number, it looks like they get access to the entire list of phone numbers with whom you have corresponded in the past. Then, they do a second level lookup to identify the Facebook profiles of those corresponding phone numbers.

Also, interestingly, there are some people with whom I may not have ever corresponded with on phone. But of course, Facebook can find them through other people I know and suggest them to me.

Am I correct that Facebook was able to do all the correlation of people I know in real life using my phone number?

It would be interesting to see whether they could still correlate it if I use another phone number.

So, yes, they appear to have a deal with the Telecommunication
  Providers in different Countries.

Well that's ONE explanation.

Another one that I like better is simply that they have all their users' contact lists, thanks to their mobile application which no doubt reads everything and sends it back to their headquarters.

All they have to do after you register with your real phone number is look through all those contact lists, and find the people who possess your number.

This idea that they may have arrangements with telecom providers seems a little far-fetched to me, in great part because it is simply illegal in many countries to disclose phone records to anyone without a court order.


Should I avoid authenticating the user unnecessarily to a server? I’m thinking a RESTful client–server architecture. There will be routes that require users to authenticate before they can access the resource (e.g. creating or editing resources). Other resources are freely available—no authentication required.

I’m writing a frontend module to provide an abstraction from the request/response internals. Its job would be maintaining user state (once provided by the user) and sending the appropriate authentication headers.

Is it appropriate to send the authentication headers on every request? Or should the module be aware of which routes require authentication and only send the headers on those routes? Would the answer be different depending on whether I control the server or not?

Let’s assume for the sake of argument that the messages will be encrypted. Also note that I’m not intending to prompt the user for authentication whenever they access any route. I’m considering the case where the user has already provided authentication information for a previous request and now requests an unprotected resource.

Personally I cannot see a reason not to send the headers. If security were compromised by sending them, then that would also be the case where the headers are required. The added message length should not be worth considering. But then again, I don’t know the first thing about security, and spamming credentials doesn’t feel right to me.

Application complexity

Let us assume that client uses a secure channel like TLS to communicate with server. Then, sending authentication info like tokens does not expose any sensitive data.

If you make the client decide when to send authentication info, it increases complexity of your application. Each time you change protected or unprotected routes on the server, or each time you add new protected or unprotected resources, you have to update the client. There will always be some probability that you don't send authentication info when server requires it.

Header types

You are talking about authentication headers. If you mean cookies, then it is fine. But if you mean that you have your own header type, then this can be a problem. Browsers send cookies related to particular domain / path automatically. You can be sure that the cookies set by your application will not be sent to some external domain.
Whereas if you use your own header type, there is some chance that you send your authentication headers also to external domains. The owner of the external domain can use it to attack your application. That's why I discourage you from using custom headers for authentication and use cookies instead.


How can Facebook possibly stop a bruteforce attack on an account? I've been doing some research on hacking recently and I found some very interesting tutorials on brute force cracking. I have some questions to ask and I'll be using Facebook as an example.

Let's say that a man decides to crack the password of a single account. He downloads some Brute force program and sets it to try every single password combination with different IPs. The same IP would be used until the captcha comes up, then it will be replaced by a new one, so facebook would not be able to stop the attack by blocking the IP.

Maybe I'm just stupid (I'm not experienced in this stuff) but the only way I see that facebook can use to stop this attack is by locking the account itself (preventing anyone from logging in), but that's bad for business because it would prevent the owner itself from logging in. Does facebook actually do this for such attacks? I assume that a large website like facebook would receive these sort of attacks frequently from different crackers, and some of them may actually attack several accounts at the same time.

So what does facebook do to prevent this? Because I don't see anything it can do besides lock the account, which is not practical. It may take months or years but with this method the hacker will eventually be able to log in, would he not?

Despite the fact that Facebook locks accounts after many tries as the other answers said, you should keep in mind that "replacing your IP with a new one" is not as trivial as you make it sound.

Most internet service provider allow you to release your IP lease and get a new IP address, but the attacker will usually receive one from a small pool of a few hundred to a few thousand IP addresses.

Sure, there are proxy servers and the TOR network, but these will only supply the attacker with a few hundred additional IPs.

When the attacker has access to a botnet, they could use each bot of the botnet to request as many IP as they can get from their respective ISPs (good luck doing this without the user noticing anything). But large botnets are expensive.

Spoofing the IP address won't work either, because IP spoofing isn't easy to do when you want to not just sent packets but also receive the responses. When you want to brute-force a HTTP login, you will need to perform a TCP handshake first, which requires to be able to receive responses. Also you certainly want to parse the response to know if you were successful.

And even when you could theoretically use all ~4 Billion IPv4 addresses the internet has to offer, multiplying your tries with 4 billion only gives you enough tries to bruteforce passwords with 4-6 additional characters (depending on how many different characters the password contains).

And last but not least, the main problem with brute-forcing via network is the bandwidth you have available which greatly limits your attempts per seconds compared with what you can do when you brute-force hashes locally.

When you use a 8 character password with mixed case, numbers and special characters and no dictionary relation (common recommendation), brute-forcing via network is completely infeasible, even when the system uses no flood protection whatsoever.


When does chrome generate the prime number for TLS communication? In a TLS handshake using DH to negotiate a session key, a prime number on both sides must exist. Calculating a prime number large enough to be secure takes a litte computational time.

When does chrome generate this number?

Never. Chrome does not support FFDHE (which is how TLS calls Diffie–Hellman over the multiplicative group of integers modulo p) since 2016. Chrome only support ECDHE over NISP P-256, NISP P-384 and X25519 (ECDHE over Curve25519).

When chrome used to support FFDHE, the group and generator were set by the TLS server, not the TLS client, and the group prime was generally either a value from the RFCs (Oakley group) or a value generated by running openssl dhparam 2048 (or the moral equivalent), usually once on install and never again.

In TLS DHE, the values generated online (the ephemeral keys) do not require generating large prime numbers. For more, see for example these two excellent crypto.SE answer on the properties of p and q for secure DH by Thomas Pornin.

The cryptography code in Chrome is open source, and is used also in Android and in Google's servers. It's Google's fork of openssl called BoringSSL. The main people working on it are Adam Langley and David Benjamin. You can read the code and often the explanation for why something was done. For example, DHE support was dropped because too many servers use a 2048 bit cert and 1024 bit DHE, and this is insecure, but the servers will only get fixed in practice when they upgrade the OS, which for LTS OSs can take a very long time.

If you want to read similar code that is in use, you can look at openssl 1.1.1.

TLS state machine, choosing DH group, generating key.

Inside choosing group, we see that if not overriden, you get values from RFC (not generating new group online during TLS handshake).

Inside generate_key we see the private key is just random bits. No generating large prime numbers in sight. The public key is computed from the private key by modular exponentiation.


Linux forensics- How can I extract sysfs procfs from a raw image? I'm developing a software that extracts files from a Linux raw image (dd).
I use SluethKit to get the partitions & files, however i can't find any files inside /sys & /proc folders.

I know the sysfs & procfs do not contain regular files (as they are loaded to the kernel), but i need to "extract" them somehow...

Any idea how can it be done for forensics (withou mount)?

You can't. Those are virtual filesystems that don't correspond to anything on disk. They're just file-based interfaces to inside the kernel, and only ever exist on-the-fly in RAM.


How to prevent XSS when inserting untrusted data into a textarea? I want to prevent XSS in my demo chat application (written in PHP) that I've created for blog posts. The chat insert messages into a textarea. The problem is that it's vulnerable to XSS. Somone can write </textarea><script>alert('x');</script>. To prevent this I can write:

preg_replace("%</(text)(area)>%i", "</\\1&#8203;\\2>", $message);

It split textarea with zero width space.

Is this enough to prevent XSS for this one case of textarea messages? Can you encode textarea in a different way like <script tag that will bypass this filter?

It's not enough. One possible bypass:

</textarea ><script>alert(1)</script>

This works because of the extra space after textarea.

You could adapt your filter to catch this as well. But I wouldn't recommend it because the approach is already flawed.

If you do not need to allow a user-supplied subset of HTML to be rendered (which isn't the case here), then the proper protection against XSS in a HTML context is HTML-encoding:

$message = htmlentities($message, ENT_QUOTES, 'UTF-8');
echo "<textarea>$message</textarea>";

Now the message will be displayed as intended, but there is no possibility for XSS.


Question about XSS example A friend of mine received an email from someone (a reputable sender), and I noticed that the following was tagged onto the end of the subject:

</title>
 (contents)

where

(site) = 116jurist(dot)ru
(contents) = þðèäè÷åñêèå-êîíñóëüòàöè

My question has two parts:

Do the given contents mean anything? It looks like its encoded for XSS filter evasion, but I don't know what kind of encoding it could be. The contents may only be partial (may have been cut short in the subject line).
Is there something that I can/should do about this? Googling parts of the given code returns tens of thousands of sites with this code present.

I think the confusion lies in some incorrect text encoding.

With proper encoding the (contents) decodes to Russian:

юридические-консультаци

Which can be Google translated to the English:

legal-consulting

So it appears to be some form of SEO Spam. However, how it got to the end of the subject line of your friend's email is not clear, though I would hazard a guess that it was inserted by some malware.

You should inform your friend that they are potentially infected by malware.


Can an OpenPGP primary key revoke a subkey of a subkey? I would like to have the following setup: a main Key, saved in a hardware wallet or paper wallet. Let's call this level 1. The idea is to never have to use this key again, as a sort of last resort.

The main key has a subkey, let's call this level 2, this key is on the main device and still follow some strict security rules, but is actively used to generate subkey used by applications.

The level 2 subkey, let's call them level 3 key, follow less secure rules, for example are duplicated in multiple device, maybe even via storage cloud services (encrypted).

The question is: if my main device get stolen or compromised, can I use the level 1 key to revoke level 2 and level 3 keys?

As far as I understand, as long as level 2 and level 3 public keys are uploaded on the key server, it should be possible to follow this chain of thrust.

It seems you mix up two concepts of OpenPGP: subkeys and certifications.

OpenPGP subkeys allow a primary key, which is used to create subkeys (technically, the subkeys are created independently but signed by the primary key using a binding signature). Such a set of keys (one primary key, an arbitrary number of subkeys) is considered a single trust unit in OpenPGP, ie. the subkeys are trusted if the primary key is trusted (trusting the subkey is not considered an additional step in the web of trust).

There is no such thing as "subkeys of subkeys", though -- only primary keys are allowed to bind subkeys. You may not construct multi-level subkey hierarchies.

On the other hand, there are certifications in-between primary keys. Using them, you can construct arbitrary trust networks (the web of trust) -- but each primary key is considered a single trust entity, and any other party would have to follow that trust chain and issue trust on each of the keys manually (in contrast to real subkeys, as soon as the primary key is considered valid, automatically the same applies to the subkeys).

Generally, consider a primary key to form an identity -- and (a single level of) subkeys can be used for key management and "keeping the primary key safe". You will have to use it from time to time -- creating and manipulating user IDs, subkeys and certifications on other keys is only allowed using the primary key. Certifications are used to provide trust statements among such identities. While you could of course use certifications to construct such a "two-level-subkey-hierarchy" technically, you are breaking the expected use of certifications and will have a hard time explaining third parties which key to trust (and use for what purpose).