Fundraising/techops/procedures/services-system-authentication

This is an outline of Fundraising system authentication and two-factor authentication as of 10/2015.
= goals =
* meet 2FA and password requirements for PCI
* easy to support
* flexibility and resiliency
= design decisions =
* SSH key authentication is standard and secure
* 2FA on bastions with single factor on internal hosts is acceptable because internal hosts are only accessible via bastions
* Yubikeys with Yubico cloud API are easy to administer and use
* Kerberos passwords are a well-integrated on-premise option that enforces PCI password standards
* Shadow passwords can be used as a fallback for SREs if network is inaccessible
* Radius->Auth Server->Yubico API limits outbound firewall policy requirements
= authentication scheme overview =
* all servers use ssh public keys as the primary authentication method
* bastion server ssh requires 2FA
:* first factor is the ssh public key
:* second factor is "a password" satisfied by yubikey, kerberos, or local shadow password
* all servers use the same yubikey->kerberos->shadow-password scheme where passwords are used
:* yubikeys are authenticated via radius server passthrough to the yubico cloud API
:* kerberos preempts local shadow passwords
:* fallback to shadow password authentication as backup if auth servers are inaccessible
* firewall policies enforce bastion use and restrict access to internal hosts by unix group
* users, groups, ssh keys, unix passwords, and yubikey user mappings are all managed with puppet
* users can change their kerberos password from any host
= modified PAM password backend =
= authentication-related configuration =
== all hosts ==
=== /etc/ssh/sshd_config ===
We enable 2FA for bastion servers, first checking ssh public key and then password:
AuthenticationMethods publickey,keyboard-interactive
We disable 2FA for other servers:
AuthenticationMethods publickey
Note that OpenSSH can be configured to behave differently based on various session criteria using the Match directive. For example to do a soft rollout, we initially enabled 2FA only for members of the sysops group:
AuthenticationMethods publickey
Match Group sysops
	AuthenticationMethods publickey,keyboard-interactive
=== /etc/pam.d/sshd ===
With OpenSSH set to use PAM, this is the next step with keyboard-interactive. Our config just forwards to pam.d/common-auth. 
@include common-auth
=== /etc/pam.d/common-auth ===
This is the default password handling sequence, which is used for ssh, terminal, sudo, etc.
auth [success=ignore default=1] pam_exec.so expose_authtok quiet /usr/local/bin/yubikey_otp_filter
auth [success=3 default=2] pam_radius_auth.so
auth [success=2 default=ignore] pam_krb5.so minimum_uid=500 use_first_pass
auth [success=1 default=ignore] pam_unix.so use_first_pass
auth requisite pam_deny.so
=== /etc/pam.d/common-passwd ===
User-accessed password utilities administer kerberos passwords.
password [success=1 default=ignore] pam_krb5.so minimum_uid=500
password requisite pam_deny.s
password required pam_permit.so
=== /usr/local/bin/yubikey_otp_filter ===
This is the simple script pam_exec.so invokes in an auth context.
#!/usr/bin/perl
use Sys::Syslog;
if (<STDIN> =~ /^[a-z]{44}[\x00\r\n]*$/) {
 printlog("$ENV{'PAM_USER'} Yubikey OTP detected");
 exit 0;
} else {
 printlog("$ENV{'PAM_USER'} non-Yubikey password");
 exit 1;
}
sub printlog {
 Sys::Syslog::setlogsock('unix');
 Sys::Syslog::openlog('pam_yubikey_filter','ndelay,pid','auth');
 Sys::Syslog::syslog('info',shift);
 Sys::Syslog::closelog();
}
=== /etc/pam_radius_auth.conf ===
The radius client configuration tries two possible servers with a 3 second timeout.
# server secret timeout
10.1.1.5 pa$$word 3
10.2.0.5 pa$$word 3
=== /etc/krb5.conf ===
The kerberos client config tries two possible kdc servers for authentication. Password admin is directed to the master.
WMF-FUNDRAISING = {
 kdc = 10.1.1.5
 kdc = 10.2.0.5
 admin_server = 10.1.1.5
}
== auth servers ==
Authentication servers have the above settings plus kerberos and freeradius server configuration. The kerberos and freeradius settings are fairly generic, and mostly beyond the scope of this overview.
=== /etc/freeradius/users ===
Freeradius is configured to act as a simple pass through to the yubico cloud API. The server is configured to use PAM to authenticate users. This means every user must exist on the auth server, though login can be disabled.
DEFAULT Auth-Type := PAM
=== /etc/pam/radiusd ===
This configuration comes into play with freeradius using PAM to authenticate users. This is where pam_yubico makes the call out to the yubico cloud API.
auth required pam_yubico.so id=16 debug authfile=/etc/yubikey_mappings url=https://api.yubico.com/wsapi/2.0/verify?id=%d&otp=%s
=== /etc/yubikey_mappings ===
This file maps unix username to yubikey public ID, which are included as the first 12 characters of every OTP. 
jg:ccccccebebeb
bk:ccccccebebeb
...
= administration =
* users and groups are administered via (frack)puppet/hieradata/user/*.yaml
* yubikey user mappings and ssh public keys are in (frack)puppet-private
* shadow passwords are in (frack)puppet-private as hashes
* kerberos kdc is administered on the frauth server, users can change their passwords from any host via passwd command
= user experience =
== ssh directly to bastion ==
jg@laptop:~> ssh frbast-eqiad.wikimedia.org
(jg@frbast) Password: 
frbast1002 fundraising bastion host (role::bastion)
== ssh to internal host, via tunnel through bastion ==
jg@laptop:~> ssh frlog1002
(jg@frbast) Password:
frlog1002 fundraising logger host (role::logger)