SAC:LDAP

OSGeo endevours to offer a "single signon" for OSGeo services. User help is available in Osgeo Userid Help.
It is currently used for Subversion, Trac, Drupal, QGIS services (hub.qgis.org and plugins.qgis.org), www.osgeo.org, Gitea, Nextcloud, Matrix-Synapse and shell access (opt-in). 
A staged migration to internal LDAP for mediawiki was initiated in 2021. Other services may be possible in the future. 
LDAP management is the responsibility of SAC (the System Administration Committee).
= LDAP Server (OpenLDAP) =
The userid database is kept in LDAP on ldap.osgeo.org (secure vm).
It is served by OpenLDAP (OpenLDAP Admin Guide).
The ldap daemon can manipulated as 'root' user with,
 $ service slapd [start|stop|restart]
We have moved to OLC:
http://www.zytrax.com/books/ldap/ch6/slapd-config.html#intro
So config file is in:
 /etc/ldap/slapd.d/cn-config # should be modified with ldapmodify
IP access filter is set by via OSGeo7 ufw firewall:
refer to OSGeo7 host for details
== LDAP structure ==
Currently ldap structure is pretty basic. The purpose for keeping this structure simple is to allow for a more complex structure to be evolved as ldap becomes increasingly integrated into the full osgeo systems structure.
*dc=osgeo,dc=org
**cn=Manager
**ou=people 
***Separate entity for each user
*:uid=login,ou=people,dc=osgeo,dc=org
*:objectClass=inetOrgPerson
*:cn=firstName lastName
*:sn=lastName
*:uid=login
*:mail=email@address
*:userPassword={md5}YPTyViiMKhiuWKEmFUOKLA==
*:Also contains posixAccount and shadowAccount fields if this account is login enabled.
**ou=projects,dc=osgeo,dc=org
*:objectClass=organizationalUnit
*:ou=project
*:description=separate entity for each osgeo project with list of members
***Separate entity for each project group
*:cn=admin,ou=projects,dc=osgeo,dc=org
*:objectClass=groupOfNames
*:cn=admin
*:description=osgeo sysadmin group
**ou=svn
*:objectClass=organizationalUnit
*:ou=svn
*:description=separate entity for each repository with list of members with commit rights
***separate entity for each svn group (for example FDO)
*:cn=fdo,ou=svn,dc=osgeo,dc=org
*:objectClass=groupOfNames
*:cn=fdo
*:member= dn of member
**ou=Shell (parent for "login" groups)
*:description=NextUID:nnnnn (the next unix uidNumber value to assign)
***cn=telascience,ou=Shell,dc=osgeo,dc=org (list of all userids with telascience login access)
*:uniqueMember=dn of member
Example ldif file
 version: 1
 dn: dc=osgeo,dc=org
 objectClass: dcObject
 objectClass: organization
 description: OSGeo ldap dit
 o: OSGeo
 dc: osgeo
 dn: cn=Manager,dc=osgeo,dc=org
 objectClass: organizationalRole
 cn: Manager
 dn: ou=people,dc=osgeo,dc=org
 ou: people
 description: all users of osgeo
 objectClass: organizationalUnit
 dn: uid=jsmith,ou=people,dc=osgeo,dc=org
 objectClass: inetOrgPerson
 uid: jsmith
 cn: Jon Smith
 sn: Smith 
 givenName: Jon
 mail: jsmith@somewhere.com
 userPassword: {md5}5Or4zfzGqo3jh/6iIUgKcA==
 dn: uid=jbrown,ou=people,dc=osgeo,dc=org
 objectClass: inetOrgPerson
 uid: jbrown
 cn: Jane Brown
 sn: Brown
 givenName: Jane 
 mail: jbrown@someotherplace.com
 userPassword: {md5}1iWhTyvkK2m4Uuar+Dp/IA==
 dn: ou=projects,dc=osgeo,dc=org
 ou: projects
 description: separate entity for each osgeo project with list of members
 objectClass: organizationalUnit
 dn: cn=admin,ou=projects,dc=osgeo,dc=org
 cn: admin
 description: osgeo sysadmin group
 objectClass: groupOfNames
 member: uid=jbrown,ou=people,dc=osgeo,dc=org
 member: uid=jsmith,ou=people,dc=osgeo,dc=org
 dn: ou=svn,dc=osgeo,dc=org
 ou: svn
 description: separate entity for for each repository.list of members with commit rights
 objectClass: organizationalUnit
 dn: cn=fdo,ou=svn,dc=osgeo,dc=org
 objectClass: groupOfNames
 cn: fdo
 member: uid=jsmith,ou=people,dc=osgeo,dc=org
 member: uid=jbrown,ou=people,dc=osgeo,dc=org
 dn: cn=gdal,ou=svn,dc=osgeo,dc=org
 cn: gdal
 objectClass: groupOfNames
 objectClass: top
 member: uid=jbrown,ou=people,dc=osgeo,dc=org
 member: uid=jsmith,ou=people,dc=osgeo,dc=org
 dn: cn=mapbender,ou=svn,dc=osgeo,dc=org
 objectClass: groupOfNames
 cn: mapbender
 member: uid=jsmith,ou=people,dc=osgeo,dc=org
 dn: cn=mapguide,ou=svn,dc=osgeo,dc=org
 objectClass: groupOfNames
 cn: mapguide
 member: uid=jbrown,ou=people,dc=osgeo,dc=org
= LDAP Clients =
== Web interface ==
The LDAP web admin tools were initially written by Frank Warmerdam and subsequently improved by Sandro Santilli.
They are written in python and can currently be found in /usr/lib/cgi-bin on the "old-web" container on OSGeo7 (ie. id.osgeo.org).
From may 12, 2016, cgi-bin scripts are managed with a local git repository (/usr/lib/cgi-bin/.git). A clone of the git repository is published as a private repository (you will get a 404 unless you are part of the SAC team) on the SAC:Gitea service.
 
The scripts read the LDAP manager password from a credentials file that will need to be updated anytime the master LDAP manager password is changed.
Some public notes on OSGeo userids available at:
 http://www.osgeo.org/osgeo_userid
=== User registration ===
The main entry point for OSGeo users is the registration form:
 https://id.osgeo.org/ldap/create
NOTE: as of June 2016 the public user registration form may require entering a "mantra" (a pseudo-secret passphrase) - already registered trusted users can find out what the mantra is and how to share on https://git.osgeo.org/gitea/sac/userid-mantra/wiki (requires login and access grant, otherwise the page will appear as 404).
=== User edit ===
To edit an individual userid use /ldap/edit (ldap_user_edit.py). If you add ?userid=osgeo_userid you can edit someone elses ldap entry as long as you are in the admins group. 
 https://id.osgeo.org/ldap/edit
 https://id.osgeo.org/ldap/edit?userid=osgeotest123
=== Password reset ===
Users can request a password reset using this form:
 https://id.osgeo.org/ldap/reset
=== Users lookup ===
Administrators can login to this special ldap search tool, and will see email address, and will have a link to edit the LDAP entries. You need to be in the cn=admin,ou=projects listing. 
 https://id.osgeo.org/ldap/admin/search
There is also a non-admin version of the search, but will not show email nor links to edit:
 https://id.osgeo.org/ldap/search
=== Groups management ===
SVN and other groups can be administered with the group editor. You need to either be in the group being viewed/modified or in the cn=admin,ou=projects group in order to edit a group. 
Edit GDAL commit list:
 https://id.osgeo.org/ldap/group?group=gdal
Edit Admins list:
 https://id.osgeo.org/ldap/group?group=admin&ou=projects
=== Shell group management ===
To edit the list of people who have shell access to the various projects servers, including the download server use the following url. You have to be in the group already, or in the Admins group in order to add and remove people. Adding someone will add the required posixAccount and related attributes to the users LDAP entry. 
 https://id.osgeo.org/ldap/shell
The ldap_shell.py script can also operate on other groups used for shell access using the group attribute. Currently the only other group is the sac group used for shell access on the core services VMs at OSU OSL.
 https://id.osgeo.org/ldap/shell?group=sac
== Command line interface ==
The commandline interface can be used by any of the OSGeo machines but not from outside the network.
The tools are from the ldap-utils debian package. 
All ``ldap-utils`` commands will use systemwide or user-specific files for default configuration so things like LDAP Base (-b dc=osgeo,dc=org) and LDAP URI (-H ldaps://ldap.osgeo.org) need not be specified on each commandline from properly configured machines (grep URI /etc/ldap/ldap.conf; grep BASE /etc/ldap/ldap.conf).
=== Editing the LDAP database users and groups ===
Editing LDAP database requires an admin password or root access to secure container. The -W switch asks for a password prompt (known by SAC:Primary Administrators).
Loading an ldif file into an ldap directory:
 $ ldapadd -a -W -x -D "cn=Manager,dc=osgeo,dc=org" -f fileName.ldif
Deleting an account (by uid) from the database through "ldapdelete":
 $ uid="account-to-be-deleted" ldapdelete -W -D cn=Manager,dc=osgeo,dc=org -x "uid=${uid},ou=People,dc=osgeo,dc=org"
Deleting an account (by uid) from the database through an ad-hoc script from the "www.osgeo.org" machine:
 $ /osgeo/tools/ldap/ldap_delete_user "account-to-be-deleted"
Resetting a user password:
 $ uid="account-to-be-reset" ldappasswd -S -W -D cn=Manager,dc=osgeo,dc=org -x "uid=${uid},ou=People,dc=osgeo,dc=org"
 # Or from the "www.osgeo.org" machine, run:
 $ /osgeo/tools/ldap/ldap_reset_user_password "account-to-be-reset"
 # A new random password will be sent to the user. Asks for confirmation.
=== Editing the LDAP database config ===
Increasing default page limit -
 $ ssh root@osgeo9-secure
 $ ldapmodify -H ldapi:/// -Y EXTERNAL -f modify_max_page_limit.ldif
 $ systemctl restart slapd
 $ ldapsearch -H ldapi:/// -Y EXTERNAL -b cn=config | grep olcSizeLimit #to confirm change
Where modify_max_page_limit.ldif looks something like
 dn: olcDatabase={-1}frontend,cn=config
 changetype: modify
 replace: olcSizeLimit
 olcSizeLimit: 500
=== Querying the LDAP database ===
The ldapsearch command is used to query a LDAP database.
This command dumps the first 500 entries of the ldap database. Note that secure-dev on osgeo4 has limit set to 40000 for easier pull by admins. refer to modify section for instructions how to set the limits.
 $ ldapsearch -x
Refine your search based on the LDAP structure listed above.
Example: list all SVN groups (override BASE to limit the ou to svn, then search for objects with the right objectClass).
 $ ldapsearch -s one -b ou=svn,dc=osgeo,dc=org -x "objectClass=groupOfNames" | grep ^cn
Example: list all entries created after April 1st 2016:
 $ ldapsearch -x "createTimestamp>=20160401100000Z"
Example: list all entries created in 2019:
 $ ldapsearch -x '(&(createTimestamp>=20190101000000Z)(!(createTimestamp>=20200101000000Z)))'
The fields to be shown in the output can be specified as additional parameters, like:
 $ ldapsearch -x "uid=strk" createtimestamp modifytimestamp
Find the email addresses of all members in the GRASS GIS core group:
 $ ldapsearch -x '(|'`ldapsearch -x 'cn=grass' | grep ^member | sed 's/.* uid=\([^,]*\),.*/(uid=\1)/' | tr -d '\n'`')' | grep ^mail | sed 's/^mail: //'
Find the email addresses of all members in the GRASS GIS core group and store as CSV file (ID, name, email):
 $ ldapsearch -x '(|'`ldapsearch -x 'cn=grass' | grep ^member | sed 's/.* uid=\([^,]*\),.*/(uid=\1)/' | tr -d '\n'`')' | \
 grep '^u: /\n|/'| sed 's/^cn: //' | sed 's/^sn: //' | sed 's/^mail: //' | sed 's/^cn:: //' | sed 's/^sn:: //' \
 | tr '\n' ',' | tr '|' '\n' | sed 's+,,$++g'
If you are on a server that is whitelisted to query osgeo, but the ldap.conf is not setup, you migh need to explicitly specify the host like so
 $ ldapsearch -x "uid=robe" -b "dc=osgeo,dc=org" -H ldaps://ldap.osgeo.org 
If you get an error you can debug source with:
 $ ldapsearch -x "uid=robe" -b "dc=osgeo,dc=org" -H ldaps://ldap.osgeo.org -d 1
For more complex queries, see http://www.ldapexplorer.com/en/manual/109010000-ldap-filter-syntax.htm
= Monitoring =
Web-based users creation and editing can be monitored looking at the '/var/log/apache2/www_access.log' files on 'www.osgeo.org'.
A cronjob on TracsvnVM is setup to run hourly ('/etc/cron.hourly/check_excessive_new_ldap_users')
and report to the SAC mailing list if more than a given max users are created in that timespam (check the script for details). The scripts to generate the reports are again on 'trac.osgeo.org' machine under '/osgeo/tools/ldap', where a git repository exists.
= Troubleshooting =
* "sudo" unavailable to LDAP-authenticated users on Debian7. See:
*:https://bugs.launchpad.net/ubuntu/+source/gnutls26/+bug/926350 and
*:https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=579647
** Short form: "libgnutls" in Debian7 is linked against "libgcrypt" which refuses to work in setuid-binaries - like "sudo".
** Solution: Rebuild "libgnutls" using "libnettle" instead of "libgcrypt":
*** aptitude install libhogweed2 libnettle4 nettle-dev libp11-kit-dev
*** apt-get remove libgnutls-dev
*** apt-get source gnutls26
*** # remove --with-libgcrypt from the debian/rules file
*** # build using this command:
*** debuild -i -uc -us -b
** Functional GnuTLS packages available on "secure" in /home/martin/GnuTLS.Deb/
* "sudo" unavailable to LDAP-authenticated users on Debian6 with backports. See:
*:http://lists.osgeo.org/pipermail/sac/2012-October/004120.html
** Solution:
 <hostname>:~# cat /etc/apt/preferences.d/sudo.pref 
 Package: sudo
 Pin: version 1.7.4*
 Pin-Priority: 1000
Category:Infrastructure
Category:Services