add a clone hook for ubuntu-cloud images

This allows ability to now specify '--userdata' arguments to 'create' or
to 'clone'. So now, the following means very fast start of instances with
different user-data.

$ sudo lxc-create -t ubuntu-cloud -n precise -- \
   -r precise --arch amd64

$ sudo lxc-clone -B overlayfs -o precise -s -n ephem1 \
   --userdata="my.userdata1"
$ sudo lxc-clone -B overlayfs -o precise -s -n ephem2 \
   --userdata="my.userdata2"

Also present here is
 * an improvement to the static list of Ubuntu releases. It uses
   ubuntu-distro-info if available degrades back to a static list on failure.
 * moving of the replacement variables to the top of the create template This
   is just to make it more obvious what is being replaced and put them in a
   single location.

Signed-off-by: Scott Moser <smoser@ubuntu.com>
ubuntu-cloud-prep: fix bad declare of VERBOSITY

Signed-off-by: Scott Moser <smoser@ubuntu.com>
Signed-off-by: Serge Hallyn <serge.hallyn@ubuntu.com>
lxc-destroy: Fix regular expression for getting rootfs

The `lxc-destroy` script was using a simple `grep` for extracting
`lxc.rootfs` from the lxc config. This regex also matches commented lines
and breaks at least removing btrfs subvolumes if the string `lxc.rootfs`
is mentioned in a comment. Furthermore, due to the unescaped dot in the
regex it would also match other wrong strings like `lxc rootfs`.

This patch modifies the regular expression to correctly match the beginning
of the line plus potential whitespace characters and the string
`lxc.rootfs`.

Signed-off-by: Franz Pletz <fpletz@fnordicwalking.de>
Signed-off-by: Serge Hallyn <serge.hallyn@ubuntu.com>
ubuntu-cloud-prep: cleanup, fix bug with userdata

--userdata was broken, completely missing an implementation.
This adds that implementation back in, makes 'debug' logic
correct, and then also improves the doc at the top.

Signed-off-by: Scott Moser <smoser@ubuntu.com>
Signed-off-by: Serge Hallyn <serge.hallyn@ubuntu.com>
mountcgroups: use the right configuration file!

Signed-off-by: Serge Hallyn <serge.hallyn@ubuntu.com>
lxc.conf.sgml.in: note the arguments and environment variables passed to hooks

Signed-off-by: Serge Hallyn <serge.hallyn@ubuntu.com>
hooks/Makefile.am: add ubuntu-cloud-prep

Signed-off-by: Serge Hallyn <serge.hallyn@ubuntu.com>
add lxc-user-nic

It is meant to be run setuid-root to allow unprivileged users to
tunnel veths from a host bridge to their containers.  The program
looks at /etc/lxc/lxc-usernet which has entries of the form

	user type bridge number

The type currently must be veth.  Whenver lxc-user-nic creates a
nic for a user, it records it in /var/lib/lxc/nics (better location
is needed).  That way when a container dies lxc-user-nic can cull
the dead nic from the list.

The -DISTEST allows lxc-user-nic to be compiled so that it uses
files under /tmp and doesn't actually create the nic, so that
unprivileged users can compile and test the code.  lxc-test-usernic
is a script which runs a few tests using lxc-usernic-test, which
is a version of lxc-user-nic compiled with -DISTEST.

The next step, after issues with this code are raised and addressed,
is to have lxc-start, when running unprivileged, call out to
lxc-user-nic (will have to exec so that setuid-root is honored).
On top of my previous unprivileged-creation patchset, that should
allow unprivileged users to create and start useful containers.

Also update .gitignore.

Signed-off-by: Serge Hallyn <serge.hallyn@ubuntu.com>
lxc-user-nic: specify config and db files in autoconf

Signed-off-by: Serge Hallyn <serge.hallyn@ubuntu.com>
cgroups: rework to handle nested containers with multiple and partial mounts

Currently, if you create a container and use the mountcgruop hook,
you get the /lxc/c1/c1.real cgroup mounted to /.  If you then try
to start containers inside that container, lxc can get confused.
This patch addresses that, by accepting that the cgroup as found
in /proc/self/cgroup can be partially hidden by bind mounts.

In this patch:

Add optional 'lxc.cgroup.use' to /etc/lxc/lxc.conf to specify which
mounted cgroup filesystems lxc should use.  So far only the cgroup
creation respects this.

Keep separate cgroup information for each cgroup mountpoint.  So if
the caller is in devices cgroup /a but cpuset cgroup /b that should
now be ok.

Change how we decide whether to ignore failure to set devices cgroup
settings.  Actually look to see if our current cgroup already has the
settings.  If not, add them.

Finally, the real reason for this patch: in a nested container,
/proc/self/cgroup says nothing about where under /sys/fs/cgroup you
might find yourself.  Handle this by searching for our pid in tasks
files, and keep that info in the cgroup handler.

Also remove all strdupa from cgroup.c (not android-friendly).

Signed-off-by: Serge Hallyn <serge.hallyn@ubuntu.com>
lxc_destroy: print an error if the container is not defined.

Signed-off-by: Serge Hallyn <serge.hallyn@ubuntu.com>
lxc-stop: exit with 1 or 2, not -1 or -2.

Signed-off-by: Serge Hallyn <serge.hallyn@ubuntu.com>
Fix return type of read/write utility functions.

Signed-off-by: Christian Seiler <christian@iwakd.de>
Acked-by: Serge E. Hallyn <serge.hallyn@ubuntu.com>
lxc-attach: Completely rework lxc-attach and move to API function

 - Move attach functionality to a completely new API function for
   attaching to containers. The API functions accepts the name of the
   container, the lxcpath, a structure indicating options for attaching
   and returns the pid of the attached process. The calling thread may
   then use waitpid() or similar to wait for the attached process to
   finish. lxc-attach itself is just a simple wrapper around the new
   API function.

 - Use CLONE_PARENT when creating the attached process from the
   intermediate process. This allows the intermediate process to exit
   immediately after attach and the original thread may supervise the
   attached process directly.

 - Since the intermediate process exits quickly, its only job is to
   send the original process the pid of the attached process (as seen
   from outside the pidns) and exit. This allows us to simplify the
   synchronisation logic by quite a bit.

 - Use O_CLOEXEC / SOCK_CLOEXEC on (hopefully) all FDs opened in the
   main thread by the attach logic so that other threads of the same
   program may safely fork+exec off. Also, use shutdown() on the
   synchronisation socket, so that if another thread forks off without
   exec'ing, the synchronisation will not fail. (Not tested whether
   this solves this issue.)

 - Instead of directly specifying a program to execute on the API
   level, one specifies a callback function and a payload. This allows
   code using the API to execute a custom function directly inside the
   container without having to execute a program. Two default callbacks
   are provided directly, one to execute an arbitrary program, another
   to execute a shell. The lxc-attach utility will always use either
   one of these default callbacks.

 - More fine-grained control of the attached process on the API level
   (not implemented in lxc-attach utility yet, some may not be sensible):
     * Specify which file descriptors should be stdin/stdout/stderr of
       the newly created process. If fds other than 0/1/2 are
       specified, they will be dup'd in the attached process (and the
       originals closed). This allows e.g. threaded applications to
       specify pipes for communication with the attached process
       without having to modify its own stdin/stdout/stderr before
       running lxc-attach.
     * Specify user and group id for the newly attached process.
     * Specify initial working directory for the newly attached
       process.
     * Fine-grained control on whether to do any, all or none of the
       following: move attached process into the container's init's
       cgroup, drop capabilities of the process, set the processes's
       personality, load the proper apparmor profile and (for partial
       attaches to any but not mount-namespaces) whether to unshare the
       mount namespace and remount /sys and /proc. If additional
       features (SELinux policy, SMACK policy, ...) are implemented,
       flags for those may also be provided.

Signed-off-by: Christian Seiler <christian@iwakd.de>
Acked-by: Serge E. Hallyn <serge.hallyn@ubuntu.com>
Add helper functions to convert va_list of char* to char**.

Signed-off-by: Christian Seiler <christian@iwakd.de>
Acked-by: Serge E. Hallyn <serge.hallyn@ubuntu.com>
Add attach support to container C API

Signed-off-by: Christian Seiler <christian@iwakd.de>
Acked-by: Serge E. Hallyn <serge.hallyn@ubuntu.com>
apparmor/attach: make sure buffer is NUL-terminated

Signed-off-by: Christian Seiler <christian@iwakd.de>
Acked-by: Serge E. Hallyn <serge.hallyn@ubuntu.com>
python: improve convert_tuple_to_char_pointer_array

convert_tuple_to_char_pointer_array now also accepts lists and not only
tuples when converting to a C array. Other fixes:

 - some checking that it's actually a list/tuple before trying to
   convert
 - off-by-a-few-bytes allocation error
   (sizeof(char *)*n+1 vs. sizeof(char *)*(n+1)/calloc(...))

Signed-off-by: Christian Seiler <christian@iwakd.de>
Acked-by: Serge E. Hallyn <serge.hallyn@ubuntu.com>
python: add attach support

Add methods attach() and attach_wait() to the Python API that give
access to the attach functionality of LXC. Both accept two main
arguments:

1. run: A python function that is executed inside the container
2. payload: (optional) A parameter that will be passed to the python
            function

Additionally, the following keyword arguments are supported:

attach_flags: How attach should operate, i.e. whether to attach to
              cgroups, whether to drop capabilities, etc. The following
              constants are defined as part of the lxc module that may
              be OR'd together for this option:
                LXC_ATTACH_MOVE_TO_CGROUP
                LXC_ATTACH_DROP_CAPABILITIES
                LXC_ATTACH_SET_PERSONALITY
                LXC_ATTACH_APPARMOR
                LXC_ATTACH_REMOUNT_PROC_SYS
                LXC_ATTACH_DEFAULT
namespaces: Which namespaces to attach to, as defined as the flags that
            may be passed to the clone(2) system call. Note: maybe we
            should export these flags too.
personality: The personality of the process, it will be passed to the
             personality(2) syscall. Note: maybe we should provide
             access to the function that converts arch into
             personality.
initial_cwd: The initial working directory after attaching.
uid: The user id after attaching.
gid: The group id after attaching.
env_policy: The environment policy, may be one of:
              LXC_ATTACH_KEEP_ENV
              LXC_ATTACH_CLEAR_ENV
extra_env_vars: A list (or tuple) of environment variables (in the form
                KEY=VALUE) that should be set once attach has
                succeeded.
extra_keep_env: A list (or tuple) of names of environment variables
                that should be kept regardless of policy.
stdin: A file/socket/... object that should be used as stdin for the
       attached process. (If not a standard Python object, it has to
       implemented the fileno() method and provide a fd as the result.)
stdout, stderr: See stdin.

attach() returns the PID of the attached process, or -1 on failure.

attach_wait() returns the return code of the attached process after
that has finished executing, or -1 on failure. Note that if the exit
status of the process is 255, -1 will also be returned, since attach
failures result in an exit code of 255.

Two default run functions are also provided in the lxc module:

attach_run_command: Runs the specified command
attach_run_shell: Runs a shell in the container

Examples (assumeing c is a Container object):

c.attach_wait(lxc.attach_run_command, 'id')
c.attach_wait(lxc.attach_run_shell)
def foo():
  print("Hello World")
  # the following line is important, otherwise the exit code of
  # the attached program will be -1
  # sys.exit(0) will also work
  return 0
c.attach_wait(foo)
c.attach_wait(lxc.attach_run_command, ['cat', '/proc/self/cgroup'])
c.attach_wait(lxc.attach_run_command, ['cat', '/proc/self/cgroup'],
              attach_flags=(lxc.LXC_ATTACH_DEFAULT &
              ~lxc.LXC_ATTACH_MOVE_TO_CGROUP))

Note that while it is possible to execute Python code inside the
container by passing a function (see example), it is unwise to import
modules, since there is no guarantee that the Python installation
inside the container is in any way compatible with that outside of it.
If you want to run Python code directly, please import all modules
before attaching and only use them within the container.

Signed-off-by: Christian Seiler <christian@iwakd.de>
Acked-by: Serge E. Hallyn <serge.hallyn@ubuntu.com>
attach: implement remaining options of lxc_attach_set_environment

This patch implements the extra_env and extra_keep options of
lxc_attach_set_environment.

The Python implementation, the C container API and the lxc-attach
utility are able to utilize this feature; lxc-attach has gained two new
command line options for this.

Signed-off-by: Christian Seiler <christian@iwakd.de>
Acked-by: Serge E. Hallyn <serge.hallyn@ubuntu.com>