A brief overview of DRM/KMS and its status inNetBSD
Taylor ‘Riastradh’ Campbell
campbell@mumble.net
riastradh@NetBSD.org

AsiaBSDcon 2015
Tokyo, Japan
March 13, 2015

No, not that DRM!
I DRM: Direct rendering manager: Linux kernel interface forcommanding GPU to render directly into framebuffer fordisplay.
I Originally, DRM was only a kernel interface for mapping

graphics card MMIO registers and waiting for vertical blankinterrupts.
I Actual driver for display lived in userland: used DRM todisable kernel’s idea of VGA console and grant exclusive
access to display registers to X server, peeked and poked themin userland to detect and configure displays.
I Userland used legacy /dev/agp device to allocate physicalmemory for graphics and program it into the GPU’s pagetables.

DRM/KMS: DRM with a real kernel display driver

I Maybe userland shouldn’t be mapping the device’s MMIO

registers, handling mode-setting, etc.: ‘user mode-setting’, orUMS.
I Would be nice if kernel could suspend/resume display without

X’s help.
I DRM/KMS: DRM with kernel mode-setting.

GEM and TTM: Graphics buffer management

I GEM: Graphics Extent Manager
I TTM: Texture and Tiling Manager
I Fancy names for two different sets of ioctls to manage

swappable buffers shared by CPU and GPU.

DRM portability

I DRM implementation maintained in Linux.
I Used to be a coordinated porting effort to BSDs.
I Lost coordination in switch from UMS to KMS.
I New ports to *BSD all different now!
I NetBSD: shims to make most Linux code run unmodified andupdates less painful.
I FreeBSD: modify all the Linux code, including indentation.
I OpenBSD and DragonflyBSD: somewhere in the middle.

Problems

I Userland can still wedge GPU.
I Linux kernel code is very large:% wc -l drm/*.c29149 total% wc -l drm/i915/*.c76242 total% wc -l drm/nouveau/**/*.c95675 total% wc -l drm/radeon/*.c152315 total
I . . . and I made some stupid mistakes porting it.

Status

I Intel graphics: works, minor bugs in display detection on somedevices, minor rendering glitches on some devices.
I Radeon: works.
I Nouveau: compiles but does not work yet.
I Everything is much better as of this month after I fixed three

stupid bugs I caused ages ago. . .

Bug 1: Timed waits: Linux code
I Linux has no easy API for interlocked waits.

unsigned long start = jiffies;
unsigned long end = start + timeout
unsigned long now;
DEFINE_WAIT(wait);
int ret;for (;;) {prepare_to_wait(&dev->waitq, &wait,TASK_INTERRUPTIBLE);
if (signal_pending(current)) {ret = -ERESTARTSYS;
break;}...

Bug 1: Timed waits: Linux code...
now = jiffies;
if (now > end) {ret = (CONDITION) ? 1 : 0;
break;}
if (CONDITION) {ret = MAX(end - now, 1);
break;}...

Bug 1: Timed waits: Linux code...
ret = schedule_timeout(timeout);
if (ret < 0)
break;
timeout = ret;}finish_wait(&dev->waitq, &wait);
return ret;
I Where’s the lock to read dev->done excluding interrupts?
I You’re on your own.
I Every driver does it differently, usually with a complicated(read: wrong) dance involving atomics.

Bug 1: Timed waits: Linux code simplified

I Linux has a collection of macros to do this for you:ret = wait_event(dev->waitq, dev->done)
ret = wait_event_interruptible(dev->waitq,dev->done);ret = wait_event_timeout(dev->waitq, dev->done,timeout);
I Return negative error on interrupt.
I Return zero on success. . . if no timeout.
I Return positive on success if there is a timeout.
I Return zero on timeout.
I (What about lock for dev->done? Still on your own.)

Bug 1: Timed waits: Linux DRM code

I Old DRM code from last decade used a portability macro

DRM WAIT ON:
DRM_WAIT_ON(ret, dev->waitq, timeout, dev->done);I Return negative error on interrupt.
I Return negative error -ETIME on timeout.
I Return zero on success.
I (Also: poll every tick, just for good measure.)I (What about lock for dev->done? Still on your own.)

Bug 1: Timed waits: NetBSD code

I NetBSD has:while (!dev->done) {error = cv_timedwait_sig(&dev->waitq,&dev->lock, timeout);
if (error)
return error;
now = hardclock_ticks;
timeout -= now - start;
start = now;}

Bug 1: Timed waits: NetBSD code

I No non-interlocked timed waits: no dances with atomics andno race conditions.
I Required putting in device interrupt spin locks whereappropriate, since Linux doesn’t have them.
I Return EINTR/ERESTART on interrupt.
I Return EWOULDBLOCK on timeout.
I Return zero on success.

Bug 1: Timed waits

I I focussed on getting locks correct for interlocked waits.
I Didn’t pay enough attention to the return codes.
I Totally mixed them all up.
I Waits for i2c commands, graphics commands – always timedout or returned early.
I Sometimes worked by accident, hard to diagnose.
I Oops.

Bug 2: Cache-flushing needs memory barriers

I Intel CLFLUSH instruction flushes cache lines.

size_t clflush_size =cpu_info_primary.ci_cflush_lsize;
vaddr_t p;
for (p = start; p < end; p += clflush_size)
x86_clflush(p);I That should do it, right?

Bug 2: Cache-flushing needs memory barriers

I Intel CLFLUSH instruction flushes cache lines.

size_t clflush_size =cpu_info_primary.ci_cflush_lsize;
vaddr_t p;
x86_mfence();for (p = start; p < end; p += clflush_size)
x86_clflush(p);x86_mfence();
I Except it is not instruction-ordered. It is ordered only byMFENCE. I forgot MFENCE. Oops.

Bug 3: Cacheability flags

I Entries in the GPU page table, or graphics translation table‘GTT’, have cacheability flags.
I Everything should correct – but slow – if we disable caching,right?

Bug 3: Cacheability flags

I Held off turning on these bits for months while trying to findthe source of unusable rendering glitches.
I Figured turning on caching would make things worse.
I Turned on the bits. Everything worked.
I Oops.