xHCI Driver Implementation
2016-11-30 @ BitVisor Summit 5
Ake Koomsin

Agenda xHCI Overview
 Relationship between BitVisor’s host controller
drivers and USB drivers
 Implementation Walkthrough– Changes in BitVisor

Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

2

xHCI Overview
Host memory

MMIO
Capability
Registers
OperationalRegisters

Devicectx base
addr
array

Device ctx

Transfer
Ring

Device ctx

Transfer
Ring

Ext. Capability
Registers 1

Event Ring
segment table

Databuffer

Databuffer

Command ring

Runtime Registers

Databuffer

Event ring

Doorbell Registers
Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

3

xHCI Overview
Host memoryIndex

Point to0

Scratchpad1

Dev Slot ID 12Dev Slot ID 2

Device ctx

Transfer
Ring

Databuffer

Device ctx

Transfer
Ring

Databuffer

DatabufferCommand ring…
NDevice
ctx base
addr
array

Dev Slot ID N

Event Ring
segment table

Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

Event ring

4

xHCI Overview
Host memory
Slot Context
EP 0 BiDir Context
EP 1 OUT Context

Devicectx base
addr
array

Device ctx

Transfer
Ring

Device ctx

Transfer
Ring

EP 1 IN Context

Event Ring
segment tableData
bufferDatabufferCommand ring

EP 15 Out Context

DatabufferEvent ring

EP 15 IN Context
Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

5

xHCI Overview
Transfer Ring
Host memory
Segment 1

Transfer Ring
Segment 0

TRB

TRB

TRB

TRB

TRB

TRB

TRB

TRB

TRB

…

TRB

TRB

…

Link TRB

Transfer
Ring

TransferRing

DatabufferDatabufferDatabufferTRB

Link TRB

Event Ring
segment table

Event ring

TRB = Transfer Request Block

Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

6

xHCI Overview
Host memory
Entry 0311550Ring Segment Base Addr LOW
Ring Segment Base Addr HIGHTransfer
Ring

Ring Segment Size

TransferRing

DatabufferDatabufferDatabufferEvent Ring
segment tableCopyright© 2016 IGEL Co., Ltd. All Rights Reserved.

Event ring7

USB Device Initialization
Host

2. Port Status
Change EventxHCUSB Device
1. Attach

3. Enable Command
CMD Comp. EV

4. Addr Dev Command(BSR = 1)
CMD Comp. EV

Block SetAddress() Request
Enabled → Default

5. Prepare TRBs for
GetDescriptor()
6. Ring the Doorbell

Send the request

Transfer EV

ResponseCopyright© 2016 IGEL Co., Ltd. All Rights Reserved.

8

USB Device InitializationxHC

HostUSB Device

7. Addr Dev Command(BSR = 0)

SetAddress()

CMD Comp. EV

Default → Addressed

8. Prepare TRBs formore requests
9. Ring the Doorbell
Transfer EV

Send the request
Response

Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

9

USB Device InitializationxHC

HostUSB Device

10. Config Dev Command
CMD Comp. EV
11. Prepare TRBs for
SetConfigure()

Addressed → Configured

12. Ring the Doorbell

Send the request

Transfer EV

Response

14. Port Status
Change Event

13. Detach

15. Disable CommandCMD Comp. EV
Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

10

*HCI Driver & USB Driver
Guest OS

BitVisor
Determine URB completion+4 Process reply hooks

USB Driver
5
1 *HC Driver
2Intercept MMIOUSB Driver*HC Driver
3DeviceCreate URBs
+Process request hooks

Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

11

*HCI Driver & USB Driver *HCI Driver main responsibilities– Call usb_register_host() and initialize USB drivers– Construct USB Request Blocks (URBs) from data the guest
OS submits– Pass URBs to USB Drivers by calling usb_hook_process()– Host Controller state synchronization– Provide a method for data shadowing• Not only data buffer, but also data structures used by the hostcontroller

Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

12

Implementation
Shadow
Host memory

Intercept
MMIO
Capability
Registers
OperationalRegisters

Devicectx base
addr
array

Device ctx

Transfer
Ring

Device ctx

Transfer
Ring

Ext. Capability
Registers 1

Event Ring
segment tableData
bufferDatabufferCommand ring

Runtime Registers

Data
bufferEvent ring

Doorbell Registers
Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

13

Implementation Outline
 Intercept Operational Regs and Runtime Regs
– Write : Replace the guest physical addresses
with the host ones
– Read : Return the guest physical addresses Intercept Doorbell Regs
– Commands (Index 0)– USB Requests (Index 1 onwards)

 Synchronize states when there is an event

Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

14

URB Construction
Data transfer related TRB
31

150

Pointer or data lower bits
Pointer or data upper bits
Status
TypeType
1. Normal TRB
2. Setup Stage TRB
3. Data Stage TRB
4. Status Stage TRBICOHCISPC5. Isoch TRB
6. Link TRB
7. Event TRB

Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

15

URB Construction
Transfer Descriptor (TD)TD 0TD 1TD 2TD 3Link

CH = 1 CH = 1 CH = 0 CH = 0
ISP = 1 ISP = 1 IOC = 1 IOC = 1

CH = 1 CH = 0 CH = 0
ISP = 1 IOC = 1 IOC = 0TD 5TD 4Link
CH = 1 CH = 0
ISP = 0 IOC = 1CH = 1 CH = 1
ISP = 1CH = 0
IOC = 1Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

16

URB Construction
 One TD = One URB, with exceptions– USB Request TRBs, (3 TDs, 2 URBs in this example)

Setup

Data (optional)

StatusCH = 0

CH = 0

CH = 0– First TRB with not buffer

Link TRB

Normal TRB

Normal TRB

CH = 0

CH = 1

CH = 0Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

17

URB Construction
struct xhci_urb_private {u32 slot_id;
u32 ep_no; /* Start from 0 */struct usb_buffer_list *ub_tail;/* For URB completion */struct xhci_trb_meta *intr_tail;struct xhci_trb_meta *intr_list;

u32 start_idx;
u32 end_idx;
u32 next_td_idx;/* For TR cloning */struct xhci_trb_meta *link_trb_tail;struct xhci_trb_meta *link_trb_list;

u32 start_seg;
u32 end_seg;
u32 next_td_seg;

struct xhci_trb *not_success_ev_trb;
u8 start_toggle;
u8 end_toggle;
u8 next_td_toggle;};

u8 event_data_exist;

u32 total_buf_size;
Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

18

URB Construction
static u8
handle_slot_write(...){...
struct usb_request_block *g_urb = construct_gurbs(host,slot_id, ep_no);while (g_urb) {struct usb_request_block *h_urb = shadow_g_urb(g_urb);
u8 ret = usb_hook_process(host->usb_host, h_urb,USB_HOOK_REQUEST);
if (ret == USB_HOOK_DISCARD) {...
}
append_h_urb_to_ep(h_urb, host, slot_id, ep_no);...
g_urb = g_urb->link_next;}...
return 1;}

Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

19

URB Construction
 We don’t know the number of TR segments and TR
segment size at the beginning– We know only the base address of the first TR segment fromConfigure Device Command– Have to identify TR segment size and number of TR
segments while traversing– Treat the Link TRB as the end of TR segments

Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

20

URB Construction
struct usb_request_block *construct_gurbs(struct xhci_host *host, uint slot_id, uint ep_no)
{
...
do {
...
for (i_trb = current_idx; i_trb < n_trbs; i_trb++) {...
switch (type) {case XHCI_TRB_TYPE_LINK:
next_seg = get_next_seg(&g_trbs[i_trb], current_seg,
h_ep_tr, g_ep_tr);... /* Preparing for traversing the next segment */break;...
}...
}} while (!stop);...
}
Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

21

URB Construction
struct usb_request_block *construct_gurbs(struct xhci_host *host, uint slot_id, uint ep_no)
{
...
do {
...
for (i_trb = current_idx; i_trb < n_trbs; i_trb++) {...
if (type == XHCI_TRB_TYPE_LINK) {/* Stop current scanning to start at the next segment */break;}
if (i_trb + 1 == n_trbs && type != XHCI_TRB_TYPE_LINK) {g_ep_tr->tr_segs[current_seg].n_trbs *= 2;
h_ep_tr->tr_segs[current_seg].n_trbs *= 2;
g_ep_tr->current_idx = n_trbs;
h_ep_tr->current_idx = n_trbs;}}} while (!stop);...
}

Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

22

EP Ownership
Device Context

 Problems

Slot Context
EP 0 BiDir Context

Transfer
Ring

EP 1 OUT Context
EP 1 IN Context

Transfer
Ring– BitVisor needs toobtain device info
during devive
initialization– Not all USB drivers
need shadowing

EP 15 Out Context
EP 15 IN Context
Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

23

EP Ownership BitVisor need to obtain device info during device
initialization– It needs to own Endpoint 0 to do its job properly– For xHCI, there is no SetAddress() to intercept for USB
address• xHC does this for us after receiving the Set Address Command(BSR = 0)

– Current usb_device_init_monitor() implementation
relies on intercepting SetAddress() response• This assumption is not valid anymore– The last interrupter in Runtime Registers is reserved for
BitVisor• Polling the Event Rings for eventsCopyright© 2016 IGEL Co., Ltd. All Rights Reserved.

24

Changes in BitVisor (1) Changes in usb_device_init_monitor()– *HCI drivers are responsible for creating USB hooks fordevice monitoring from now• xHC does this for us after receiving
the Address Device Command (BSR = 0)

 Changes in struct usb_hook– Introduce before_callback, after_callback to structusb_hook– Introduce exec_once flag to struct usb_hook• If exec_once is set, the hook is removed after its execution

Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

25

EP0 Ownership If we put our TRBs into the shadow TR, it will makesynchronization much harder
Guest TR

Shadow TR

Guest TRB

Shadow TRB

Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

BitVisor TRB

26

EP0 Ownership Jump to a dedicated TR owned BitVisor only, andjump back the original position

Link

Link
BitVisor TR

Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

27

EP Ownership Not all USB drivers need shadowing– Let BitVisor owns all EPs at the beginning– A USB Driver must tell xHCI driver that it is going to own thedevice– If no USB driver claims the slot, put a Link TRB pointing to theguest Transfer Ring when the guest starts sending a requestthat is not EP 0
Guest TRB
EP 1 IN Context

Shadow TRB
LinkCopyright© 2016 IGEL Co., Ltd. All Rights Reserved.

28

Changes in BitVisor (2) Changes in struct usb_operations– Implement will_take_ctrl()• It takes a USB host object, and a USB device object as itsarguments

 USB Drivers– Need to explicitly tells *HCI drivers if it needs to take control
a device by calling usbhc->op->will_take_control()

Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

29

State Synchronization
 Two things to do– Device Context copyback• Both Slot Context and EP contexts• Replace the host pointer addresses with the guest ones

– Event Ring copyback

Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

30

State Synchronization
void
clone_er_trbs_to_guest(struct xhci_host *host){...
for (i = 0; i < host->usable_intrs; i++) {
...
do {
...
for (i_trb = start_idx; i_trb < n_trbs; i_trb++) {toggle = XHCI_TRB_GET_C(&h_trbs[i_trb]);
if (toggle == current_toggle) {handle_ev_trb(host, &h_trbs[i_trb]);} else {...
}}...
memcpy(&g_trbs[start_idx], &h_trbs[start_idx],(i_trb - start_idx + 1) * XHCI_TRB_NBYTES);} while (!stop);}}
Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

31

State Synchronization
static void
handle_ev_trb(struct xhci_host *host, struct xhci_trb *h_ev_trb){...
switch (type) {...
case XHCI_TRB_TYPE_TX_EV:
slot_id = XHCI_EV_GET_SLOT_ID(h_ev_trb);
ep_no= XHCI_EV_GET_EP_NO(h_ev_trb);
process_tx_ev(host, h_ev_trb);
if (ep_no != 0 &&host->slot_meta[slot_id].host_ctrl == HOST_CTRL_NO) {break;}
patch_tx_ev_trb(host, h_ev_trb);
break;...
}}
Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

32

State Synchronization
static void
process_tx_ev(struct xhci_host *host, struct xhci_trb *h_ev_trb){... /* Get h_urb list to find the target h_urb */... /* Process all h_urbs that exist before */u8 is_last_intr = check_last_intr(h_urb, h_ev_trb, &trb_code);...
if (is_last_intr) {if (h_urb->shadow->buffers) {int res = usb_hook_process(host->usb_host,h_urb, USB_HOOK_REPLY);
if (res == USB_HOOK_DISCARD) {...
}}remove_h_urb_head_from_ep(host, slot_id, ep_no);delete_urb_xhci(h_urb->shadow);delete_urb_xhci(h_urb);}}Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

33

State Synchronization
 When to synchronize status?
– It is possible to check for OPR.USBSTS access for events– Unfortunately, it is not a requirement. Linux writes to thisregister, but Windows occasionally reads it only
– We have to intercept interrupts

Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

34

Changes in BitVisor (3) Intercepting interrupts– Every external interrupts cause VM_EXIT events– Implement exint_pass_intr_register_callback()– Implement pci_register_intr_callback()• It calls exint_pass_intr_register_callback() internally– xHCI driver calls pci_register_intr_callback() forregistering its state synchronization callback

Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

35

Summary xHCI Overview– Registers, data structures, and operational model Relationship between BitVisor’s host controller
drivers and USB Drivers– *HCI driver responsibilities

 xHCI Driver implementation– Implementation walkthrough– Changes in BitVisor

Copyright© 2016 IGEL Co., Ltd. All Rights Reserved.

36