SR-IOV is a PCI device virtualization technology that allows a single PCI device to appear as multiple PCI devices on the physical PCI bus; the real physical device is called the Physical Function (PF) while the others are called Virtual Functions (VFs).

The purpose of this is for the hypervisor to directly assign one or more VFs to a virtual machine (VM) using Intel VT-D technology: the guest will be able to use the VF as any other directly assigned PCI device.

The common use case is an SR-IOV NIC. Assigning one or more VFs to a virtual machine allows the virtual machine to directly exploit the hardware without any mediation by the hypervisor. This means better performances and scalability since it has very little or no impact on dom0.

On the downside the VFs have no relations with VIFs and bridges in dom0 so they have to be configured separately and independently by the user. Also a virtual machine loses all the assigned VFs after being migrated to a different host. Thus, virtual machines that are migrated between hosts using XenMotion or High Availability (HA) failover require manual reinstatement of VFs on the new host.

Because Provisioning Services hosts tend to be network I/O bound rather than memory or CPU bound, they are ideal candidates to take advantage of this capability. The limitations on XenServer virtual machine failover and XenMotion are not significant in a Provisioning Services deployment because Provisioning Services implements its own HA and load balancing mechanisms.

Requirements
To make use of this capability, you must have a host server in which a SR-IOV capable network device is installed. The device tested for this article is the Intel® 82599 10 Gigabit Ethernet Controller.

Note: The setup procedure below requires that the 10 GigE NIC not be used as the management interface for the host. A second physical NIC must be installed on the system for that purpose.

Procedure
Use the following procedure to configure an SR-IOV-enabled Provisioning Services VM on XenServer 5.6, provided that your system meets the hardware and firmware requirements described above.

Enable iommu on the host
Edit /boot/extlinux.conf and add iommu=1 to the xen command line options. Regenerate the bootloader by executing the following within the XenServer host console:

extlinux /boot

Loading the pciback driver into dom0
This must be done every time the host boots. To do this automatically, add the following line to /etc/rc.local:

modprobe pciback

Reboot the host

Assign one or more VFs to a virtual machine.
Get a list of VFs in the system executing the lspci command in the XenServer console. You should see many (such as 120) devices like this: 07:10.0 Ethernet controller: Intel Corporation 82559 Ethernet Controller Virtual Function (rev 01)

Assign one of these VFs to the target virtual machine executing the following command:

xe vm-param-set other-config:pci=0/0000:07:10.0 uuid=uuid_of_the_VM

Substitute 07:10.0 in the example above with the pci bus address of the VF you want to assign.

Boot the virtual machine and install the correct VF driver in it. Once the driver is installed, Provisioning Services should be installed and configured as normal.

Source
Citrix Xen Knowledge Center

Read More...