Porting Virtio Drivers from NetBSD to DragonFly BSD to speed up DragonFly BSD as a KVM guest

Porting Virtio Drivers from NetBSD to DragonFly BSD to speed up DragonFly BSD as a KVM guest

Student: Stéphanie OUILLON

(arroway on Freenode)

E-mail: steph.ouillon@gmail.com

Télécom SudParis

Evry, France

Project Description and Goals:

With “full virtualization”, the hypervisor has to emulate device hardware, which produces reduced performances. With paravirtualization, the guest operating system and the hypervisor can work cooperatively to provide a more efficient emulation: the guest must include front-end drivers for the particular device emulation and the hypervisor implements the back-end drivers for it. Virtio comes in between: it is an abstraction layer between the guest and the hypervisor. It provides a standardized interface for a set of devices emulated by the hypervisor. It has two main advantages:

• it provides a common framework for I/O virtualization;

• it enables better performances for network, disk operations and memory reclamation from the guest  (results seem to depend on the guest machine OS and the virtualization software used).

Virtio was chosen to be the main platform for IO virtualisation in Linux hypervisor KVM. Three devices are currently available:

• network device driver;

• block device driver;

• ballooning device driver.

The ballooning device will be used when the hypervisor needs to reclaim virtual machine memory. A balloon module runs within the guest, which allocate pages and pin them in physical memory. The machine pages backing this memory can then be reclaimed. If we inflate the size of the balloon, memory pressure increases. The guest will page out to its virtual disk when memory is limited. If we deflate the balloon, memory pressure decreases and it frees guest memory.

http://www.usenix.org/events/osdi02/tech/full_papers/waldspurger/waldspurger_html/node6.html

These device drivers has been ported on NetBSD by Makoto Minoura under BSD license (code is available there : http://www.minoura.org/~minoura/virtio-100605/). The drivers have been partly ported from NetBSD to DragonFly BSD :

• a virtio-ring driver has been ported;

• a block device driver has been ported and tested;

• a network device driver exists as a skeleton of driver, but doesn’t create any working network interface nor connection;

• a ballooning driver is to be completely ported from NetBSD.  

This project aims at implementing those drivers as modules to be loaded in the kernel and doing benchmarks running DragonFly BSD on KVM to prove that virtio does improve performances. The drivers will have to respect specifications as described in Rusty Russell’s draft about Virtio PCI Card Specification:http://ozlabs.org/~rusty/virtio-spec/virtio-spec.pdf. Broadly speaking, that means that virtio devices:

• should use normal PCI mechanisms of interrupts and DMA;

• consist of separated rings of descriptors for input and output;

• are coded without making any supposition on the environment where they  operate;

• contain features bits which are acknowledged by the guest during device setup and allow forwards and backwards compatibility.

Project Deliverables and Deadlines

 

• July 15th  (midterm evaluation)

• virtio block device driver code, benchmarks and  documentation

• virtio network driver code, benchmarks and documentation

• August 15th (‘pencil down’ date)

• virtio ballooning driver code, benchmarks and documentation

• August 22nd (final evaluation)

• August 29th (final results)

 

 

Proposed Schedule

Timezone : Paris - CEST (UTC+2:00)

Note: Summer break only begins on the 18th of June in France. So I plan to do most of the preparation and documentation work before coding during the community bonding period. I have already marked papers and documentation to study during this period.

I have adapted my timeline with my scholar schedule.

Before April 25th

• Install DragonFly BSD on KVM (currently, I have installed it on VirtualBox, which doesn’t fully support virtio block device and ballooning device drivers)

• configure libvirt to use virtio with KVM guests, eventually test it with a NetBSD guest: http://wiki.libvirt.org/page/Virtio

• Get used to PCI devices driver (e.g. porting sample driver from FreeBSD documentation http://www.freebsd.org/doc/en/books/arch-handbook/pci.html)

• Read documentation about bus_dma

• Study virtio specifications, be comfortable with the notions of virtqueue, device configuration, virtqueue configuration, etc.

• Get used with DragonFly kernel and NetBSD kernel specificities to define important differences in the code of the drivers to be ported. First look at  device driver I/O routines (uio), resource management functions (rman), kernel buffer I/O schemes used in DragonFly VM system, mbuf, etc.

   

1. April 25th – May 22nd

• Be comfortable with virtio block devices specification.

• If necessary, clean redundant functions between blk.c (block device driver code) and virtio.c (virtio ring) and make sure it still negotiates features with virtio.c

  Unit tests on the block device driver.

  
  

 2. May 23rd – May 29th      

  

• Do again benchmarks to insure that there is, at minimal, no performance regressions. If so, review code. (note : devstat tests are included in the code of the ported block device driver : do again DD read/write performance). Do benchmarks while kernel compile.

 

3. May 30th – June 5th

• Finish testing and modifying code if necessary.

• Ensure that code is clean and well commented.

• Write documentation on the block device driver.

• Submit code and results to community review.

 

4. June 6th – June 12th

o Be comfortable with virtio network devices specification.

o Test probe and attach functions already written for virtio network device driver.

o Write interface functions with ifnet for virtio network device drive (create a network interface).

 

5. June 13th – June 19th

o Finish writing interface functions if not.

o Receive implementation functions for virtio network device driver (allocate and initialize a mbuf to store network packets).

 

6. June 20th – June 26th

o Transmission implementation functions for virtio network device driver (dequeue and free mbuf).

o Write functions dealing with promiscuous mode.

o Unit tests for virtio network device driver.

7. June 27th – July 3rd

o Keep on running tests if necessary on the virtio network device driver.

o Benchmarks with iperf to insure that, at minimal, there is no performance regressions. If so, review code.

o Ensure that virtio network device driver code is clean and well commented.

 

8. July 4th – July 10th

o Write documentation on the network device driver.

o Submit network device driver code and documentation to community review.

o Prepare midterm submission by improving network device and block device drivers and documentation thanks to the community feedback.

 

9. July 11th – July 17th - Mid-term submission

o Ensure I have supplied all code and documentation to my mentor for the mid-term evaluation.

o Be comfortable with virtio ballooning devices specification.

Focus on physical memory allocation specifications in DragonFly BSD.

o Write probe and  attach functions for virtio ballooning device driver.

 

10. July 18th – July 24th

o Write configuration change callback function : kthread wakes up and sleeps.

o Write functions to implement the inflation of the amount of physical memory available in the balloon (allocate).

o Write functions to implement the deflation of the amount of physical memory available in the balloon (free allocated memory during the inflation).

 

11. July 25th – July 31st

o Unit tests for virtio ballooning device driver.

o Study how to correctly benchmark the virtio ballooning device driver.

o Benchmarks : kernel compile, eventually running databases.

 

12. August 1st – August 7th

o Keep on running benchmarks for the virtio ballooning device driver if necessary.

o Ensure that the virtio ballooning device driver code is clean and well-commented.

o Write documentation on virtio ballooning device driver.

 

13. August 8th – August 14th

o Submit network device driver code and documentation to community review.

o Prepare the final review by improving virtio ballooning device driver and documentation thanks to the community feedback.

14. August 15th - August 21st - Firm ‘pencil down’

o Ensure all code and documentation are available for the final review.

Possible Future Improvements:

 

o Depending on benchmark results, improve performances.

o Do benchmarks with DragonFly BSD running as a guest on other virtualization softwares supporting virtio (such as VMWare)

o Study the possibility and relevance to write a virtio console driver and a virtio entropy device driver.

Additional resources needed for the project:

 

Since at the moment, I have a DragonFly BSD virtual machine running on VirtualBox, I plan to install during the period left before coding:

o A DragonFly BSD virtual machine on Qemu to run code and tests on my computer.

o DragonFly BSD virtual machines on KVM a server.

References related on virtio drivers:

 

o NetBSD virtio drivers code: http://www.minoura.org/~minoura/virtio-100605/

o Rusty Russell’s draft about Virtio PCI Card Specification:http://ozlabs.org/~rusty/virtio-spec/virtio-spec.pdf

o Gitorious page with some information about  the ported block device driver and benchmarks results:http://gitorious.org/virtio-drivers/pages/Home

o Partly ported code to DragonFly BSD sent on the mailing list kernel@crater.dragonflybsd.org by Pratyush Kshirsagar

o Documention about virtio on KVM:http://wiki.libvirt.org/page/Virtio

Previous work on virtio device drivers on DragonFly BSD :

 

o From the code partly ported from NetBSD that was sent via the mailing list, I load the virtio.ko  and virtio-net.ko modules in a DragonFly BSD guest machine running on VirtualBox. I faced problems with the VirtualBox configuration.

o After talking on the mailing list, I was able to check the results that were expected for the probe and attach functions (checking dmesg).

o I begun to read code to understand the relationship between modules and surround functions to be written, comparing with NetBSD code.

 

 

Personnal:

 

I am a first year student at the Engineering High School of Networking and Telecommunication Telecom SudParis (http://www.telecom-sudparis.eu/en_accueil.html). I am in BAC+3, two years before engineering diploma which is equivalent to a Master degree.

 

I am interested in many various fields: networking, code security (security flaws in C, shellcodes), OS internal architectures (kernel, memory management, C/assembly interface, etc) and all kind of topics I can hear of by attending to conferences (27c3, Parinux).

 

I am an active member of MiNET, the  association managing the campus network of my school. Here are some activities: network maintenance and monitoring (wired and wireless network, VoIP), virtualization with Xen and KVM on the servers, clustering with Proxmox managing projects like the installation of a wireless network, helping with every kind of computer problems that students can face on the campus...

 

Only open source solutions and software are used since we promote open source on the campus (e.g. we organized a conference inviting the Debian Project Leader S. Zacchiroli in February).

Background Experience:

 

o During two years of intensive preparation to competitive examination to Engineering High Schools, I studied programming in Caml and Mapple for mathematical and physical applications: recursion, automates, trees, sorts...

o  I lead an eight month individual project about Simulation of Surface Growth: the goal was to simulate surfaces following different physical models in Caml and then study and compare their growth evolution and properties (data were graphically exploited with Mapple). I did a presentation of my work during the competitive examination to Engineering High Schools.

o  I am part of scholar team project of programming in C: the goal is to create an interface to communicate with a LED Display Board via a serial port and send it the messages to be displayed. We communicate with the board through a machine constantly connected, via a web interface or directly via ssh from a terminal.

 

Language Skill Set:

 

o   Very good knowledge of C

o   Good knowledge of Caml

o   Basic knowledge of x86 Assembly language

o   Basic knowledge of PHP