Xen seems to be gaining speed these days and has a lot of useful features for those who want to resale or otherwise split a single box. Now that you have your Xen system set up you may be interested in going farther with bandwidth limiting.

The hardest part of setting up bandwidth limiting is understanding the traffic control system under Linux. This mainly revolves around the tc command.

The first thing you will want to do is select a queue discipline. Although you can select from a number of disciplines I picked HTB for the following instructions because it seems to be the simplest to set up. All of the following is done on Xen0.

1. The first thing you will need to do is find the name of your real ethernet device. This seems to change depending on what version of Xen you are running. For my setup it was peth0 and I was able to find it by looking at dmesg right after the system booted.
2. After finding the name of your ethernet device you will need to set up a default queuing discipline that will catch anything that doesn't hit a child rule. Here we set the handle to 1 and a class sub id of 99.

   tc qdisc add dev peth0 root handle 1: htb default 99
   

3. Next we define a default rate that will be used as a total for all child rates as well as anything that doesn't fall into a child bucket. In this case I'm setting the total rate to 20mbps with a burst of 15k.

   tc class add dev peth0 parent 1: classid 1:1 htb rate 20mbps burst 15k
   

   Setting a burst lets small amounts of traffic go faster than the normal rate. The burst is also shared with the children so make sure to set it higher than any one child. Also note that parent 1: references the parents classid that we created above.

4. Now that we have our default class and rate set up we set up child classes and rates for each node. Here I set up classes for 2 XenU nodes and the Xen0 node.

   tc class add dev peth0 parent 1:1 classid 1:13 htb rate 5mbps burst 15k
   tc class add dev peth0 parent 1:1 classid 1:14 htb rate 10mbps burst 15k
   tc class add dev peth0 parent 1:1 classid 1:99 htb rate 5mbps burst 15k
   

   Note that the parent classid is referenced here as 1:1 that we created above. We also assign each bucket its own unique classid.

5. Now we need to determine who gets serviced in what order. In this example I use sfq for each class so that each class should get equal time as traffic is coming in.

   tc qdisc add dev peth0 parent 1:13 handle 13: sfq perturb 10
   tc qdisc add dev peth0 parent 1:14 handle 14: sfq perturb 10
   tc qdisc add dev peth0 parent 1:99 handle 22: sfq perturb 10
   

6. Now the final step is to attach the defined classes to the routing system. This is done by using iptables and the given classid from the class setup step.

   iptables -t mangle -A POSTROUTING -p tcp -s 192.168.1.103 -j CLASSIFY --set-class 1:13
   iptables -t mangle -A POSTROUTING -p tcp -s 192.168.1.104 -j CLASSIFY --set-class 1:14
   iptables -t mangle -A POSTROUTING -p tcp -s 192.168.1.111 -j CLASSIFY --set-class 1:21
   

Tags: xen, linux