OVS – OVS stands
for Open Virtual Switch, which is an interface between physical network
port and virtual machine of a host. It act as an bridge for communicating between
VMs inside or outside. By the help of OVS we can manage security ( VLAN
isolation, traffic filtering) , monitoring the network for the VMs inside a
host.
Below we have some discussion about OVS work flow.
Isolating
VM Traffic Using VLANs
The goal of
this configuration cookbook is to isolate VM traffic using VLANs.
Environment
This
walkthrough assumes the environment is configured as described in the following
sections.
Two
Physical Networks
• Data Network: Ethernet network for VM
data traffic, which will carry VLAN-tagged traffic between VMs. Your physical
switch(es) must be capable of forwarding VLAN-tagged traffic and the physical
switch ports should operate as VLAN trunks. (Usually this is the default
behavior. Configuring your physical switching hardware is beyond the scope of
this document.)
• Management Network: This network is not
strictly required, but it is a simple way to give the physical host an IP
address for remote access, since an IP address cannot be assigned directly to
eth0 (more on that in a moment).
Two
Physical Hosts
The
environment assumes the use of two hosts, named Host1 and Host2. Both hosts are
running Open vSwitch. Each host has two NICs, eth0 and eth1, which are
configured as follows:
• eth0 is connected to the Data Network.
No IP address is assigned to eth0.
• eth1 is connected to the Management
Network (if necessary). eth1 has an IP address that is used to reach the
physical host for management.
Four
Virtual Machines
Each host
will run two virtual machines (VMs). VM1 and VM2 are running on Host1, while
VM3 and VM4 are running on Host2.
Each VM has
a single interface that appears as a Linux device (e.g., "tap0") on
the physical host. (Note: for Xen/XenServer, VM interfaces appears as Linux
devices with names like "vif1.0". Other Linux systems may present
these interfaces as "vnet0", "vnet1", etc.)
The
following diagram graphically illustrates the environment assumed by this
configuration cookbook.
Configuration
Steps
Perform the
following configuration on Host 1:
1 Create an OVS bridge:
ovs-vsctl
add-br br0
2 Add eth0 to the bridge (by default, all
OVS ports are VLAN trunks, so eth0 will pass all VLANs):
ovs-vsctl
add-port br0 eth0
Note
that when you add eth0 to the OVS bridge, any IP addresses that might have been
assigned to eth0 stop working. IP address assigned to eth0 should be migrated
to a different interface before adding eth0 to the OVS bridge. This is the
reason for the separate management connection via eth1.
3 Add VM1 as an "access port"
on VLAN 100. This means that traffic coming into OVS from VM1 will be untagged
and considered part of VLAN 100:
ovs-vsctl add-port br0 tap0 tag=100
4 Add VM2 on VLAN 200.
ovs-vsctl
add-port br0 tap1 tag=200
Repeat these
steps on Host 2:
1 Setup a bridge with eth0 as a VLAN
trunk:
ovs-vsctl add-br br0 ovs-vsctl add-port br0 eth0
2 Add VM3 to VLAN 100:
ovs-vsctl
add-port br0 tap0 tag=100
3 Add VM4 to VLAN 200:
ovs-vsctl
add-port br0 tap1 tag=200
Testing
Pings from
VM1 to VM3 should succeed, as these two VMs are on the same VLAN.
Pings from
VM2 to VM4 should also succeed, since these VMs are also on the same VLAN as
each other.
Pings from
VM1/VM3 to VM2/VM4 should not succeed, as these VMs are on different VLANs. If
you have a router configured to forward between the VLANs, then pings will
work, but packets arriving at VM3 should have the source MAC address of the
router, not of VM1.
Monitoring VM Traffic Using sFlow
The goal of this configuration cookbook is to monitor traffic
sent between two VMs on the same physical host using an sFlow collector.
Environment
This configuration cookbook assumes an environment configured as
described in the sections below.
Two Physical Networks
•
Data Network: This Ethernet network is for VM data traffic. For
experimentation, this physical network is optional. You can instead connect all
VMs to a bridge that is not connected to a physical interface.
•
Management Network: This network must exist, as it is used to
send sFlow data from the agent to the remote collector.
Two Physical Hosts
The cookbook assumes the use of two physical hosts, referred to
here as Host1 and Monitoring Host.
Host1 is a hypervisor that runs Open vSwitch and has two NICs:
•
eth0 is connected to the Data Network. No IP address can be
assigned on eth0 because it is part of an OVS bridge.
•
eth1 is connected to the Management Network. eth1 has an IP
address for management traffic, including sFlow.
Monitoring Host can be any computer that can run the sFlow
collector. For this cookbook entry, we use sFlowTrend, a free
sFlow collector that is a simple cross-platform Java download. Other sFlow
collectors should work equally well. Monitoring Host has a single NIC, eth0,
that is connected to the Management Network. eth0 has an IP adress that can
reach eth1 on Host1.
Two Virtual Machines
This cookbook entry uses two virtual machines (VM1 and VM2)
running on Host1.
Each VM has a single interface that appears as a Linux device
(e.g., "tap0") on the physical host. (Note: for Xen/XenServer, VM
interfaces appears as Linux devices with names like "vif1.0". Other
Linux systems may present these interfaces as "vnet0",
"vnet1", etc.)
This
diagram depicts the environment used by this configuration cookbook entry:
Configuration Steps
•
On Host1, define the following configuration values in your
shell environment.
COLLECTOR_IP=10.0.0.1
COLLECTOR_PORT=6343
AGENT_IP=eth1
HEADER_BYTES=128
SAMPLING_N=64
POLLING_SECS=10
Port 6343 (COLLECTOR_PORT) is the default port number for sFlowTrend. If you are
using an sFlow collector other than sFlowTrend, set this value to the
appropriate port for your particular collector. Set your own IP address for the
collector in the place of 10.0.0.1 (COLLECTOR_IP). Setting the AGENT_IP value to eth1 indicates that the sFlow agent should send
traffic from eth1's IP address. The other values indicate settings regarding the
frequency and type of packet sampling that sFlow should perform.
•
Still on Host1, run the following command to create an sFlow
configuration and attach it to bridge br0:
ovs-vsctl --
--id=@sflow create sflow agent=${AGENT_IP} \
target=\"${COLLECTOR_IP}:${COLLECTOR_PORT}\" header=${HEADER_BYTES}
\
sampling=${SAMPLING_N} polling=${POLLING_SECS} -- set bridge br0
sflow=@sflow
Make
note of the UUID that is returned by this command; this value is necessary to
remove the sFlow configuration.
•
On Monitoring Host, go to the sFlowTrend homepage
and click "Install" in the upper right-hand corner. If you have Java
installed, this will download and start the sFlowTrend application. Once
sFlowTrend is running, the light in the lower right-hand corner of the
sFlowTrend application should blink green to indicate that the collector is
receiving traffic.
The sFlow configuration is now complete, and sFlowTrend on
Monitoring Host should be receiving sFlow data from OVS on Host1.
To configure sFlow on additional bridges, just replace
"br0" in the above command with a different bridge name.
To remove sFlow configuration from a bridge (in this case,
"br0"), run this command, where "sFlow UUID" is the UUID
returned by the command used to set the sFlow configuration initially:
ovs-vsctl remove bridge br0 sflow
<sFlow UUID>
To see all current sets of sFlow configuration parameters, run:
ovs-vsctl list sflow
Troubleshooting
If sFlow data isn't being collected and displayed by sFlowTrend,
check the following items:
•
Make sure the VMs are sending/receiving network traffic over
bridge br0, preferably to multiple other hosts and using a variety of
protocols.
•
To confirm that the agent is sending traffic, check that running
the following command shows that the agent on the physical server is sending
traffic to the collector IP address (change the port below to match the port
your collector is using):
tcpdump -ni eth1 udp port
6343
•
If no traffic is being sent, there is a problem with the
configuration of OVS. If traffic is being sent but nothing is visible in the
sFlowTrend user interface, this may indicate a configuration problem with the
collector.
Check
to make sure the host running the collector (Monitoring Host) does not have a
firewall that would prevent UDP port 6343 from reaching the collector.
Openvswitch Native tunneling configuration guide
This guide is for configuring userspace tunneling in Open
vSwitch. The traditional OVS with kernel datapath uses kernel module to perform
the tunneling, however this setup performs all the tunneling operations purely
in the userspace. This way userspace-tunneling is platform independent.
This setup needs an additional bridge called “br-phy1” when
compared to the kernel based OVS. The purpose of this bridge is to make
available the kernel network stack for routing and arp resolution. The datapath
needs to look-up the routing table and arp table to prepare the tunnel header
and xmit data to the output port.
The tunneling setup is found as below:
+--------------+
| VM-0
| 192.168.1.1/24
+--------------+
(vm_port0)
|
|
|
|
|
+--------------+
|
br-int | 192.168.1.2/24
+--------------+ +--------------+
|
vxlan0 | | vxlan0
|
+--------------+ +--------------+
| |
| |
| |
| |
| |
| |
172.168.1.1/24 |
+--------------+ |
|
br-phy1 | 172.168.1.2/24
+--------------+ +---------------+
| dpdk0/eth1 |==========================| eth1
|
+--------------+ +---------------+
Host A with OVS. Remote host.
Prerequisites
The host must be pre-installed with OVS-DPDK, QEMU and virtual
machine images. Please refer the installation guide of relevant modules for
these instructions. ovs-vswitchd and OVSDB server must be up and running before
proceeding to any of the configuration steps.
This setup guide covers only the required steps for setting up
VxLAN userspace tunneling. The same approach can be used for any other
tunneling protocols, by specifying the appropriate tunneling protocol type.
“eth1” interface on Host A is managed by kernel driver by
default. The interface must be attached to DPDK driver first, to perform the
tunneling over DPDK interface. Use the following script to bind “eth1”
interface to DPDK driver.
HOST-A$ ./tools/dpdk_nic_bind.py –bind=igb_uio eth1
“dpdk_nic_bind.py” is a utility script to bind/unbind interfaces
to DPDK/Linux kernel.More details on bind/unbind can be found at following link
Note: This configuration guide is for setting up the VxLAN
tunneling on one host (local host). The same steps have to be performed on the
remote host as well for setting up a VM<->VM VxLAN tunnel setup. The only
difference for setting up the userspace-tunneling in remote node is, VM and
VxLAN tunnel ip addresses are different.
Configuration steps
•
Create the “br-int” bridge as below,
HOST-A$ ovs-vsctl
–may-exist add-br br-int – set Bridge br-int datapath_type=netdev –
br-set-external-id br-int bridge-id br-int – set bridge br-int
fail-mode=standalone
•
Add the VM interface to the “br-int”.
HOST-A$ ovs-vsctl
add-port br-int vm_port0 – set Interface vm_port0 type=dpdkvhostuser
[“vm_port0”
is the vhost-user interface name for the VM0. This interface name must be used
while qemu starting the virtual machine.]
•
Start the VM(VM-0) using vhost-user network backend. The
vhost-user interface name is “vm_port0”.
•
Set the Ip address 192.168.1.1 to the VM interface. Run the
following command inside the VM to set the Ip address.
VM-0$ ip addr add
192.168.1.1/24 dev eth0
[“eth0” is the interface inside VM. It’s
possible to set the ip address using “ifconfig” command as “ifconfig eth0
192.168.1.1/24”. Please use either one command to set the ip address.]
•
Attach the VxLAN tunnel interface to the “br-int“ bridge.
HOST-A$
ovs-vsctl add-port br-int vxlan0 – set interface vxlan0 type=vxlan
options:remote_ip=172.168.1.2
[“172.168.1.2” is the remote tunnel
end point address, On remote host this will be 172.168.1.1.]
•
Create the “br-phy1” bridge for attaching the physical
interface.
HOST-A$ ovs-vsctl –may-exist add-br br-phy1 – set Bridge
br-phy1 datapath_type=netdev – br-set-external-id br-phy1 bridge-id br-phy1 –
set bridge br-phy1 fail-mode=standalone other_config:hwaddr=”<mac address of
eth1 interface>“
[“eth1” is the physical interface on the host,
Assign mac address of “eth1” to the “br-phy1”.]
•
The physical port “eth1” can operate either as a kernel network
interface or a DPDK interface. Depending on the operating mode, attach “eth1”
to the “br-phy1” bridge as follows.
Use step-7 if “eth1” is managed by kernel
driver. Please follow step-8 rather than this step in case “eth1” is a DPDK
Interface.This step will cause to loose the connectivity through “eth1” (refer
configuration problems in FAQ for more details) for a while. The
connectivity can be restored by moving the IP address to the “br-phy1” internal
interface. The following command-set will do that,
HOST-A$ ovs-vsctl
–timeout 10 add-port br-phy1 eth1
HOST-A$ ip addr add
172.168.1.1/24 dev br-phy1
HOST-A$ ip link set br-phy1 up
HOST-A$
ip addr flush dev eth1 2>/dev/null
HOST-A$ ip link set eth1 up
HOST-A$
iptables –F
•
Steps for attaching “eth1” to “br-phy1” is slightly different in
case “eth1” is a DPDK interface. DPDK interfaces are not managed by the kernel,
so the port details are not visible on any “ip” commands.
HOST-A$
ovs-vsctl –timeout 10 add-port br-phy1 dpdk0 – set Interface dpdk0 type=dpdk
HOST-A$
ip addr add 172.168.1.1/24 dev br-phy1
HOST-A$ ip link set
br-phy1 up
HOST-A$ iptables –F
[“eth1” is mapped to
DPDK port “dpdk0”. DPDK driver assign port names that starts with “dpdk”]
Now the traffic from “VM0” will be VxLAN encapsulated and send
out over eth1/dpdk0 interface.
TCPDUMP doesn’t work on DPDK interfaces(eth1) as its no longer
managed by the kernel.
Debugging
•
Verify the created tunnel port details.
HOST-A$ ovs-appctl
tnl/ports/show
•
As mentioned before, it’s necessary that the vswitch should have
the arp-table entries to do tunneling at userspace. The learned arp entries in
the vswitch can be verified by,
HOST-A$ ovs-appctl tnl/arp/show
the
arp entries can be flushed using,
HOST-A$ ovs-appctl tnl/arp/flush
To
set a specific arp entry,
HOST-A$ ovs-appctl tnl/arp/set <bridge>
<ip addr> <mac addr>
In the above test setup, the following
arp entries have to be set in case they are not present.
HOST-A$
ovs-appctl tnl/arp/set br-int 172.168.1.1 <mac addr of br-phy1>
HOST-A$
ovs-appctl tnl/arp/set br-phy1 172.168.1.2 <mac addr of remote TEP>
•
Similarly OVS uses routing table entries to xmit the tunnel
packets. The vswitch routing entries can be verified by
HOST-A$
ovs-appctl ovs/route/show
To delete the route entries.
HOST-A$
ovs-appctl ovs/route/del
•
To lookup a route entry for a destination please use,
HOST-A$
ovs-appctl ovs/route/lookup
•
In case the route entries are missing for tunnel packet
forwarding, it can be added using the following command,
HOST-A$
ovs-appctl ovs/route/add 172.168.1.1/24 br-phy1
This command instructs
OVS to route all traffic destined for “172.168.1.2” to bridge “br-phy1”
•
To verify the range of tunneling source ports,
HOST-A$
ovs-appctl tnl/egress_port_range
•
To see the configured datapath ports,
HOST-A$ ovs-appctl
dpif/show
To
verify the flows programmed on the OVS datapath,
HOST-A$ ovs-appctl
dpctl/dump-flows
[This command shows how OVS forwards the packets in
datapath.]
No comments:
Post a Comment