Open vSwitch Network Monitoring Using sFlow and sFlow-RT

This post describes how to use Open vSwitch and sFlow collector for monitoring network traffic and for building VM-to-VM traffic matrix. The instruction aims at traffic monitoring for VMs connected to a software Bridge and is described based on an OpenStack setup environment (VMs are connected to br-int on each compute host).

Configuration Steps

1. On each physical host that the software bridge (Open vSwitch) is installed, define the following environment variables in the shell:


COLLECTOR_IP is the IP address of the host which is responsible for collecting monitoring data and sFlow-RT is installed. Port 6343 (COLLECTOR_PORT) is the default port number for sFlow-RT. If you are using the sFlow collector other than sFlow-RT, the appropriate port for that particular collector should be set. Setting the AGENT_IP value to eno1 indicates that the sFlow agent sends traffic through the IP address associated with this network interface. The other values indicate settings regarding the number of bytes in the packet header and frequency of sampling that sFlow should perform.

2. Now you should create sFlow agent for the bridge using the following command:

$ ovs-vsctl -- --id=@sflow create sflow agent=${AGENT_IP} target="${COLLECTOR_IP}\:${COLLECTOR_PORT}" header=${HEADER_BYTES} sampling=${SAMPLING_N} polling=${POLLING_SECS} -- set bridge br-int sflow=@sflow

Note down the UUID of the sFlow agent returned by this command; this value is necessary to remove the sFlow configuration. You can also see the list of sFlow agents  using the following command:

$ ovs-vsctl list sflow

To remove sFlow agent configuration from a bridge, in our case br-int, run the following command

$ ovs-vsctl remove bridge br-int sflow <UUID>

where UUID is the id of the sFlow agent returned in the earlier configuration.

3. Now you should download and install sFlow-RT as an engine for collecting stream from sFlow Agents embedded in the software switch. sFlow-RT converts monitoring streams into metrics accessible through the REST APIs.  To download, install and run sFlow-RT use the following command on the collector host (COLLECTOR_IP).

tar -xvzf sflow-rt.tar.gz
cd sflow-rt

For more info on sFlow-RT installation click here.

4. Now use a web browser to connect to to interact with the REST API. You can define flows using the flows tab to match packets or transactions that share common attributes and compute rate information. For example, the following flow  defines a flow called VMS that captures the source and destination IP addresses of VMs connected to the bridge and calculates bytes per second for each flow:


The following Python code defines the same flow using the REST API:

#!/usr/bin/env python
import requests
import json

flow = {'keys':'ipsource,ipdestination','value':'bytes','log':True}

To get your defined flow now you can use the following REST API:


Please leave your feedback and question on this article.
In case you found any bugs, please leave comments.


Installing OpenStack on a small cluster using CentOS and RDO

The Cluster

Below is our cluster setup. Please note that we are constrained by the devices we have and the service provider we are using. Your configuration might be different. Different network topologies might require some changes in the following instructions. Please be aware of what you are doing.


CentOS 7 Installation

  1. Install CentOS 7 with the following configuration on the head node of your cluster:
    You need a minimal version of CentOS and you can download the .iso file here (
           hostname: controller
           password: YOURPASSWORD
    choose “manually configure partition”, delete all the existing partitions, and then click “automatically generate partitions”. Adjust the amount of capacity assigned to the root and make it as large as possible. You can remove /home partition if you are not going to use it at all and allocate its space to /root.
  2. Do the same for all other nodes in the cluster and set the hostnames as follows:
          hostname: compute2,compute3, compute4, compute5, compute6, compute7
          password: YOURPASSWORD
    For partitioning choose “Use All Space” and check “Review and modify partitioning layout” then you can remove lv_home (/home) partition and add all the free space to lv_root (/).

Network Configuration

In our scenario, controller node has two interfaces, interface 1 (eno1) is connected to the public network and interface 2 (eno2)  is connected to a local switch that connects all the nodes in the cluster.

1. Controller (compute1 and gateway):

  1. Login with root username and password
  2. Stop the first network interface (eno1) from being managed by the NetworkManager daemon
    vi /etc/sysconfig/network-scripts/ifcfg-eno1

    save and exit.

  3. Set a static private IP address for the controller (
    vi /etc/sysconfig/network-scripts/ifcfg-eno2

    save and exit.

  4.  Restart the network service.
    systemctl restart network
  5. Check if you have the internet connection is working!
  6. Update your repository and install openssh-server openssh-clients nano and wget
    yum -y update
    yum install -y openssh-server openssh-clients nano wget net-tools
  7. Change the state of SELINUX to permissive:
    nano /etc/selinux/config
  8. Set the domain name for compute nodes.
    nano /etc/hosts controller compute1 gateway compute2 compute3
  9. Disable Network Manager and firewall to avoid conflicts with OpenStack
    systemctl stop firewalld
    systemctl disable firewalld
    systemctl stop NetworkManager
    systemctl disable NetworkManager
    systemctl enable network
    systemctl restart network


2. NAT Configuration on the Controller Node:

To provide Internet access to other machines in the cluster, you should enable NAT. If all machines in the cluster are getting public IPs by default you can skip this step.

  1. Enable the NAT forwarding from iptables to give Internet access to compute hosts by executing the following commands:
    yum install -y iptables-services
    chkconfig iptables on
    iptables -F
    iptables -t nat -F
    iptables -t mangle -F
    iptables -t nat -A POSTROUTING -o eno1 -j MASQUERADE
    iptables -A FORWARD -i eno2 -j ACCEPT
    iptables -A FORWARD -o eno2 -j ACCEPT
    service iptables save
    service iptables restart
  2. Check if iptable has been properly configured:
    iptables -S

    The output should include these:
    -A FORWARD -i eno2 -j ACCEPT
    -A FORWARD -o eno2 -j ACCEPT

  3.   To make sure you do not lose iptables configuration do the following:
    vi /etc/sysconfig/iptables-config
    service iptables restart
  4.  Enable forwarding
    nano /etc/sysctl.conf
  5. Reboot the controller machine and make sure the changes are persistent.

3. Compute Nodes

  1. Login with root username and password
  2. Set a static private IP address for each node
    vi /etc/sysconfig/network-scripts/ifcfg-eno2
    IPADDR= (
  3. Define some nameservers for your compute nodes
    vi /etc/resolv.conf
    nameserver #this is our first private DNS server
    nameserver #this is our second private DNS server
  4. Restart your network service.
    service network restart
  5. Update your repository and install openssh-server openssh-clients nano and wget
    yum -y update
    yum install -y openssh-server openssh-clients nano wget net-tools
  6. Change the state of SELINUX to permissive:
    nano /etc/selinux/config
  7. Set the domain name for compute nodes.
    nano /etc/hosts controller compute1 gateway compute2 compute3
  8. Disable Network Manager and firewall to avoid conflicts with OpenStack Networking Service.
    systemctl stop firewalld
    systemctl disable firewalld
    systemctl stop NetworkManager
    systemctl disable NetworkManager
    systemctl enable network
    systemctl restart network
  9. Reboot all machines to make sure the changes are persistent.

OpenStack Installation

Make sure all nodes (controller, compute2, compute3, …) are already configured and ready. Please refer to: if you have not sure about previous steps for your cluster setup.

  1. Make sure your /etc/environment is populated:
  2. vi /etc/environment
    LANG=en_US.utf-8 LC_ALL=en_US.utf-8
  3.  Install RDO release:
    yum install -y
    yum update -y
  4. Install openstack-packstack, a set of scripts to install all peaces of OpenStack, and generate the default settings for packstack:
    yum install -y openstack-packstack
    packstack --gen-answer-file=~/answers.cfg
  5. Export these environment variables
    export OS_USERNAME=admin
  6. Edit answers.cfg based on your requirements, make sure following setting is done.
  7., #these are our private ntp servers, use yours.
    # change the IP address of the controller to
    CONFIG_COMPUTE_HOSTS=,,,,,, # Add IP addresses of all compute nodes.
    CONFIG_NEUTRON_OVS_BRIDGE_IFACES=br-ex:eno2 #Pay attention here!!
  8.  Install packstack based on your config.
    packstack --answer-file=~/answers.cfg
  9. Source the keystonerc_admin before using command line for openstack commands. You can see the admin user and password for accessing the dashboard in this file.
    source keystonerc_admin
  10. If you have a domain name for your public IP address and you want to access your dashboard with domain name follow this instruction.
    vi /etc/httpd/conf.d/15-horizon_vhost.conf
    #for example
  11. Automate OpenStack environments sourcing on startup
    echo "source /root/keystonerc_admin" >> ~/.bashrc
  12. Now open OpenStack dashboard on your browser http://YOURDAMIN/dashboard for example can skip this step if you have aleardy set CONFIG_NEUTRON_OVS_BRIDGE_IFACES=br-ex:eno2. If external bridge is not properly created and you have network issues you can do it manually as explained below. Make sure you set CONFIG_NEUTRON_OVS_BRIDGE_IFACES=. first, you should create a bridge.
    vi /etc/sysconfig/network-scripts/ifcfg-br-ex

    Note that you are allocating the IP address of the controller to the bridge now.
    Now, you introduce controller node as a port to this

    vi /etc/sysconfig/network-scripts/ifcfg-eno2

    Restart your network to see everything’s working fine.

    service network restart


Virtual Network in OpenStack

For the network setup in OpenStack follow the steps in this clip.
Note that you need to create some images before perfoming these steps.

x11 forwarding in Windows using git-bash and Xming

This tutorial explains how you can get x11 forwarding working for Windows and git-bash (Not putty). Putty has another way of doing this.

  1. Install Xming:
  2. Run Xming from the programs.
  3. Open GitBash from the Start Menu.
  4. Export display environment on the bash command.
    export DISPLAY=localhost:0
  5. ssh to the target machine with x11 forwarding enabled
    ssh -XY me@myhost
  6. Try your x11 forwarding e.g. nautilus on the target machine.

How to fix “A certificate could not be found that can be used with this Extensible Authentication Protocol. (Error 789)” for Azure Point-to-Site VPN

In my previous post, I discussed on how you can configure Azure point-to-site VPNs to create a virtual network on the Azure platform. In some circumstances, you may get error 798 with the status “A certificate could not be found that can be used with this Extensible Authentication Protocol”. In the following instruction, I will go through options to resolve this issue.

After installing and configuring the Point-To-Site VPN client, sometimes the following error occurs when dialing the connection:

clip_image001_thumb2 - Copy

If you are sure the error is not related to the following problems then follow the instruction here.

  1. You did not add the client digital certificate on the computer you are trying to make the VPN connection. Check that you followed all steps in Part 3.2 Generate and install the client certificates of our instruction on how to configure Azure point-to-site VPNs.
  2. The corresponding digital certificate exists, but it has not been imported into the Personal Store. Maybe it is imported into the Computer Store of the certificate store.

The instruction is designed based on Windows 7. But other Windows versions would be very similar with minor changes.

Manual Configuration of VPN in Windows

  1. Open Control Panel> Network and Sharing Center> Set up a new connection or network
  2. Connect to a workplace


  3.  Use my Internet Connection (VPN)
  4. Now you should give a name for your connection. For the address, you should indicate the TunnelAddress to which you want to connect.
    addressTo find the address follow you should check the log file of your unsuccessful VPN connection you created based on the VPN client package downloaded from Azure.
    a) Right click on your VPN connection.Screenshot 2017-04-12 14.56.31
    b) Open properties.
    Screenshot 2017-04-12 13.13.10

    c) Open view log.
    You should find something like Tunnel DeviceName =  TunnelAddress = azuregateway-59cc4……………………
    This is the address.

  5. Right click on your VPN connection. You should modify properties of the Manual VPN you created. Open properities in the Security tab.
  6. Then enter the properties to choose the option “User a certificate on this computer”.smart
  7. Finally, to continue having Internet connection even when we are connected to the VPN go to the “Networking” tab and from there go to the properties of the IPv4 protocol.
  8. Then we will choose the “Advanced”
  9.  And uncheck the option “Use default Gateway on remote network”.
  10. Now test the connection to Azure and it should be giving you the option to select the client certificate.
  11.  Select your Azure Client Certificate and ok.
  12. You must be already connected to Azure!!!

How to activate the “Enable Strong private key protection” option.

If you are trying to import a certificate into Personal Local Certificate store, and you face the situation that the “Enable strong private key protection. You will be prompted every time the private key is used by an application if you enable this option” option is grayed out as below picture.


You can follow the below instruction to resolve the issue:

a. Open the mmc.exe on run prompt


b. File add/Remove Snap-ins, double click on Group policy object:


c. Just finish and ok.


d. Open computer configuration> windows Settings> Security Settings> Local Policies>Security Options on the right panel. Then find System Cryptography: Force Strong key protection for user key stored on the computer and open it.


e. Select User is not required when keys are stored and used


f. Apply and ok.

Configure Azure Point-to-Site VPN Connections – Azure Resource Manager (ARM)


#Part 1: create the VPN and Gateway

First You need to create your VNet and VPN gateway in Azure. You can do this using PowerShell or Azure portal. Below I provided commands for PowerShell.

Select-AzureRmSubscription -SubscriptionName "Your Subscription Name"
$VNetName = "Aneka-VNET-SITE"
$FESubName = "FrontEnd"
$BESubName = "Backend"
$GWSubName = "GatewaySubnet"
$VNetPrefix1 = ""
$VNetPrefix2 = ""
$FESubPrefix = ""
$BESubPrefix = ""
$GWSubPrefix = ""
$VPNClientAddressPool = ""
$RG = "Adel_Aneka_Test"
$Location = "Australia Southeast"
$DNS = ""
$GWName = "GW"
$GWIPName = "GWIP"
$GWIPconfName = "gwipconf"
#Create a new resource group.
New-AzureRmResourceGroup -Name $RG -Location $Location
#Create a front-end, gateway and backend subnet
$fesub = New-AzureRmVirtualNetworkSubnetConfig -Name $FESubName -AddressPrefix $FESubPrefix
$besub = New-AzureRmVirtualNetworkSubnetConfig -Name $BESubName -AddressPrefix $BESubPrefix
$gwsub = New-AzureRmVirtualNetworkSubnetConfig -Name $GWSubName -AddressPrefix $GWSubPrefix
#Create a virtual network
New-AzureRmVirtualNetwork -Name $VNetName -ResourceGroupName $RG -Location $Location -AddressPrefix $VNetPrefix1,$VNetPrefix2 -Subnet $fesub, $besub, $gwsub -DnsServer $DNS
#Specify the variables for the virtual network you just created.
$vnet = Get-AzureRmVirtualNetwork -Name $VNetName -ResourceGroupName $RG
$subnet = Get-AzureRmVirtualNetworkSubnetConfig -Name "GatewaySubnet" -VirtualNetwork $vnet
#Request a dynamically assigned public IP address. This IP address is necessary for the gateway to work properly. You will later connect the gateway to the gateway IP configuration
$pip = New-AzureRmPublicIpAddress -Name $GWIPName -ResourceGroupName $RG -Location $Location -AllocationMethod Dynamic
$ipconf = New-AzureRmVirtualNetworkGatewayIpConfig -Name $GWIPconfName -Subnet $subnet -PublicIpAddress $pip
# Generate and upload certificates: for doing this follow the instruction after this code and copy the public key of the generated certificate here
$MyP2SRootCertPubKeyBase64 = "MIIDETCCAf2g…….j4/FrCI"
$p2srootcert = New-AzureRmVpnClientRootCertificate -Name $P2SRootCertName -PublicCertData $MyP2SRootCertPubKeyBase64
New-AzureRmVirtualNetworkGateway -Name $GWName -ResourceGroupName $RG -Location $Location -IpConfigurations $ipconf -GatewayType Vpn -VpnType RouteBased -EnableBgp $false -GatewaySku Standard -VpnClientAddressPool $VPNClientAddressPool -VpnClientRootCertificates $p2srootcert

# Part 2: generate a root certificate

If you are not using an enterprise certificate solution, you’ll need to generate a self-signed root certificate. The steps in this section were written for Windows 7 (Should be similar for other Windows with some minor changes). If you had issues with windows 8, 8.1 and 10 at the end of this post I will explain some tricks to resolve possible problems.

You can follow either of the following methods:

  1. Run command prompt of windows as administrator (right click on command prompt, run as administrator).
  2. Change directory to the location of makecert.exe.
    1. For my case: cd C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\Bin
  3. Run this command:
makecert -sky exchange -r -n "CN=RootCertificateName" -pe -a sha256 -len 2048 -ss My "RootCertificateName.cer"

RootCertificateName is the name of the root certificate authority (CA), it can be your name. RootCertificateName.cer is the name of the file to store this certificate.

-r means to create a self-signed certificate

-ss is the certificate store name that stores the output certificate

-a and -len are used for encryption algorithm and length of the key.

By executing this command your self-signed certificate will be added to CurrentUser store location.

# Part 2.1: to get the public key

1. First, check that your certificate from the previous section is added to your personal certificates.
Run certmgr.msc, your Certificate should be there in personal certificates.


2. To get the public key, export the .cer file as a Base-64 encoded X.509 (.CER) file then open that file with notepad. There copy everything in between: —–BEGIN CERTIFICATE—– & —–END CERTIFICATE—–
Make sure you remove hidden ENDOFLINE characters.

A) Right click on the RootCertificate you created> all Tasks> Export


B) Select No, do not export the private key


C) Select Base-64 encoded X.509 (.CER)


D) Select a path and a name for your certificate file


E) Next and finish.


F) Open the file you saved with notepad and make a single line form of the public key between —–BEGIN CERTIFICATE—– & —–END CERTIFICATE—–


G) This is the public you should be copied to $MyP2SRootCertPubKeyBase64 variable in # Part 2.1 of the creation of VPN.

# Part 3: Connecting Clients to the Virtual Network

If your virtual network is created successfully (you can check in Azure portal) now you can connect clients to the virtual network. For doing so you need two things:

1) the VPN client and

2) a client certificate installed.

# Part 3.1 Download the VPN client configuration package.

1. To download the client configuration package, run the following commands in powershell. Make sure Azure Resource Manager PowerShell cmdlets is installed (You can use the Microsoft Web Platform Installer from this address:


Get-AzureRmVpnClientPackage -ResourceGroupName $RG -VirtualNetworkGatewayName $GWName -ProcessorArchitecture Amd64


2. The PowerShell cmdlet will return a URL link. Copy-paste the link that is returned to a web browser to download the package to your computer.

You can also download it from the portal:

Resource groups>Adel_Aneka_Test>Aneka-VNET-SITE>GW- Point-to-site configuration


3. Install the package. You should see the VPN connection on by clicking on your network access icon on the tray.


# Part 3.2 Generate and install the client certificates.

Follow these steps and generate a certificate for each computer needs to be connected to the virtual network.

Look at if you are not sure what you are doing.
1. First run the following command to generate the Client certificate.

makecert.exe -n "CN=ClientCertificateName" -pe -sky exchange -m 96 -ss My -in "RootCertificateName" -is My -a sha256

You can generate as many as client certificate you need this way. If you do not have makecert.exe, you can install it with Microsoft Windows SDK for Windows 7 and .NET Framework 4. 

2. Run certmgr.msc Make sure the ClientCertificateName is added to your personal certificates.

3. Right click on the Client Certificate and export


4. Select, Yes export the private key


5. Leave the default selection:


6. Provide a password for the private key


7. Select a path and a name for your certificate file.


8. Next and finish.

9. Then copy the exported .pfx file to the target machine wants to connect to the virtual network.

10. Double click on the file on the target machine and follow the steps:

a. Next,


b. Leave the path as it is,

c. Type the password for the private key and make sure Enable strong key protection is not checked. If this option is grayed out you should follow the instruction here to make it selectable.


d.  Next and select place all certificates in the following store and browse and find personal.


e. Next and finish.

# Part 3.3 Connect to the VPN

1. Click on connect button on your VPN connection created on part 3.1.3.


2. Click on connect.


3. Click on continue and accept yes.


4. Select client certificate you imported and ok, if installed more than one certificate, otherwise it will connect automatically.

Congratulations!!! you finished Rostam’s Seven Labours to setup Point-to-Site VPN connection for Azure.

In case, unfortunately, you have got  the following error:

A certificate could not be found that can be used with this Extensible Authentication Protocol. (Error 789)

You can try my post on How to fix “A certificate could not be found that can be used with this Extensible Authentication Protocol. (Error 789)” for Azure Point-to-Site VPN. Hopefully, you can get rid of the problem.

clip_image001_thumb2 - Copy

# Part 4: To add and remove extra Root Certificates.

You can add up to 20 root certificates to Azure. Follow the steps below to add a root certificate.
1. Create and prepare the new root certificate for upload based on method explained in #part 2, the run following PowerShell commands:

$P2SRootCertName2 = "ARMP2SRootCert2.cer"
$MyP2SCertPubKeyBase64_2 = "MIIC/zC……...m7ju"

2. Upload the new root certificate. Note that you can only add one root certificate at a time.

Add-AzureRmVpnClientRootCertificate -VpnClientRootCertificateName $P2SRootCertName2 -VirtualNetworkGatewayname $GWName -ResourceGroupName $RG -PublicCertData $MyP2SCertPubKeyBase64_2

3. You can verify that the new certificate was added correctly by using the following cmdlet.

Get-AzureRmVpnClientRootCertificate -ResourceGroupName $RG -VirtualNetworkGatewayName $GWName

4. You can remove a certificate using the following cmdlet.

Remove-AzureRmVpnClientRootCertificate -VpnClientRootCertificateName $P2SRootCertName2 -VirtualNetworkGatewayName $GWName -ResourceGroupName $RG -PublicCertData "MIIC/z……qgTWCIcb7ju"

You can also do the same in Azure Portal by copying everything in between: —–BEGIN CERTIFICATE—– & —–END CERTIFICATE—– in the following page at the following path

Resource groups>Adel_Aneka_Test>Aneka-VNET-SITE>GW- Point-to-site configuration:

Please leave comments for me if you found any bugs in the instruction.

Passwordless SSH login

Your aim

You need an automatic login from host A / user ‘a’ to Host B / user ‘b’. You don’t want to enter any passwords, maybe because you want to call ssh from a within a shell script.

How to do it

First log in on A as user ‘a’ and generate a pair of authentication keys. Do not enter a passphrase:

a@A:~> ssh-keygen -t rsa

Generating public/private rsa key pair.

Enter file in which to save the key (/home/a/.ssh/id_rsa):

Created directory ‘/home/a/.ssh’.

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /home/a/.ssh/id_rsa.

Your public key has been saved in /home/a/.ssh/

The key fingerprint is:

3e:4f:05:79:3a:9f:96:7c:3b:ad:e9:58:37:bc:37:e4 a@A

Now use ssh to create a directory ~/.ssh as user ‘b’ on B. (The directory may already exist, which is fine):

a@A:~> ssh b@B mkdir -p .ssh

b@B’s password:

Finally append a’s new public key to b@B:.ssh/authorized_keys and enter b’s password one last time:

a@A:~> cat .ssh/ | ssh b@B 'cat >> .ssh/authorized_keys'

b@B’s password:

From now on you can log into B as b from A as a without password:

a@A:~> ssh b@B hostname

ssh-copy-id -i ~/.ssh/ username@mystery