NSX-T: Part 3 – Deploy NSX-T Manager on KVM Host

Lab env (nested)

Perform a CentOS 7.6 installation with 2 vNICS and expose HW assited virtualization to the guest OS.

kvm1

Host preparation

#Disable SELinux

#Modify YUM configuration to avoid any unsupported package upgrades

echo ‘exclude=kernel* redhat-release* kubelet-* kubeadm-* kubectl-*docker-*’ >> /etc/yum.conf

#Install additional packages

yum groupinstall “Virtualization Hypervisor”
yum groupinstall “Virtualization Client”
yum groupinstall “Virtualization Platform”
yum groupinstall “Virtualization Tools”
yum install guestfish

#Verify KVM module (kvm_intel or kvm_amd)

[root@cen-s1-20 ~]# lsmod | grep kvm
kvm_amd 2177212 0
kvm 586948 1 kvm_amd
irqbypass 13503 1 kvm

Network setup

In the following steps I’ll try to simulate a possible real scenario with a VLAN (110) and network redundancy.

KVM Net

#Modify ens161 & ens192 configuration file as follow to setup a bond interface (edit file with vi editor or modify with nmtui tool).

[root@cen-s1-20 network-scripts]# cat ifcfg-ens161
HWADDR=00:50:56:90:23:6F
TYPE=Ethernet
NAME=”ens161″
DEVICE=ens161
ONBOOT=yes
UUID=6d530e5a-74ea-442c-ab2d-2bceefaa5975
MASTER_UUID=370cce9c-a0a6-4df2-87b9-8f5217113133
MASTER=bond0
SLAVE=yes

[root@cen-s1-20 network-scripts]# cat ifcfg-ens192
HWADDR=00:50:56:90:E1:A4
TYPE=Ethernet
NAME=”ens192″
DEVICE=ens192
ONBOOT=yes
UUID=6d4062cc-5bd2-4999-baf1-8ac6f08b52ec
MASTER_UUID=370cce9c-a0a6-4df2-87b9-8f5217113133
MASTER=bond0
SLAVE=yes

#Create the bond configuration file

[root@cen-s1-20 network-scripts]# cat ifcfg-bond0
BONDING_OPTS=”downdelay=0 miimon=1 mode=active-backup updelay=0″
TYPE=Bond
BONDING_MASTER=yes
BOOTPROTO=none
NAME=”bond0″
DEVICE=bond0
UUID=370cce9c-a0a6-4df2-87b9-8f5217113133
ONBOOT=yes

#Create the bond/VLAN configuration file

[root@cen-s1-20 network-scripts]# cat ifcfg-bond0.110
VLAN=yes
TYPE=Vlan
VLAN_ID=110
NAME=”bond0.110″
DEVICE=bond0.110
BOOTPROTO=none
ONBOOT=yes
BRIDGE=br0.110

#Create the bridge configuration file with the management IP parameters

[root@cen-s1-20 network-scripts]# cat ifcfg-br0.110
DEVICE=br0.100
NAME=”br0.100″
TYPE=Bridge
ONBOOT=yes
BOOTPROTO=static
IPADDR=192.168.110.20
NETMASK=255.255.255.0
DEFROUTE=yes
GATEWAY=192.168.110.1
DNS1=192.168.10.4
DOMAIN=nvlabs.local

#Restart network service

[root@cen-s1-20 network-scripts]# systemctl restart network

#Verify network configuration

[root@cen-s1-20 ~]# ifconfig
bond0: flags=5187<UP,BROADCAST,RUNNING,MASTER,MULTICAST> mtu 1500
inet6 fe80::250:56ff:fe90:236f prefixlen 64 scopeid 0x20<link>
ether 00:50:56:90:23:6f txqueuelen 1000 (Ethernet)
RX packets 6891 bytes 7639766 (7.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 63 bytes 7663 (7.4 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

bond0.110: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 00:50:56:90:23:6f txqueuelen 1000 (Ethernet)
RX packets 1487 bytes 880292 (859.6 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 55 bytes 7007 (6.8 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

br0.110: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.110.20 netmask 255.255.255.0 broadcast 192.168.110.255
inet6 fe80::3402:5bff:fe56:ac0d prefixlen 64 scopeid 0x20<link>
ether 00:50:56:90:23:6f txqueuelen 1000 (Ethernet)
RX packets 56 bytes 6797 (6.6 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 57 bytes 7139 (6.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

ens161: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST> mtu 1500
ether 00:50:56:90:23:6f txqueuelen 1000 (Ethernet)
RX packets 6884 bytes 7639346 (7.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 63 bytes 7663 (7.4 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

ens192: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST> mtu 1500
ether 00:50:56:90:23:6f txqueuelen 1000 (Ethernet)
RX packets 7 bytes 420 (420.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 144 bytes 16176 (15.7 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 144 bytes 16176 (15.7 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

#Verify bridge status

[root@cen-s1-20 ~]# brctl show
bridge name bridge id STP enabled interfaces
br0.110 8000.00505690236f no bond0.110

NSX Manager image setup

Download nsx-unified-appliance-2.4.0.0.0.12456291.qcow2 file from VMware and scp to the KVM machine (/var/lib/libvirt/images/).

In the same directory where you saved the QCOW2 image, create a file called guestinfo.xml with following deployment parameters.

<?xml version=”1.0″ encoding=”UTF-8″?>
<Environment
xmlns=”http://schemas.dmtf.org/ovf/environment/1″
xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance”
xmlns:oe=”http://schemas.dmtf.org/ovf/environment/1″>
<PropertySection>
<Property oe:key=”nsx_cli_passwd_0″ oe:value=”Your Password“/>
<Property oe:key=”nsx_cli_audit_passwd_0″ oe:value=”Your Password“/>
<Property oe:key=”nsx_passwd_0″ oe:value=”Your Password“/>
<Property oe:key=”nsx_hostname” oe:value=”nsx-s1-04.nvlabs.local“/>
<Property oe:key=”nsx_role” oe:value=”nsx-manager nsx-controller“/>
<Property oe:key=”nsx_isSSHEnabled” oe:value=”True“/>
<Property oe:key=”nsx_allowSSHRootLogin” oe:value=”True“/>
<Property oe:key=”nsx_dns1_0″ oe:value=”192.168.10.4“/>
<Property oe:key=”nsx_ntp_0″ oe:value=”192.168.10.5“/>
<Property oe:key=”nsx_domain_0″ oe:value=”nvlabs.local“/>
<Property oe:key=”nsx_gateway_0″ oe:value=”192.168.110.1“/>
<Property oe:key=”nsx_netmask_0″ oe:value=”255.255.255.0“/>
<Property oe:key=”nsx_ip_0″ oe:value=”192.168.110.25“/>
</PropertySection>
</Environment>

Use guestfish to write the guestinfo.xml file into the QCOW2 image

[root@cen-s1-20 ~]# guestfish –rw -i -a nsx-unified-appliance-2.4.0.0.0.12456291.qcow2 upload /var/lib/libvirt/images/guestinfo.xml /config/guestinfo

Deploy NSX Manager

Deploy the QCOW2 image with the virt-install command

[root@cen-s1-20 ~]# virt-install –import –vnc –name nsx-s1-04 –ram 16000 –vcpus 4 –network=bridge:br0.110,model=e1000 –disk path=/var/lib/libvirt/images/nsx-unified-appliance-2.4.0.0.0.12456291.qcow2,format=qcow2

Verify NSX Manager kvm machine

[root@cen-s1-20 ~]# virsh list
Id Name State
----------------------------------------------------
5 nsx-s1-04 running

Verify bridge status

[root@cen-s1-20 ~]# brctl show
bridge name bridge id STP enabled interfaces
br0.110 8000.00505690236f no      bond0.110
                                  vnet0

Now the NSX Manager vInterface (vnet0) is bridged to br0.110

[root@cen-s1-20 ~]# virsh console nsx-s1-04
Connected to domain nsx-s1-04
Escape character is ^]

VMware NSX Unified Appliance 2.4.0.0.0.12456291 – release – ttyS0
nsx-s1-04 login: admin
Password:
NSX CLI (Manager, Policy, Controller 2.4.0.0.0.12456291). Press ? for command list or enter: help

nsx-s1-04> get interface
Interface: eth0
Address: 192.168.110.25/24
MAC address: 52:54:00:ac:66:ce
MTU: 1500
Default gateway: 192.168.110.1
Broadcast address: 192.168.110.255
Link status: up
Admin status: up
RX packets: 28
RX bytes: 2859
RX errors: 0
RX dropped: 0
TX packets: 27
TX bytes: 2052
TX errors: 0
TX dropped: 0
TX collisions: 0

Try to ping default GW

nsx-s1-04> ping 192.168.110.1
PING 192.168.110.1 (192.168.110.1) 56(84) bytes of data.
64 bytes from 192.168.110.1: icmp_seq=1 ttl=64 time=1.05 ms
64 bytes from 192.168.110.1: icmp_seq=2 ttl=64 time=1.18 ms
^C
— 192.168.110.1 ping statistics —
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 1.058/1.123/1.188/0.065 ms

Ok! NSX Manager is up and running! Let’s try to login via web UI

kvm2