Tag Archives: NSX

Monitoring Physical Switches with vRealize Operations

This is a blog contributed by my friend Luciano Gomes, a VMware PSO Consultant in Rio de Janeiro Area, Brazil. Thank you Lucky!

 

If you were, like I was, dreaming of the possibility to monitoring physical network devices using vRealize Operations, your dream has finally come true! VMware released the Management Pack for Network Devices. I’m going to call it MPND for brevity.

It complements the Management Pack for Storage Devices (MPSD). These 2 are not just a standalone management. Think of them as the core or foundation adapters, that other management pack can leverage. For example, NSX uses MPND, and VSAN uses the MPSD.

Copying from the official Release Notes, it collects data on data center switch objects with the use of several different protocols and API’s. These include neighbor switch to switch objects via CDP and LLDP, and health, operations and performance metrics via SNMP.  This management pack discovers and collects data for all the leaf switches and spine switches in your physical network environment, and discovers the relationships between these switches and vCenter objects, namely hosts and VM.

The following diagram shows the architecture. I got this from a presentation in our intranet, but not able to figure out the colleague who did it (likely it’s our Product Manager Bill Erdman). As you can see, it collects standard properties/metrics via SNMP, and proprietary properties/metrics via vendor specific adapter.

Capture

I’ve tested both V2 and V3. While not supported, it can actually collect from non-directly connected switch.

BTW, if you are using NSX, you do not need to install this management pack. It’s automatically installed when you install the NSX Management Pack 3.0. Yes, it install both solutions automatically so you get to see the physical layer too. Romain Decker has shared a great post on the NSX Management pack here.

We know that all v1.0 products are released with a lot of space to improvements. For example, the five minute collection intervals remains, so this not a real time SNMP monitoring or trap alerting. In my opinion, this ability to see physical device is a big step. It finally closes the gap in the visibility area for VMware administrators. It provides basic functionalities, such as

  • Physical Network Overview (main visibility dashboard)
  • Network Device Connectivity (troubleshooting dashboard)
  • Standard resource mapping tree views, operation scores, spark lines
  • Problem alerting, based on HA redundancies within data center switch fabrics
  • Topology visualization with underlying traversal spec determining neighbor relationships
  • Troubleshooting for network connectivity tracing
  • Top N leaf and spine switch by traffic volumes

In this blog I will explain in a few steps how you can monitor them:

Pre-requisites

Take note of the pre-requisite before you eagerly deploying this new cool toy.

  • vRealize Operations 6.1 and above. Yes, 6.0.x is not supported. Here is an upgrade guide.
  • vCenter 5.1 and above
  • LLDP or CDP enabled and configured on all switches. Someone has written a guide here. The blog does not have the owner name, so if you know him/her, let me know and I’ll add the name here.
  • IP address advertising enabled
  • TTL enabled
  • IPv4 addresses must be configured on all switches. IPv6 is not supported.

Installation

After configuring both your Physical and Virtual Switches, download the MPND from Solutions Exchange. To install the MPND, go to Administration->Solutions and click green plus icon:

image004

Follow the wizard:

image006

1) Click Browse and choose the PAK file
2) Click Upload
3) Click Next and wait
4) Finally, click Finish

Configure the adapter:

Click Management pack for Network Devices

image010

Click Configure, and you will get the following dialog box.

image014

You must provide the information from your Physical Switch. If you are using SNMP v3, you must provide the username and password. In my case, I was using just SNMP v2.

Click on green plus icon and provide your community and click ok.

image016

Click Test Connection to ensure it works. Click Save Settings. You should see the status of the MPND like what I have below.

image018

Go for your well deserved coffee break. Come back and you will see your physical switches!

image020

Things to note if you are using NSX Management Pack. From the MPND official Release Notes:

You cannot install the Management Pack for Network Devices on top of the Management Pack for NSX for vSphere version 2.0. If you want to use both management packs to manage your network environment, you must upgrade to the Management Pack for NSX for vSphere 3.0. This management pack automatically installs the Management Pack for Network Devices.

Enjoy MPND ☺

VMware NSX 6.2.1 upgrade

Please read the preparation before upgrading. The official manual states the upgrade procedure is as follows:

  1. NSX Manager
  2. NSX Controller
  3. Host Clusters
  4. NSX Edge
  5. Guest Introspection

There is only 1 download, as everything is contained within the Manager.

NSX 1

Stage 1: NSX Manager Upgrade

To update the NSX Manager, login to its UI as Admin. Click on the Upgrade button.

NSX 2

Choose the file that you’ve downloaded earlier.

NSX 3

It will upload the file. It should only take a few minutes. If it’s taking >10 minutes, just cancel it and repeat. Once done, it will show you the following screenshot.

NSX 4

Notice it gives you the choice to enable SSH. Click on Upgrade, and the following progress window appears. Again, this should only take a few minutes.

NSX 5

Once done, it may log you out (it could be due to time out). Login, and you will see 6.2.1 as the version.

NSX 6

Repeat the same step for all your NSX Managers instances. They are upgraded independently.

Stage 2: Controller Upgrade

The next step is upgrading the Controller. What a pleasant surprise! The official manual has screenshot! Please read it first.

Login as the NSX Administrator. That’s the ID you used to deploy the NSX Manager. I normally maintain a separation between the Network admin and the Server admin, so I use different accounts in the lab so I can experience what it will be in actual environment. As you can see here, I login as Server Admin (vCenter) and I do not even see the Manager.

NSX 7

I normally use 2 browsers. Here, I login as the NSX Manager. Can you spot that Upgrade Available button? Essentially, the controller upgrade is a 1-click operation. The Manager will automatically detect that there is a new version, and provide an upgrade button. Thank you Ray and team!

The VMware ASEAN lab has 2 NSX Managers in primary-secondary setup. You do not have to upgrade the secondary controller. Notice the status is Imported Controller Cluster.

NSX 8

Don’t worry if you click the button accidentally. There is a prompt, defaulted to No 🙂

NSX 9

I clicked Yes, and you can see the status changed to In progress.

NSX 10

The entire process takes around minutes, not hours. Along the way, you see update such as below

NSX 11

When it is done, it will show the software version. Notice it has upgraded the Secondary Manager’ Controllers too.

NSX 12

I notice that vCenter recent tasks do not show any activity. I was thinking that VM reboot would show up in the vCenter. If you know why, let me know.

You might notice that the Peers column has issue. It’s showing red, and Controller 3 has no status. Just wait for a few minutes. It will look like this after a while

NSX 123

Stage 3: VMkernel VIB Upgrade on Cluster

Once the controller is upgraded, your next step is to upgrade the cluster. Once again, the manual has screenshots, so I do not have to repeat it. Please take note that ESXi is rebooted.

NSX 13

Again, clicking the Upgrade available button will prompt a dialog box, giving you a chance to confirm. Once you confirm, the UI will show the progress, as shown below.

NSX 15

This time around, vCenter recent tasks shows the progress.

NSX 14

There seems to be 2 pair of Scan + Uninstall for each host. I think this is installing the VIB.

NSX 16

I hit some issue due to my DRS setting, so I will need to spend some time on it. The manual said that ” If hosts require manual intervention to enter maintenance mode (for example, due to HA requirements or DRS rules), the upgrade process stops and the cluster Installation Status displays Not Ready

NSX 17

That’s what I hit, so I will need to fix that first. I manually entered the host into maintenance mode. Once done, I click the upgrade again. I did it on one of the host first, and you can see the status is now showing 6.2.1

NSX 21

I continued on the remaining of the hosts in the cluster.

NSX 34

Stage 4: NSX Edges and Router Upgrade

Once the hosts are upgraded, the next stage is to upgrade the NSX Edge (if you have any). Once again, the manual has the screenshot and it’s a pretty straight forward process. The manual said that “NSX Edges can be upgraded without any dependency on the NSX Controller cluster or host cluster upgrades. You can upgrade an NSX Edge even if you have not yet upgraded the NSX Controller cluster or host clusters.”

NSX 31

You get a warning that there will be service disruption, that’s why it’s important to know the operational impact of the upgrade.

NSX 32

Here is what the activities look like in vCenter recent tasks.

NSX 35

The upgrade was fast, each was less than a minute. The result was below. Notice the version is now 6.2.1

NSX 36

That wraps up the basic upgrade steps of NSX.

NSX Monitoring with Log Insight

BTW, it’s a good practice to have NSX logs analysed. As you can see below, Log Insight captures the errors that I encountered during the upgrade.

NSX 40

Roie Ben Haim, a PSO Consultant in Israel, brought up to me (Thanks Roie!) that NSX 6.2.1 release notes has the following info:

NSX 6.2 does not support Log Insight reporting
Due to an incompatibility in the vRealize Content Pack for NSX, NSX 6.2 does not support vRealize Log Insight Reporting.

It has no workaround.

I’m not sure what “reporting” means. As you can see in the above screenshot, the dashboards got populated. The screenshot below shows that both NSX Managers are sending syslog regularly. In fact, hundreds of syslog messages.

NSX 42

Log Insight has specific knowledge on NSX, and below is one such example.

NSX 41

Below is one of the out of the box dashboard provided by the Content Pack.

43

BTW, if you want to know how to enable syslog in NSX, this is a great how-to by Steven.

VMware NSX installation flow chart diagrams

[22 August 2015: updated to include NSX 6.2 deployment with 2 vCenter Servers at the end]

There are many great NSX installation blogs and videos. My favourite is this series by Sid Smith. My friend Roshan Jha has also detailed it here. In this post I will just post the flow chart diagrams. I could not find a diagram, so I created them.

Installation flow

From the above, you can see it’s pretty straight forward. I’m showing some tasks as parallel as the order does not matter. Otherwise, the sequence matters.

Please note that you cannot manually configure VXLAN. You have to let NSX configures it for you. You can manually configure the VXLAN custom TCP/IP stack, but you cannot configure the vmkernel interface.

Ray Budavari, someone I highly respect for his NSX knowledge, corrected me that NSX supports multiple sites. He did a presentation at VMworld 2014, which you can find here. Now that NSX 6.2 is out, you can do this even easier.

Once you have NSX deployed, it’s time to define the virtual network. There are 2 key virtual appliances that you need to deploy:

  1. Distributed Logical Router
  2. Edge Services

Here is the flow chart for Distributed Logical Router:

DLR

Here is the flow chart for Edge Services:

Edge

The above works for NSX 6.x and NSX 6.2 in a single vCenter setup. What about if you want to do the federation across multiple vCenter Servers? This is a new feature in NSX 6.2. Let’s cover the installation flow, as it obviously has to change a bit.

NSX 6.2 multi-vCenter setup

The main installation is very similar. Basically, you’re installing it 2x. You will have 2 NSX Managers, not 1. Each is paired to the respective vCenter. To me, this makes sense as it’s consistent with SRM. SRM does not assume anything is available when disaster strikes. So it’s not expecting you to have the SRM server not vCenter Server. There is a duplicate copy on each site.

In the diagram below, you see that you have 2 streams of installation. However, there is a step where you need to link the 2 NSX Manager. I’ve shown where you do the pairing between the 2 NSX Managers.

Installation flow 6.2

You may notice that I left a blank box on where the “Deploy NSX Controller” on the passive NSX Manager. This is because you do not have to deploy it. Ray Budavari and Moe Thwin spotted the mistake. [Thank you gentlemen!]

There is a new status called Primary and Secondary. You make the changes by clicking on the Actions icon, as shown below.

NSX Manager 2

What does it look like when it’s paired?

NSX Manager

Hope it helps. The above is certainly not the end of SDN. You still need to connect your Distributed Router to your Edge.