** [NOTE] – This blog is only applicable to NSX-T v2.3 and earlier.
Please see the NSX-T Architecture (Revamped): Part 1 for NSX-T v2.4 and later that talks about the new architecture introduced by VMware. **
Every few years we see a systemic shift in IT, the traditional technical ways of achieving the business objectives are replaced by implementing new advance technologies that are more efficient, secure and robust. In the world of networking and security, VMware NSX-T is gaining momentum and is the focus in “this era” of “clouds and containers”.
The “T” in NSX-T stands for Transformers. The use cases of NSX-T are similar to NSX-V i.e. software defined networking, micro-segmentation, automation, etc. but in addition, it also expands the scope to public clouds and other heterogeneous compute systems. NSX-T is not only designed to work well with VMware vSphere but also for other hypervisors platforms based on KVM and hence the word “multi-hypervisor” software defined networking solution.
As I began writing blog posts on NSX-T installation, I thought it was necessary to complement it with the NSX-T Architecture. This blog post is Part 1, and let’s review the following diagram to discuss the different planes of its architecture:
We will not do a deep dive on every individual component of NSX-T (perhaps in separate upcoming blogs in the future :)) but we will touch bases and cover all aspects briefly, to help make sense of what and why they are used.
Management Plane
The Management plane provides the entry point to NSX-T Datacenter, to perform operational tasks such as configuration and monitoring of all management, control, and data plane components. The changes are made using either by NSX-T Data Center UI or RESTful API.
NSX-T manager is the central point of management of the components of NSX-T. NSX-T does not have a 1 to 1 relationship with the vCenter like NSX-V, it can work with multiple vCenters servers at once.
Cloud Service Manager (CSM) provides a complete view of single pane of glass management endpoint for all your public clouds. CSM is a virtual appliance that provides the UI and REST APIs for onboarding, configuring, and monitoring your public cloud inventory.
NSX Container plugin (NCP) is a container pod deployed when using container-based applications. It provides integration between NSX-T and container orchestrators such as Kubernetes, as well as integration between NSX-T and container-based PaaS (platform as a service) products such as OpenShift and Pivotal Cloud Foundry.
Control Plane
The control plane is an advanced distributed state management system that provides control plane functions for logical switching and routing functions, along with propagating the distributed firewall rules.
It contains NSX-T Controllers which are Linux appliance deployed in the number of three instances, forming the Cluster. Unlike NSX-V, NSX-T controllers are necessary deployment of distributed firewalls.
The Control plane is further segregated into two parts the Central Control Plane (CCP) and Local Control Plane (LCP) – we’ll cover this in the Part 2 (the continuation of this blog post) when we discuss the NSX Agent Communication
Data Plane
The Data plane performs stateless packet forwarding, following the tables populated by the control plane and maintains statistics to report back to the NSX-T Manager. Many heterogeneous systems can now be part of the Data plane of NSX-T, lets discuss them briefly below:
Multi-Hypervisor
NSX-T is designed not just for ESXi but also to work with other Hypervisors based on KVM. Following are the Hypervisors supported by NSX-T v2.3 release:
Hypervisor | Version | CPU Cores | Memory |
---|---|---|---|
vSphere | ESXi 6.5U1 or later | 4 | 16 Gb |
RHEL KVM | 7.4 and 7.5 | 4 | 16 Gb |
Ubuntu KVM | 16.04.2 LTS | 4 | 16 Gb |
CentOS KVM | 7.4 | 4 | 16 Gb |
Note: Please refer the release notes of the NSX-T version you are going to deploy, to confirm the compatibility of the hypervisor version supported.
N-VDS: The module N-VDS is an NSX-T Virtual Distributed Switch. As NSX-T is not limited to vSphere environments, developing a new virtual switch from scratch was probable to cover all heterogeneous systems. N-VDS delivers network services to the virtual machines running on the hypervisors. Implementation of N-VDS on KVM hypervisors is founded by open vSwitch (OVS) which is platform independent.
Bare-metal Servers:
Starting with NSX-T v2.3, VMware introduced support for Linux based workloads as well as containers running on bare-metal servers. Following are the Operating Systems supported by NSX-T 2.3 release:
Operating System | Version | CPU Cores | Memory |
---|---|---|---|
RHEL | 7.5 and 7.4 | 4 | 16 Gb |
Ubuntu | 16.04.2 LTS | 4 | 16 Gb |
CentOS | 7.4 | 4 | 16 Gb |
Note: Please refer the release notes of the NSX-T version you are going to deploy, to confirm the compatibility of the hypervisor version supported.
EDGE:
NSX Edge nodes are service appliances dedicated to running centralised services like Load Balancing, NAT, Edge Firewall, VPN, DHCP, Connectivity to the physical network. They are grouped in one or several clusters, representing a pool of capacity. NSX Edge node can be deployed as VM appliance on ESXi OR on a bare-metal server but it is not supported on KVM based hypervisors, as of NSX-T 2.3 release.
Note: Edge node deployment on bare-metal servers have limited “Intel” CPU support and specific NIC card requirements. Therefore, I would recommend referring the compatibility of the NSX-T version before deployment.
Public Cloud:
As I mentioned in the beginning NSX-T also expands the scope to the Public Clouds, it does that by the deployment of Public Cloud Gateways (PCG) via the Cloud Service Manager (CSM). What it provides is the visibility, consistent network and security policy and end to end operational control via a single pane of glass.
Example – If you have NSX-T deployed within your Datacenter you can seamless extend the features to public cloud that you manage. Having said that, it does not mean you need to have NSX-T deployed and running in your Datacenter first, you could start with NSX on cloud and do a reverse journey to your Datacenter.
NSX-T v2.3 release supports AWS and Azure, however the idea is to add more clouds in the future, to keep in line with the preposition of “NSX Everywhere.”
NSX Cloud Gateways provides a localised NSX control plane in each VPC/VNET and is responsible for pushing policies to each public cloud instance. An NSX Agent is pushed to the VMs running in public clouds.
Containers:
Last but not the least, NSX-T also provides the network and security to the Docker based containers which is one of the main use cases for implementing NSX-T over NSX-V.
NSX-T provides a common framework to manage and increase visibility of environments that contain both VMs and containers.
The NSX-T Container Plug-in (NCP), I mentioned in the management plane (in the diagram above) provides direct integration with several environments where container-based applications exist. Container Orchestrators (sometimes referred to as CaaS), such as Kubernetes (k8s) are perfect models for NSX-T integration. Solutions that contain enterprise distributions of k8s e.g. RedHat Open Shift, Pivotal Container Service support solutions with NSX-T. Additionally, NSX-T supports integration with PaaS solutions like Pivotal Cloud Foundry.
Compatibility for a Kubernetes Environment for NSX-T v2.2 and v2.3 are as follows:
Software Product | Version |
---|---|
Hypervisor for Container Host VMs | ESXi v6.5 U1 and later RHEL KVM 7.4, 7.5 Ubuntu KVM 16.04 |
Container Host Operating System | Ubuntu 16.04, RHEL 7.4, 7.5 |
Container Orchestrator | Kubernetes 1.10 and 1.11 |
Container Host Open vSwitch | 2.9.1 (packaged with NSX-T 2.3 and 2.2) |
Compatibility for a Cloud Foundry Environment:
Software Product | Version |
---|---|
Hypervisor for Container Host VMs | ESXi 6.5 U1 or later |
Container Orchestrator | (NCP 2.3 only) Pivotal Application Service 2.1.x (except 2.1.0) and Pivotal Operations Manager 2.1.x Pivotal Application Service 2.2.0 and Pivotal Operations Manager 2.2.0 (NCP 2.3.1 only) Pivotal Application Service 2.3.x and Pivotal Operations Manager 2.3.x |
It is does get clear from above that NSX-T architecture differs quite a lot from NSX-V, though the basic concepts are similar. Having said that “NSX-T is not equivalent to NSX-V.
NSX-T excels in many areas compared to NSX-V, however it lacks in few areas (as of writing this blog post), a good example is e.g. cross vCenter Implementations. However, VMware intends to close this gap and have a complete parity between the features of NSX-V and NSX-T in the next upcoming releases.
Lets talk about the NSX Agent communication that act behind the scenes in the NSX-T Architecture Part 2.