×

Welcome to Knowledge Base!

KB at your finger tips

This is one stop global knowledge base where you can learn about all the products, solutions and support features.

Categories
All
Storage and Backups-Nutanix
Flow Virtual Networking Guide

Flow Virtual Networking pc.2022.6

Product Release Date: 2022-08-03

Last updated: 2022-12-09

Purpose

This Flow Virtual Networking Guide describes how to enable and deploy Nutanix Flow Virtual Networking on Prism Central.

Upgrading from EA Versions

If you have enabled the early access (EA) version of Flow Virtual Networking, disable it before upgrading the Prism Central and enabling the general availability (GA) version of Flow Virtual Networking.

Related Documentation

Links to Nutanix Support Portal software and documentation.

The Nutanix Support Portal provides software download pages, documentation, compatibility, and other information/

Documentation Description
Release Notes | Flow Virtual Networking Flow Virtual Networking Release Notes
Port Reference Port Reference: See this page for details of ports that must be open in the firewalls to enable Flow Virtual Networking to function.
Nutanix Security Guide Prism Element and Prism Central security, cluster hardening, and authentication.
AOS guides and release notes Covers AOS Administration, Hyper-V Administration for Acropolis, Command Reference, Powershell Cmdlets Reference, AOS Family Release Notes, and AOS release-specific Release Notes
Acropolis Upgrade Guide How to upgrade core and other Nutanix software.
AHV guides and release notes Administration and release information about AHV.
Prism Central and Web Console guides and release notes Administration and release information about Prism Central and Prism Element.

Flow Virtual Networking Overview

Enabled and administered from Prism Central, Flow Virtual Networking powers network virtualization to offer a seamless network experience with enhanced security. It is disabled by default.

To enable and use Flow Virtual Networking , ensure that you log on to Prism Central as a local account user with Prism Admin role. If you log on to Prism Central as a non-local account (IDP-based) user or without Prism Admin role privileges, then Prism Central does not allow you to enable or use Flow Virtual Networking . The task is reported as Failed with a User Denied Access message.

Note:

Nutanix deploys a number of ports and protocols in its software. ports that must be open in the firewalls to enable Flow Virtual Networking to function. To see the ports and protocols used Flow Virtual Networking , see Port Reference.

It is a software-defined network virtualization solution providing overlay capabilities for the on-prem AHV clusters. It integrates tools to deploy networking features like Virtual Private Cloud (VPC) and Virtual Private Network (VPN) to support flexible app-driven networking that focuses on VMs and applications instead of virtual LANs and network addresses.

After you enable it on Prism Central, Flow Virtual Networking delivers the following.

  • A simplified, Prism Central-based workflow that deploys the application-driven network virtualization feature.
  • A secure multi-tenancy solution allowing per-tenant isolation using VPC-based network segmentation and namespace isolation.
  • A secure VPN-based connectivity solution for multiple sites, with automated VPN bundle upgrades.
  • NAT-based secure egress to external networks, with IP address retention and policy-based routing.
  • Self-serve networking services using REST APIs.
  • Enhanced networking features for more effective disaster recovery.
    Note: You can enable network segmentation on a Layer 2 extended virtual subnet that does not have a gateway. For more information about Layer 2 subnet extensions, see Layer 2 Virtual Network Extension. For information about network segmentation of an extended layer 2 subnet, see Segmenting a Stretched L2 Network for Disaster Recovery in the Securing Traffic through Network Segmentation section of the Security Guide .

Deployment Workflow

You can enable Flow Virtual Networking using a simple Prism Central driven workflow, which installs the network controller. The network controller is a collection of containerized services that run directly on the Prism Central VM(s). The network controller orchestrates all the virtual networking operations.

  • Ensure that microservices infrastructure is enabled in Prism Central Settings > Prism Central Management . See Prism Central Guide for information about enabling microservices infrastructure.
  • Enable Flow Virtual Networking in Prism Central Settings > Advanced Networking . It is disabled by default. See Enabling Flow Virtual Networking

  • You can opt out of Flow Virtual Networking by disabling the Advanced Networking option subject to prerequisites to disable advanced networking. See Disabling Flow Virtual Networking.

  • You can deploy Flow Virtual Networking in a dark site (a site that does not have Internet access) environment. See the Deploying Flow Virtual Networking at a Dark Site topic for more information.

  • You can upgrade the Flow Virtual Networking controller. Nutanix releases an upgrade for the Flow Virtual Networking controller with AOS and Prism Central releases. See Upgrading Flow Virtual Networking.

    See the AOS Family Release Notes and Release Notes | Prism Central .

  • Flow Virtual Networking allows you to create and manage virtual private clouds (VPCs) and overlay subnets to leverage the underlying physical networks that connect clusters and datacenters. See Virtual Private Cloud.

  • You can upgrade the network gateway version. Network gateway is used to create VPN or VTEP gateways to connect subnets using VPN connections, or Layer 2 subnet extensions over VPN or VTEP.

Flow Virtual Networking Architecture

The Flow virtual networking architecture uses a three-plane approach to simplify network virtualization.

Prism Central provides the management plane, the network controller itself acts as the control plane while the AHV nodes provide the data plane. This architecture provides a strong foundation for Flow virtual networking. This architecture is depicted in the following chart.

Figure. Flow Virtual Networking Architecture Click to enlarge Flow virtual networking Architecture diagram

Deployment Scale

Flow virtual networking supports the following scale:

Entities Scale

Virtual Private Clouds

500

Subnets

5,000

Ports

50,000

Floating IPs

2,000 per networking controller-enabled Prism Central.

Routing Policies

1,000 per Virtual Private Cloud.

10,000 per networking controller-enabled Prism Central.

Essential Concepts

VPC

A Virtual Private Cloud (VPC) is an independent and isolated IP address space that functions as a logically isolated virtual network. A VPC could be made up of one or more subnets that are connected through a logical or virtual router. The IP addresses within a VPC must be unique. However, IP addresses may overlap across VPCs. As VPCs are provisioned on top of another IP-based infrastructure (connecting AHV nodes), they are often referred to as the overlay networks. Tenants may spin up VMs and connect them to one or more subnets within a VPC. Virtual Private Cloud (VPC) is a virtualized network of resources that are specifically isolated from the rest of the resource pool. VPC allows you to manage the isolated and secure virtual network with enhanced automation and scaling. The isolation is done using network namespace techniques like IP-based subnets or VLAN based networking.

VPC Subnets

You can use IP address-based subnets to network virtual machines within a VPC. A VPC may use multiple subnets. VPC subnets use private IP address ranges. IP addresses within a single VPC must be unique, in other words, IP addresses inside the same VPC cannot be repeated. However, IP addresses can overlap across multiple VPCs. The following figure shows two VPCs named Blue and Green. Each VPC has two subnets, 192.168.1.0/24 and 192.168.2.0/24, that are connected by a logical router. Each subnet has a VM with an IP address assigned. The subnets and VM IP addresses overlap between the two VPCs.

Figure. VPC Subnet Click to enlarge Displaying an illustration of VPC networks

The communication between VMs in the same subnets or different subnets in the same VPC (also called East-West communication) is enabled using GEneric NEtwork Virtualization Encapsulation (GENEVE). If a Prism Central manages multiple clusters, then the VMs that belong to the same VPC could be deployed across different clusters. The virtual switch on the AHV nodes provide distributed virtual switching and distributed virtual routing for all VPCs.

The communication from a VM in a VPC to an endpoint outside the VPC (called external communication or North-South communication) is enabled by an external network connection. Such a connection may be secured using VPN. The following figure shows the logical connectivity of the VPCs to the external network, and subsequently to the Internet.
Note: You must configure the default route (0.0.0.0/0) to the external subnet as the next hop for connectivity outside the cluster (north-south connectivity).
Figure. External Communication Click to enlarge

External Subnets

Subnets outside a VPC are external subnets. External subnets may be subnets within the deployment but not included in a specific VPC. External subnets may also be subnets that connect to the endpoints outside the deployment such as another deployment or site.

External subnets can be deployed with NAT or without NAT. You can add a maximum of two external subnets - one external subnet with NAT and one external subnet without NAT to a VPC. Both external subnets cannot be of the same type. For example, you cannot add two external subnets, both with NAT. You can update an existing VPC similarly.

Primary and Secondary IP Addresses for VMs
See VM IP Address Management.
SNAT and Floating IP Address

SNAT and Floating IP addresses are used only when you use NAT for an external subnet.

In Source Network Address Translation (SNAT), the NAT router modifies the IP address of the sender in IP packets. SNAT is commonly used to enable hosts with private addresses to communicate with servers on the public Internet.

For VMs within the VPC to communicate with the rest of the deployment, the VPC must be associated with an external network. In such a case, the VPC is assigned a unique IP address, called the SNAT IP, from the subnet prefix of the external network. When the traffic from a VM needs to be transmitted outside the VPC, the source IP address of the VM, which is a private IP address, is translated to the SNAT IP address. The reverse translation from SNAT IP to private IP address occurs for the return traffic. Since the SNAT IP is shared by multiple VMs within a VPC, only the VMs within the VPC can initiate connections to endpoints outside the VPC. The NAT gateway allows the return traffic for these connections only. Endpoints outside the VPC cannot initiate connections to VMs within a VPC.

In addition to the SNAT IP address, you can also request a Floating IP address — an IP from the external subnet prefix that is assigned to a VM via the VPC that manages the network of the VM. Unless the floating IP address is assigned to the private IP address (primary or secondary IP address) of the VM, the floating IP address is not reachable. When the VM transmits packets outside the VPC, the private IP of the VM is modified to the Floating IP. The reverse translation occurs on the return traffic. As the VM uses the Floating IP address, an endpoint outside the VPC can also initiate a connection to the VM with the floating IP address.

The translation of the private IP addresses to Floating IP or SNAT IP address, and vice versa, is performed in the hypervisor virtual switch. Therefore, the VM is not aware of this translation. Floating IP translation may be performed on the hypervisor that hosts the VM to which the floating IP is assigned to. However, SNAT translation is typically performed in a centralized manner on a specific host.

NAT Gateway

NAT Gateways are used only when you use NAT for an external subnet.

Network Address Translation (NAT) is a process for modifying the source or destination addresses in the headers of an IP packet while the packet is in transit. In general, the sender and receiver applications are not aware that the IP packets are being manipulated.

A NAT Gateway provides the entities inside an internal network with connectivity to the Internet without exposing the internal network and its entities.

A NAT Gateway is:

  • A node or a AHV host. You need a host or a node to implement a NAT Gateway because NAT gateways require operations like load balancing and routing that are automatically performed by Flow virtual networking.
  • Connected to the internal network with an internal subnet based IP address and to the external network with an externally-routable IP address.

    The externally-routable IP address may be an IP address from a private IP address space or an RFC1918 address that is used as a NAT gateway. The NAT Gateway IP address could be a static IP address or a DHCP assigned IP address.

Table 1. NAT Gateway Failover Time
Event Failover Time
Network controller stops on AHV Up to 45 seconds.
Node reboot Up to 45 seconds.
Node power off:

When NAT Gateway and network controller MSP worker VMs are not on the same node.

Up to 45 seconds.
Node power off:

When NAT Gateway and network controller MSP worker VMs are on the same node.

Up to 300 seconds (5 minutes).
Static IP Address

A static IP address is a fixed IP address that is manually assigned to an interface in a network. Static IP addresses provide stable routes that do not have to be updated frequently in the routing table since the static routes generated using static IP addresses do not need to be updated.

Usually in a large IP-based network (a network that uses IP addresses), a Dynamic Host Configuration Protocol or DHCP server assigns IP addresses to interfaces of an entity (using DHCP client service on the entity). However, some entities may require a static IP address that can be reached (manual remote access or via VPN) quickly. A static IP address can be reached quickly because the IP address is fixed, assigned manually and is stored in the routing table for a long duration. For example, a printer in an internal network would need a static IP address so that it can be connected reliably. Static IP addresses can be used to generate static routes which remain unchanged in routing tables, thus providing stable long-term connectivity to the entity that has the static IP address assigned.

Static Route

Static routes are fixed routes that are created manually by the network administrator. Static routes are more suited for small networks or subnets. Irrespective of the size of a network, static routes may be required in a variety of cases. For example, in VPCs where you use virtual private networks (VPNs) or Virtual Tunnel End Point (VTEP) over VxLAN transport connections to manage secure connections, you could use static routes for specific connections such as site-to-site connections for disaster recovery. In such a case it is necessary to have a known reliable route over which the disaster recovery operations can be performed smoothly. Static routes are primarily used for:

  • Facilitating the easy maintenance of the routing table in small networks that are not expected to grow.
  • Routing to and from other internal route or stub networks. A stub network or an internal route network is a network accessed using a single route and the router has only one neighbor.
  • Use as a default or backup route. Such a route is not expected to specifically match any other route in the routing table.

In a network that is not constantly changing, static routes can provide faster and more reliable services by avoiding the network overheads like route advertisement and routing table updates for specific routes.

Overlay networks

You can create an IP-based Overlay subnet for a VPC. An Overlay network is a virtualized network that is configured on top of an underlying virtual or physical network. A special purpose multicast network can be created as an Overlay network within an existing network. A peer-to-peer network or a VPN are also examples of Overlay networks. An important assumption for an Overlay network is that the underlying network is fully connected. Nutanix provides the capability to create Overlay network-based VPCs.

Comparing Overlay with VLAN

See how overlay networks compare with VLAN networks. A virtual local area network or VLAN network is a Layer 2 network that provides virtualized network segmentation solution. VLANs route and balance traffic in a network based on MAC addresses, Protocols such as Ethernet, ports or specific subnets. A VLAN creates a virtual Layer 3 network using Layer 2 addressing by separating broadcast domains virtually or logically. A VLAN configured network behaves as if the network is segmented using a physical layer 2 switch without implementing a layer 3 IP based subnet for the segmentation. VLAN traffic usually cannot traverse outside the VLAN.

The main advantage that VLAN networks provide is that VLAN networks require only layer 2 (L2) connectivity. VLANs do not require any of the layer 3 (L3) Flow virtual networking features.

Overlay networks can be laid on underlying physical network connections including VLAN networks. Overlay networks provide the following advantages and constraints:

  • IP address namespace is decoupled from the physical network.
  • You can create, update or delete overlay networks without requiring any configurations on the physical network and powering down the systems.
  • You can create overlay networks that can span across multiple clusters.
  • VLAN networks are necessary for Bootstrapping of Flow virtual networking.
    Note: Nutanix recommends using VLAN0 as the default untagged (also called native) VLAN for a CVM and AHV host. You can create VLANs for user VMs using the Network Configuration page. You can use the Create Virtual Switch dialog box from the Network Configuration page to create virtual switches for the user VM VLANs.
  • AHV Networking VLAN and Flow virtual networking VLAN: VLAN backed subnets for external connectivity are managed by the Flow virtual networking control plane. Traditional AHV VLAN IPAM networks are managed by Acropolis. Do not configure the same VLAN as both a Flow virtual networking external network and an AHV IPAM network, as this can lead to IP address conflicts.

Traffic Behavior

Broadcast Traffic

When all the guest VMs belonging to a subnet are in the same AHV: Flow virtual networking broadcasts the traffic to all guest VMs in the same subnet.

When some VMs belonging to a subnet are in other AHVs: Flow virtual networking tunnels the traffic to only those AHVs which have endpoints in the same subnet.

In other words, Flow virtual networking broadcasts traffic to all the guest VMs in the same subnet.

Unicast Traffic

Unicast traffic is traffic transmitted on a one-to-one basis between IP addresses and ports. There is only one sender and one receiver for the traffic. Unicast traffic is usually the most used form of traffic in any LAN network using Ethernet or IP networking. Flow virtual networking transmits unicast traffic based on the networking policies set.

Unknown Unicast Traffic

Flow virtual networking always drops unknown unicast traffic. It is not transmitted to any guest VM within or outside the source AHV.

Multicast Traffic

Flow virtual networking transmits the traffic to the VMs in the multicast group within the same subnet. If the VM is on another AHV, the destination AHV must have an endpoint in the subnet.

Multicast Group

A multicast group is defined by an IP address (called a multicast IP address, usually a Class D IP address) and a port number. Once a host has group membership, the host will receive any data packets that are sent to that group defined by an IP address/port number.

Prerequisites for Enabling Flow Virtual Networking

Make sure you meet these prerequisites before you enable Flow Virtual Networking on Prism Central.

Requirements

Important: Prism Central protection and recovery does not protect or recover Flow Virtual Networking services.

You must have the following fulfilled to enable Flow Virtual Networking :

  • Ensure that you log on to Prism Central as a local account user with Prism Admin role. If you log on to Prism Central as a non-local account (IDP-based) user or without Prism Admin role privileges, then Prism Central does not allow you to enable or use Flow Virtual Networking . The task is reported as Failed with a User Denied Access message.

  • Ensure that the Prism Central running Flow Virtual Networking is hosted on an AOS cluster running AHV.

    The network controller has a dependency only on the AHV version.

  • Ensure that microservices infrastructure on Prism Central is enabled. See Prism Central Guide for information about microservices infrastructure.
  • Choose the x-large PC VM size for Flow Virtual Networking deployments. Small or large PC VMs are not supported for Flow virtual networking.

    If you are running a small or large Prism Central VMs, upgrade the Prism Central VM resources to x-large PC VM. See Acropolis Upgrade Guide for procedure to install an x-large Prism Central deployment.

  • Although Flow Virtual Networking may be enabled on a single-node PC, Nutanix strongly recommends that you deploy a three-node scale-out Prism Central for production deployments. The availability of Flow Virtual Networking service in Prism Central is critical for performing operations on VMs that are connected to overlay networks. A three-node scale-out Prism Central ensures that Flow Virtual Networking continues to run even if one of the nodes with a PCVM fails.

  • Prism Central VM registration. You cannot unregister the Prism Element cluster that is hosting the Prism Central deployment where you have enabled Flow Virtual Networking . You can unregister other clusters being managed by this Prism Central deployment.

  • Ensure that Microservices Infrastructure (CMSP) is enabled on Prism Central before you enable Flow Virtual Networking . See the Prism Central Guide for more information.

    For the procedure to enable Microservices Infrastructure (including enable in dark site), see Enabling Micro Services Infrastructure section in the Prism Central Guide .

    Note: When you configure microservices infrastructure, ensure that the DNS name you configure for CMSP does not end with test . Flow Virtual Networking does not support test as a top level domain. For example, the following are valid domain configurations:
    • my.cluster.domain
    • my.test.cluster.test.domain
    However, the following are examples of domains that Flow Virtual Networking does not support:
    • my.cluster.test
    • my.cluster.domain.test
  • Ensure that you have created a virtual IP address (VIP) for Prism Central. The Acropolis Upgrade Guide describes how to set the VIP for the Prism Central VM. Once set, do not change this address.

  • Ensure connectivity:

    • Between Prism Central and its managed Prism Element clusters.

    • To the Internet for connectivity (not required for dark site) to:

      • ECR for Docker images
      • S3 storage for LCM portal
      Note: For dark site deployments, Nutanix provides a dark site bundle, which has the Docker images (normally hosted on ECR) and the network controller package (normally hosted on LCM portal). These dark site bundles can be downloaded using an internet-connected system outside the dark site.
  • Prism Central backup, restore, and migration. You cannot perform these operations on MSP-enabled Prism Central.
  • Nutanix recommends increasing the MTU to 9000 bytes on the virtual switch vs0 and ensure that the physical networking infrastructure supports higher MTU values (jumbo frame support). The recommended MTU range is 1600-9000 bytes.

    Nutanix CVMs use the standard Ethernet MTU (maximum transmission unit) of 1,500 bytes for all the network interfaces by default. The system advertises the MTU of 1442 bytes to guest VMs using DHCP to account for the extra 58 bytes used by Generic Network Virtualization Encapsulation (Geneve). However, some VMs ignore the MTU advertisements in the DHCP response. Therefore, to ensure that Flow Virtual Networking functions properly with such VMs, enable jumbo frame support on the physical network and the default virtual switch vs0.

    If you cannot increase the MTU of the physical network, decrease the MTU of every VM in a VPC to 1442 bytes in the guest VM console.

    Note: Do not change the MTU of the CVM.
    Figure. Sample Configurations with and without Higher MTU - VS0, CVM and UVMs Click to enlarge

Requirements for Upgrades

The following applies to upgrades of Flow Virtual Networking network controller ( Advanced Networking in Prism Control Settings ):

  • Ensure that the Prism Central host is running an AHV version compatible with the networking controller upgrade version. If necessary, upgrade the AHV version using LCM to the version compatible with the network controller upgrade version.
    Note:

    See Compatibility and Interoperability Matrix on the Nutanix Support portal for AOS and Prism Central compatibility.

  • Ensure that all the AHV hosts in the AOS cluster are running the version compatible with the network controller upgrade version.

    The network controller upgrade fails if any of the AHV hosts is running an incompatible version.

Limitations

Limitations for Flow Virtual Networking are as follows.
  • Flow Virtual Networking does not support Flow Network Security for guest VMs.

    You cannot configure rules for Flow Network Security if a guest VM has any NICs connected to VPCs.

  • Flow Virtual Networking is supported only on AHV clusters. It is not supported on ESXi or Hyper-V clusters.

  • Flow Virtual Networking is not enabled on the new PE cluster registering with the Flow Virtual Networking -enabled Prism Central if the Prism Element cluster has an incompatible AHV version.

  • Flow Virtual Networking does not support updating a VLAN-backed subnet as an external subnet.

    You cannot enable the external connectivity option in the Update Subnet dialog box. Therefore, you cannot modify an existing VLAN-backed subnet to add external connectivity.

    VLAN backed subnets for external connectivity are managed by the Flow Virtual Networking control plane. Traditional AHV VLAN IPAM networks are managed by acropolis.

    Note: Do not configure the same VLAN as both a Flow Virtual Networking external network and an AHV IPAM network, as this can lead to IP address conflicts.
  • Flow Virtual Networking cannot be disabled if any external subnets and VPCs are in use. Delete the external subnets and VPCs and then disable Flow Virtual Networking .

  • Disaster Recovery backup and migration: CMSP-enabled Prism Central does not support disaster recovery backup and migration operations both as a source and target host.

Flow Virtual Networking Configurations

Enabling Flow Virtual Networking

Before you begin

Ensure that microservices infrastructure is enabled on Prism Central. See Enabling Micro Services Infrastructure section in the Prism Central Guide .

About this task

Before you proceed to enable Flow virtual networking by enabling the Advanced Networking option, see Prerequisites for Enabling Flow Virtual Networking.

To enable Advanced Networking, go to Prism Central Settings > Advanced Networking and do the following.

Procedure

  • In the Advanced Networking pane, click Enable .

    Ensure that the prerequisites specified on the pane are fulfilled.

    Figure. Enabling Flow Virtual Networking Click to enlarge Displaying the Advanced Networking page.

    Prism Central displays the deployment in-progress.
    Figure. Deployment Progress Click to enlarge Displaying the Deployment Progress.

  • Flow virtual networking is enabled.
    Figure. Flow Virtual Networking Status Click to enlarge Displaying the enabled status of Flow virtual networking.

Disabling Flow Virtual Networking

About this task

You can disable Flow virtual networking. However, the network controller cannot be disabled if any external subnets and VPCs are in use. Delete the subnets and VPCs before you disable advanced networking.

Note:

Flow virtual networking cannot be disabled if any external subnets and VPCs are in use. Delete the external subnets and VPCs and then disable Flow virtual networking.

To disable Flow virtual networking, do the following.

Procedure

  1. On the Advanced Networking page, click Disable .
    Figure. Click to enlarge Displaying the highlighted Disable Advanced Networking link.

  2. On the confirmation message box, click Confirm to confirm disablement.

    To exit without disabling the Advanced Networking controller, click Cancel .

Unregistering a PE from the PC

Before unregistering a Prism Element from PC, disable Flow virtual networking on that Prism Element using network controller CLI (or atlas_cli).

About this task

When Flow virtual networking is enabled on a Prism Central, it propagates the capability to participate in VPC networking to all the registered Prism Elements that are running the required AHV version.

In cases where there are VMs on the Prism Element attached to the VPC network, or if the Prism Element is used to host one or more of the external VLAN networks attached to a VPC, Prism Central alerts you with a prompt. When being alerted about the aforementioned conditions, close the CLI and make adequate configuration to resolve the condition (for example, select a different cluster for the external VLAN network and delete the VMs attached to the VPC network running on the Prism Element). After making such configurations, execute the network controller CLI to disable Flow virtual networking. If the command goes through successfully, it is safe to unregister the Prism Element.

For example, in a deployment of three Prism Elements - PE1, PE2 and PE3 - registered to the Flow virtual networking-enabled PC, you want to unregister PE3 from the PC. You must first disable Flow virtual networking using the following steps:

Procedure

  1. SSH to PE3.
  2. Run the ncli cluster info or ncli cluster get-params command to get the cluster parameters.
    Copy the cluster UUID (For example: 017457d3-1012-465c-9c54-aa145f2da7d9) from the displayed cluster parameters.
  3. SSH to the Prism Central VM.
  4. Open the network controller console by executing the atlas_cli command.
    nutanix@cvm$ atlas_cli
    <atlas> 
  5. Execute the config.add_to_excluded_clusters <cluster uuid> command, providing the cluster UUID that you copied earlier.

    An example of the PC alert, for the condition that PE3 VM is attached to an external network, is as follows:

    <atlas> config.add_to_excluded_clusters 0005bf8d-2a7f-3b2e-0310-d8e34995511e 
    Cluster 0005bf8d-2a7f-3b2e-0310-d8e34995511e has 1 external subnet, 
    which will lose connectivity. Are you sure? (yes/no)
    Note: To enable Flow virtual networking on the cluster, execute the config.remove_from_excluded_clusters <cluster uuid> command, providing the cluster UUID.

What to do next

To verify if Flow virtual networking is disabled or enabled, SSH to PE3 and run the acli atlas_config.get command.

The output displays the enable_atlas_networking parameter as False if Flow virtual networking is disabled and as True if Flow virtual networking is enabled on the Prism Element.

nutanix@cvm$ acli atlas_config.get
config {
  anc_domain_name_server_list: “10.10.10.10”
  enable_atlas_networking: False
  logical_timestamp: 19
  minimum_ahv_version: “20190916.101588"
  ovn_cacert_path: “/home/certs/OvnController/ca.pem”
  ovn_certificate_path: “/home/certs/OvnController/OvnController.crt”
  ovn_privkey_path: “/home/certs/OvnController/OvnController.key”
  ovn_remote_address: “ssl:anc-ovn-external.default.anc.aj.domain:6652"
}

You can now unregister the PE from the PC.

Upgrading Flow Virtual Networking

You can upgrade the Flow virtual networking controller ( Advanced Networking Controller in Prism Central Settings ) using Life Cycle Manager (LCM) on Prism Central.

Before you begin

See Prerequisites for Enabling Flow Virtual Networking.

In case of upgrading the Flow virtual networking controller in a dark site, ensure that LCM is configured to reach the local web server that hosts the dark site upgrade bundles.

Note:

The network controller upgrade fails to start after the pre-check if one or more clusters have Flow virtual networking enabled and are running an AHV version incompatible with the new network controller upgrade version.

About this task

To upgrade the network controller using LCM, do the following.

Procedure

  1. Choose one of the following ways to reach the LCM page:
    • Go to Administration > LCM > Inventory
    • Click Check for Updates on the Advanced Networking page.

    Figure. Check for Updates Click to enlarge Displaying Check for Updates link on the Advanced Networking page.

  2. Click Perform Inventory .

    When you click Perform Inventory , the system scans the registered Prism Central cluster for software versions that are running currently. Then it checks for any available upgrades and displays the information on the LCM page under Software .

  3. Go to Updates > Software . Select the Advanced Networking Controller version you want to upgrade to and click Update .
    Figure. Networking Controller version Click to enlarge Displaying sample LCM dashboard with the available Advanced Networking Controller upgrade available

Deploying Flow Virtual Networking at a Dark Site

About this task

Dark sites are primarily on-premises installations which do not have access to the internet. Such sites are disconnected from the internet for a range of reasons including security. To deploy Flow virtual networking at such dark sites, you need to deploy the dark site bundle at the site.

This dark site deployment procedure includes downloading and deploying MSP and the network controller bundles.

Before you begin

  • See Prerequisites for Enabling Flow Virtual Networking.

  • You need access to the Nutanix Portal from an Internet-connected device to download the following dark site bundles:

    Note: For dark site deployments, Nutanix provides a dark site bundle, which has the Docker images (normally hosted on ECR) and the network controller package (normally hosted on LCM portal). These dark site bundles can be downloaded using an internet-connected system outside the dark site.
    • MSP dark site bundle: https://portal.nutanix.com/page/downloads/list > Microservices Platform (MSP)
    • Flow virtual network controller dark site bundle: See the Flow Virtual Networking Release Notes for the link to download the dark site bundle.
    • Network Gateway bundle: See the Flow Virtual Networking Release Notes for the link to download the dark site bundle with checksum text file. Also, see KB-12393 .

To deploy Flow virtual networking at a dark site, do the following.

Procedure

  1. Start a web server to host the dark site bundles and act as a source for the LCM downloads, if one is not already created.

    The web server can be a virtual machine on a cluster at the dark site. All the Prism Central VMs at the dark site must have access to this web server. This web server is used when you deploy any dark site bundle including the network controller darksite bundle.

    For more information about the server installation, see:

    • Linux web server

    • Windows web server

  2. In Prism Central, go to Administration > LCM > Inventory .

    Alternatively, SSH into the Prim Central VM as an admin user and run the following command.

    admin@pcvm$ mspctl controller airgap enable --url=http://<LCM-web-server-ip>/release

    Where <LCM-web-server-ip> is the IP address of the LCM web server and release is the name of the directory where the packages were extracted.

    For example, admin@pcvm$ mspctl controller airgap enable --url=http://10.48.111.33/release . Here, 10.48.111.33 is the IP address of the LCM web server and release is the name of the directory where the packages were extracted.

  3. Verify the configuration by running the following command:
    nutanix@cvm$ mspctl controller airgap get
  4. From a device that has public Internet access, click the Nutanix Compatibility Bundle link and down the bundle. Transfer this bundle to the LCM web server and extract the contents.
  5. From a device that has public Internet access, Nutanix recommends that you download and extract the latest MSP dark site bundle, transfer it to the LCM web server, and extract the contents.
  6. From a device that has public Internet access, download the Flow virtual networking dark site bundle (see Release Notes | Flow Virtual Networking for download links). Transfer the bundle to the LCM web server.
  7. Extract or unpack the Flow Virtual Networking dark site bundle on the LCM web server.

    After unpacking, check if the system shows a directory path that includes the following as per the example: http://<LCM-web-server-ip>/release/builds/msp-builds/msp-services/464585393164.dkr.ecr.us-west-2.amazonaws.com/nutanix-msp/atlas-hermes/ .

  8. Run the following command after unpacking to ensure that the file permissions are not disrupted during the unpacking:
    • Linux.
      chmod -R +r builds
    • Windows NTFS.
      
      $> takeown / R / F *
      $> icacls <Build-file-path> /t /grant:F 
      .
  9. Enable microservices infrastructure.

    See the Enabling Microservices Infrastructure section in the Prism Central Guide for details.

  10. Enable Flow virtual networking. See Enabling Flow Virtual Networking.

Troubleshooting Tips

This section provides information to assist troubleshooting of Flow virtual networking deployments. This is in addition to the information that the "Prism Central Guide" provides.

Audit Logs

Prism Central generates audit logs for all the flow networking activities like it does for other activities on Prism Central. See Audit Summary View in the Prism Central Guide , for more information about Audit log.

Support Bundle Collection

To support troubleshooting for Flow virtual networking, you can collect logs.

To collect the logs, run the following commands on the Prism Central VM console:

nutanix@cvm$ logbay collect -t msp,anc

An example of the command is as follows:

nutanix@cvm$ logbay collect -t msp,anc -O msp_pod=true,msp_systemd=true,kubectl_cmds=true,persistent=true --duration=-48h0m0s

Where:

  • -t flag indicates the tags to collect

    • msp tag will collect logs from the services running on MSP pods and persistent log volumes (application-level logs)

    • anc tag will collect the support bundle, which includes database dumps and OVN state

  • -O flag adds tag-level options

    • msp_pod=true collects logs from MSP service pods

      On the PC, these logs can be found under /var/log/containers .

    • persistent=true collects persistent log volumes (application-level logs for ANC)

      On the PC, these can be found under /var/log/ctrlog

    • kubectl_cmds=true runs kubectl commands to get the Kubernetes resource state

  • --duration sets the duration from the present to collect

The command run generates a zip file at a location, for example: /home/nutanix/data/logbay/bundles/<filename>.zip

Unzip the bundle and you'll find the anc logs under a directory specific to your MSP cluster, the worker VM where the pod is running, and the logging persistent volume of that pod. For example:

./msp/f9684be8-b4e8-4524-74b4-076ed53ca1fd/10.48.128.185__worker_master_etcd/persistent/default/ovn/anc-ovn_StatefulSet/

For more information about the task run, see the text file that the command generates at a location, for example: /home/nutanix/data/logbay/taskdata/<taskID>/collection_result.txt

For more information about the logbay collect command, see the Logbay Log Collection (Command Line) topic in the Nutanix Cluster Check Guide (NCC Guide).

Layer 2 Virtual Subnet Extension Alert

The L2StretchLocalIfConflict alert (Alert with Check ID - 801109) may occur while performing Layer 2 virtual subnet extensions. See KB-10395 for more information about its resolution.

Network Gateway Upgrades

Nutanix deployment can detect and install upgrades for the onprem Nutanix Gateways.

For information about identifying the current Nutanix Gateway version, see Identifying the Gateway Version.

For onprem Nutanix Gateways, the upgrades need to be detected and installed on the respective PC on which each Nutanix Gateway is installed.

For more information, see Detecting Upgrades for Gateways.

When Prism Central detects the upgrades, it displays a banner on the Gateways tab of the Connectivity page. The banner notifies you that a Gateway upgrade is available after you have run LCM inventory. The table on the Gateways tab also displays an alert (exclamation mark) icon for the network gateways that the upgrade applies to. The hover message for the icon informs you that an upgrade is available for that Gateway.

Figure. Upgrade Banner Click to enlarge Displaying sample VPN Gateway tab.

For more information about the upgrade procedure, see Upgrading the PC-managed Onprem Nutanix VPN Gateways.

Identifying the Gateway Version

About this task

To identify the current Nutanix Gateway version, do the following:

Procedure

  • Click the hamburger icon and Networking & Security > Connectivity .
  • On the Gateways tab, click the Gateway name link text to open the Gateway details page.

    In the Gateway table, the VPN Gateway name is a clickable link text.

    The Gateway Version is listed in the Properties widget.

    Figure. Gateway Version Click to enlarge Displays sample VPN Gateway details page with clickable version number.

Detecting Upgrades for Gateways

About this task

Prism Central can detect whether new Gateway upgrades are available, or not, for Nutanix Gateways using LCM. You can then install the upgrade.

Procedure

  • Click the hamburger icon of Dashboard .
  • Click Administration > LCM > Inventory .
  • Click Perform Inventory .
    Note:

    Nutanix recommends that you select Enable LCM Auto Inventory in the LCM page in Prism Central to continuously detect new Gateway upgrades as soon as they are available.

    The upgrade notification banner is displayed on the Gateways page.

Upgrading the PC-managed Onprem Nutanix VPN Gateways

About this task

Perform upgrades of PC-managed Nutanix Gateways using the respective PC on which the Gateway is created.

To upgrade the on-prem Nutanix Gateways, do the following:

Procedure

  1. Log on to the Prism Central as the admin user and click the gear icon.
  2. Go to Administration > LCM > Inventory .
  3. Click Perform Inventory .

    When you click Perform Inventory , the system scans the registered Prism Central cluster for software versions that are running currently. Then it checks for any available upgrades and displays the information on the LCM page under Software .

    Note:

    Skip this step if you have enabled auto-inventory in the LCM page in Prism Central.

  4. Go to Updates > Software . Select the Gateway version you want to upgrade to and click Update .

    LCM upgrades the Gateway version. This process takes sometime.

Network and Security View

The Network and Security category in the Entities Menu expands on-click to display the following networking and security entities that are configured for the registered clusters:

  1. Subnets : This dashboard displays the subnets and the operations you can perform on subnets.

  2. Virtual Private Clouds : This dashboard displays the VPCs and the operations you can perform on VPCs.

  3. Floating IPs : This dashboard displays a list of floating IP addresses that you are using in the network. It allows you to request for floating IP addresses from the free pool of I addresses available to the clusters managed by the Prism Central instance.

  4. Connectivity : This dashboard allows you to manage the following networking capabilities:

    • Gateways : This tab provides a list of network Gateways you have created and configured, and the operations you can perform on the network Gateways. You can check and upgrade the Gateway bundle in Administration > LCM > Inventory .

    • VPN Connections : This tab provides a list of VPN connections you have created and configured, and the operations you can perform on VPN connections.

    • Subnet Extensions : This tab provides a list of subnets that you have extended at the Layer 2 level using VPN (point-to-point over Nutanix VPN) or VTEP (point-to-multi-point including third party).

  5. Security Policies : This dashboard provides a list of security policies you configured using Flow Segmentation. For more information about Security Policies, see the Flow Microsegmentation Guide.

See Network Connections section for information on how to configure network connections.

Subnets (Overlay IP subnets), Virtual private clouds, floating IPs, and Connectivity are Flow virtual networking features. These features support flexible app-driven networking that focuses on VMs and applications instead of virtual LANs and network addresses. Flow virtual networking powers network virtualization to offer a seamless network experience with enhanced security. It is disabled by default. It is a software-defined network virtualization solution providing overlay capabilities for the on-premises AHV clusters.

Security policies drives the Flow Segmentation features for secure communications. See Flow Microsegmentation Guide.

Subnets

Manage subnets in the List view of Subnets dashboard in the Network and Security section.

To access the Subnets dashboard, select Subnets from the entities menu in Prism Central. The Subnets dashboard allows you to view information about the subnets configured for the registered clusters.

Note: This section describes the information and options that appear in the Network and Security dashboard. See Entity Exploring for instructions on how to view and organize that information in a variety of ways.
Figure. Subnets Dashboard Click to enlarge sample Subnets dashboard

The following table describes the fields that appear in the subnets list. A dash (-) is displayed in a field when a value is not available or applicable.

Table 1. Subnets Dashboard Fields
Parameter Description Values
Name Displays the subnet name. (subnet name)
External Connectivity Displays whether or not the subnet has external connectivity configured. (Yes/No)
Type Displays the subnet type. VLAN
VLAN ID Displays the VLAN identification number. (ID number)
VPC Displays the name of the VPC that the Subnet is used in. (Name of VPC)
Virtual Switch Displays the virtual switch that is configured for the VLAN you selected. The default value is the default virtual switch vs0 .
Note: The virtual switch name is displayed only if you add a VLAN ID in the VLAN ID field.
(virtual switch name)
IP Prefix Displays the IPv4 Address of the network with the prefix. (IPv4 Address/Prefix)
Cluster Displays the name of the cluster for which this subnet is configured. (cluster name)
Hypervisor Displays the hypervisor that the subnet is hosted on. (Hypervisor)

To filter the list by network name, enter a string in the filter field. (Ignore the Filters pane as it is blank.)

To view focused fields in the List, select the focus parameter from the Focus drop down list. You can create your own customised focus parameters by selecting Add custom from the drop down list and selecting the necessary fields after providing a Name , in the Subnet Columns .

There is a Network Config action button to configure a new network (see Configuring Network Connections)

The Actions menu appears when one or more networks are selected and includes a Manage Categories option (see Assigning a Category).

Go to the Subnets list view by clicking Network and Security > Subnets on the left side-bar.

Figure. Subnets Page Click to enlarge

To view or select actions you can perform on a subnet, select the subnet and click the Actions dropdown.

Figure. Subnet Actions Click to enlarge

Table 2. Subnet Actions
Action Description
Update Click this action to update the selected subnet. see Updating a Subnet.
Manage Extension Click this action to create a subnet extension. A subnet extension allows VMs to communicate over the same broadcast domain to a remote Xi availability zone (in case of Xi-Leap based disaster recovery) via the extension.
Manage Categories Click this action to associate the subnet with a category or change the categories that the subnet is associated with.
Delete Click this action to delete the selected subnet. See Deleting Subnets, Policies, or Routes.

You can also filter the list of subnets by clicking the Filters option and selecting the filtering parameters.

Subnet Summary View

View the details of a subnet listed on the Subnets page.

To view the details of a subnet, click the name of the subnet on the subnet list view.

Figure. Subnet Summary Page Click to enlarge Displaying sample subnets Summary view

The Summary page provides buttons for the actions you can perform on the subnet, at the top of the page. Buttons for the following actions are available: Update , Extend , Manage Categories , and Delete .

The subnet Summary page has the following widgets:

Widget Name Information provided
Subnet Details Provides the following:
  • Type — Displays the type of network like VLAN or Overlay.
  • VLAN ID — Displays the VLAN ID. This parameter is displayed only for VLAN networks.
  • VPC — Displays the VPC name. This parameter is displayed only for Overlay networks.
  • Cluster — Displays the cluster that the VLAN network is configured on. This parameter is displayed only for VLAN networks.
  • IP Prefix — Displays the IP address prefix configured for the network. This parameter is displayed for both VLAN and Overlay networks.
IP Pool Provides the IP address Pool Range assigned to the network.
External Connectivity Provides the following:
  • NAT — Displays whether NAT is enabled or disabled for VPCs connecting to the network. When you hover on the Enabled / Disabled status, the hover message displays details of VPCs connected to the external subnet.
  • Associated VPCs — Displays the VPCs associated with this external subnet.

Virtual Private Clouds

You can manage Virtual Private Clouds (VPCs) on the Virtual Private Clouds dashboard.

Go to the Virtual Private Clouds dashboard by clicking Network and Security > Virtual Private Clouds on the left side-bar.

Figure. Virtual Private Clouds dashboard Click to enlarge

You can configure the table columns for the VPC list table. The available column list includes Externally Routable IP Addresses that provides address space within the VPC that is reachable externally without NAT.. For the list of columns that you can add to the list table, see Customizing the VPC List View.

Note:

Ensure that the externally routable IP addresses (subnets with external connectivity without NAT) for different VPCs do not overlap.

Configure the routes for the external connectivity subnets with next hop as the Router or SNAT IP address. Also configure the routes on the router for the return traffic to reach the VPC. See External Connectivity panel in VPC Details View.

To view or select actions you can perform on a VPC, select the VPC and click the Actions drop down.

You can also filter the list of VPC by clicking the Filters option and selecting the filtering parameters.

Customizing the VPC List View

About this task

You can customize the columns in the table. Click the View by drop down and select + Add custom .

In the Virtual Network Columns dialog box, do the following.

Procedure

  1. Enter a name for the view.
  2. Select the columns you want displayed in the table.

    During the column selection, the columns you select are moved under the Selected Columns list. The Name (of the VPC) column is the default column already selected. You can add a maximum of 10 columns (including the Name column) to the Selected Column list.

    Figure. Customizing Columns in VPC View Click to enlarge

    To arrange the order of the selected columns, hover on the column name and click the up or down arrow button as appropriate.

  3. Click Save .

VPC Details View

To view the details of a VPC, click the name of the VPC on the VPC list view.

The VPC details view has the following tabs:

  • Summary
    Figure. Summary Tab Click to enlarge Displaying the Summary tab in the VPC dashboard

    The Summary tab provides the following panes:

    • DNS Servers—Provides more information about the DNS Servers used by the VPC.
    • External Connectivity—Provides the name of the external subnet, NAT Gateway host details, router/SNAT IP address and the IP address spaces or ranges configured for the VPC.
    • Floating IP Addresses—Provides details of the floating IP addresses that the VPC uses.
  • Subnets
    Figure. Subnet Tab Click to enlarge Displaying the Subnet tab in the VPC dashboard

    The Subnet tab provides the following information for the subnets:

    • Name—Displays the name of the subnet.
    • IP Range—Displays the IP address range configured for the subnet.
    • DHCP IP Pool—Displays the DHCP IP address pool configured for the subnet.
    • Default Gateway IP—Displays the IP address used as the default gateway by the entities in the subnet.
    • Actions—Displays the actionable links to Edit or Delete the subnet.
  • Policies
    Figure. Policies Tab Click to enlarge Displaying the Policy tab in the VPC dashboard

    The Policies tab maps the following information about the security-based traffic shaping policies you configure:

    • Priority—The traffic priority.
    • Rule—The Allow or Deny rule set for the priority.
    • Traffic—The traffic type that the priority and rule should be applied to.
    • Actions—Actions you can take on the policy. You can perform three actions: Clear counters , Edit the policy or Delete the policy.
  • Routes
    Figure. Routes Tab Click to enlarge Displaying the Router tab in the VPC dashboard

    The Routes tab provides the following information about the routes:

The VPC details view has the following configuration options for the VPC:

  • Update : Use this option to update the VPC. For more information, see Updating Virtual Private Cloud.
  • Add Subnet : Use this option to add a subnet to the VPC. For more information, see Creating a Subnet.
  • Create Static Routes : Use this option to create a static route. For more information, Creating Static Routes.
  • Update Static Routes : Use this option to update static route configurations that you already created. For more information, see Updating Static Routes.
  • Create Policy : Use this option to create traffic policies in addition to the pre-configured default policy. When you create a VPC, there is one default policy that Advanced Networking creates for the VPC. This policy is pre-configured and cannot be edited. For more information, see Creating a Policy.
  • Clear All Counters : Allows you to clear all the counters for the VPC.
  • Delete : Allows you to delete the VPC. For more information, see Deleting a Virtual Private Cloud.

Floating IPs

You can access floating IPs on the Floating IPs dashboard or list view in the Network and Security section.

For information about floating IP addresses and their role in Flow virtual networking, see SNAT and Floating IP Address.

Go to the Floating IPs dashboard by clicking Network and Security > Floating IPs on the left side-bar.

Figure. Floating IPs dashboard Click to enlarge Displaying the Floating IP dashboard

To view or select actions you can perform on a floating IP address assigned, select the floating IP address and click the Actions drop down. The following actions are available for a selected floating IP address:

  • Update—Assign or change the assignment of the floating IP address. You can assign the floating IP address to a IP address such as a private IP address in a VPC or the primary IP address of a VM or a secondary IP address created on a VM.
  • Delete—Delete the floating IP address. The deleted IP address returns to the IP address pool as unused. Before you delete, ensure that it is not assigned to a private IP address or a VM. Change the assignment to None if it is already assigned, using the Update action.
Note: Floating IP addresses are not reachable (Pings fail) unless you associate them to primary or secondary IP addresses of VMs. For more information about assigning floating IP addresses to secondary IP addresses of VMs, see Assigning Secondary IP Addresses to Floating IPs .

To filter the list of floating IP address assignments, click the Filters option and select the appropriate filtering parameters.

To request floating IP addresses, see Requesting Floating IPs.

Connectivity

You can access network Gateways, VPN connections and subnet extensions on the Connectivity dashboard.

Click Network & Security > Connectivity to see the Connectivity dashboard.

The Connectivity dashboard opens on the Gateways tab. To see the VPN connections, click the VPN Connections tab. To see the subnets extended across AZs, click the Subnet Extensions tab.

Gateways Summary View

The Connectivity dashboard opens on the Gateways dashboard or summary view.

The Gateway dashboard provides a list of gateways created for the clusters managed by the Prism Central.

The Gateways dashboard provides a Create Gateway dropdown menu that lets you create a Local or a Remote gateway. You can create a local or remote gateway with VPN or VTEP service. For more information, see Creating a Network Gateway.

You can select a gateway from the list (select the checkbox provided for the gateway) and then perform an action provided in the Actions dropdown list. The Actions dropdown list allows you to Update or Delete the selected gateway.

Figure. Gateways dashboard Click to enlarge Displaying the Connectivity dashboard with the Gateways dashboard

The Gateway summary list view provides the following details about the gateway.

Table 1. Gateway List Fields
Parameter Description Values
Name Displays the name of the gateway. (Name of gateway)
Type Displays the gateway type. (Local or Remote)
Service Displays the service that the gateway uses. (VPN or VTEP)
Service IP Displays the IP address used by the service. (IP address)
Status Displays the operational status of the gateway. (Up or Down)
Attachment Type/Vendor Displays the type of subnet associated with the gateway. (VLAN or Overlay-VPC name)
Connections Displays the number of service connections (such as VPN connections) configured and operational on the gateway. (number)

You can click the name of a gateway to open the gateway details page that presents the information about the gateway in widgets.

Gateway Details View

You can click the name of a gateway in the Gateway dashboard list to open the gateway details page that presents the information about the gateway in widgets.

The gateway details page displays the name of the gateway on the top left corner.

  • On the top right corner, the close button (X) allows you to close the details page.

  • The Update button opens the Update Gateway page. For more information, see Updating a Network Gateway in Flow Networking Guide .

  • The Delete button allows you to delete the gateway. For more information, see Deleting a Network Gateway in Flow Networking Guide .

Figure. Gateway Details View Click to enlarge Displays the gateway details page that provides details of the gateway in two widgets - Properties and Service configuration

The details about the gateway are organized in widgets as follows:

Table 1. Gateway Details
Parameter Description Values
Properties widget
Type Displays the gateway type. (Local or Remote)
Attachment Type Displays the network entity like VLAN or VPC that the gateway is attached to. (VLAN or VPC)
VPC or Subnet (VLAN) Displays the name of the attached VPC or VLAN subnet. (Name of VLAN or VPC)
Floating or Private IP Address Displays the Floating (for VPC) or Private (for VLAN) IP address assigned to the gateway. (IP Address)
Status Displays the operational status of the gateway. (Up or Down)
Gateway Version Displays the version of the Nutanix gateway appliance deployed. (Version)
Cluster Displays the name of the cluster on which the gateway is created. (Cluster name)
Gateway VM Displays the name of the VM on which the gateway is created. (Name of VM - actionable link. Click the name-link to open the VM details page of the gateway VM.)
Service Configuration
Service Displays the service used by the gateway. (VPN or VTEP)
External Routing Displays the type of routing associated with the gateway for external traffic routing. (Static or eBGP with ASN)
Internal Routing Displays the type of routing associated with the gateway for internal traffic routing. (Static or eBGP with ASN)
VPN Connections Displays the total number of VPN connections associated with the gateway. (Number - actionable link. Click the link to open the VPN connection details page for the associated VPN connection.)
View VPN Connections Click this link to open the VPN Connections tab. -

VPN Connections Summary View

The Connectivity dashboard allows you to open the VPN Connections dashboard or summary view.

VPN Connection: Represents the VPN IPSec tunnel established between local gateway and remote gateway. When you create a VPN connection, you need to select two gateways between which you want to create the VPN connection.

The VPN Connections dashboard provides a list of VPN connections created for the clusters managed by the Prism Central.

The VPN Connections dashboard provides a Create VPN Connection button that opens the Create VPN Connection . For more information, see Creating a VPN Connection in Flow Networking Guide .

You can select a VPN connection from the list (select the checkbox provided for the VPN connection) and then perform an action provided in the Actions dropdown list. The Actions dropdown list allows you to Update or Delete the selected VPN connection.

The VPN Connections summary list view provides the following details about the VPN connection.

Figure. VPN Connections dashboard Click to enlarge Displaying the VPN Connections dashboard.

Table 1. VPN Connections List Fields
Parameter Description Values
Name Displays the name of the connection. (gateway name)
IPSec Status Displays the connection status of IPSec tunnel. (Connected or Not Connected)
EBGP Status Displays the status of the EBGP gateway connection. (Established or Not Established)
Local Gateway Displays the name of the local gateway used for the connection. (Name of local gateway)
Remote Gateway Displays the name of the remote gateway used for the connection. (Name of remote gateway)
Dynamic Routing Priority Displays the dynamic routing priority assigned to the connection for throughput management. You can assign any value in the range of 100-1000. Flow Virtual Networking assigns the first VPN connection the value 500 by default. Thereafter, subsequent VPN connections are assigned values decremented by 50. For example, the first connections is assigned 500, then the second connection is assigned 450, the third one 400 and so on. (Number in the range of 100-1000. User assigned.)

VPN Connections Details View

You can click the name of a VPN connection in the VPN Connections dashboard list to open the VPN connection details page that presents the information about the VPN connection in widgets.

The VPN connection details page displays the name of the VPN connection on the top left corner.

  • On the top right corner, the close button (X) allows you to close the details page.

  • The Update button opens the Update VPN Connection page. For more information, see Updating a VPN Connection in Flow Networking Guide .

  • The Delete button allows you to delete the VPN connection. For more information, see Deleting a VPN Connection in Flow Networking Guide .

Figure. VPN Connection Details Click to enlarge Displaying the detailed view of the selected VPN connection with the information organized in widgets.

The details about the VPN connection are organized in widgets as follows:

  • Summary tab—See the VPN Connection Summary Tab Details table below.
  • Throughput tab—See the VPN Connection Throughput Tab Details table below.
  • IPSec Logging tab—Provides logs for the IPSec tunnel.
  • Routing Protocol Logging tab—Provides logs for the routing protocol used in the VPN connection.
Table 1. VPN Connection Summary Tab Details
Parameter Description Values
VPN Connection widget
IPSec Status Displays the connection status of IPSec tunnel. (Connected or Not Connected)
EBGP Status Displays the status of the EBGP gateway connection. (Established or Not Established)
Dynamic Routing Priority Displays the dynamic routing priority assigned to the connection for throughput management. You can assign any value in the range of 100-1000. Flow Virtual Networking assigns the first VPN connection the value 500 by default. Thereafter, subsequent VPN connections are assigned values decremented by 50. For example, the first connections is assigned 500, then the second connection is assigned 450, the third one 400 and so on. (Number in the range of 100-1000. User assigned.)
Local Gateway Properties
Gateway Name Displays the name of the local gateway used for the connection. (Name of local gateway)
Type Displays the type of gateway. (Local)
Attachment Type Displays the network entity like VLAN or VPC that the gateway is attached to. (VLAN or VPC)
VPC or Subnet (VLAN) Displays the name of the attached VPC or VLAN subnet. (Name of VLAN or VPC)
Tunnel IP Displays the Tunnel IP address of the local gateway. (IP Address)
Connection Type Displays the connection type you selected while creating the VPN connection. The connection type may be Initiator or Acceptor of a VPN connection between the local and remote gateways. T (Initiator or Acceptor)
External Routing Displays the type of routing associated with the gateway for external traffic routing. (Static or eBGP with ASN)
Internal Routing Displays the type of routing associated with the gateway for internal traffic routing. (Static or eBGP with ASN)
Floating or Private IP Address Displays the Floating (for VPC) or Private (for VLAN) IP address assigned to the gateway. (IP Address that you assigned to the local gateway with /30 prefix when you configured the VPN connection.)
Status Displays the operational status of the gateway. (Up or Down)
Cluster Displays the name of the cluster on which the gateway is created. (Cluster name)
Gateway VM Displays the name of the VM on which the gateway is created. (Name of VM - actionable link. Click the name-link to open the VM details page of the gateway VM.)
Remote Gateway Properties
Gateway Name Displays the name of the remote gateway used for the connection. (Name of remote gateway)
Type Displays the type of gateway. (Remote)
Tunnel IP Displays the Tunnel IP address of the remote gateway. (IP Address)
Connection Type Displays the connection type you selected while creating the VPN connection. The connection type may be Initiator or Acceptor of a VPN connection between the local and remote gateways. T (Initiator or Acceptor)
External Routing Displays the type of routing associated with the gateway for external traffic routing. (Static or eBGP with ASN)
ASN Displays the ASN of the EBGP route. This information is only displayed if you configured EBGP as the External Routing protocol. (Number)
Vendor Displays the name of the vendor of the gateway appliance at the remote site. (Name of vendor of gateway appliance)
External IP Displays the IP address assigned to remote the gateway. (IP Address that you assigned to the remote gateway with /30 prefix when you configured the VPN connection.)
Status Displays the operational status of the gateway. -
Protocol Details
Service Displays the service used by the gateway. (VPN or VTEP)
Gateway Routes Displays the status of the routes used by the gateways. (Sent)

Subnet Extensions Summary View

The Connectivity dashboard opens on the Subnet Extensions dashboard or summary view.

This Guide does not provide detailed information about Security Policies . It only describes the Summary and Details views for the same. For detailed information about Security Policies , see Flow Microsegmentation Guide

The Subnet Extensions dashboard provides a list of subnet extensions created for the clusters managed by the Prism Central.

The Subnet Extensions dashboard provides a Create Subnet Extension dropdown menu that lets you extend a subnet Across Availability Zones or To a Third Party Data Center . You can extend a subnet using VPN or VTEP service. See Layer 2 Virtual Network Extension for more information.

You can select a subnet extension from the list (select the checkbox provided for the subnet extension) and then perform an action provided in the Actions dropdown list. The Actions dropdown list allows you to Update or Delete the selected subnet extension.

Figure. Subnet Extensions dashboard Click to enlarge Displaying the Subnet Extension dashboard.

The Subnet Extensions summary list view provides the following details about the gateway.

Table 1. Subnet Extensions List Fields
Parameter Description Values
Name Displays the name of the subnet extension. (Name of subnet extension)
Type Displays the subnet extension type. ( Across Availability Zones or To a Third Party Data Center )
Extension Over Displays the service that the subnet extension uses. (VPN or VTEP)
Extension Uses Displays the name of the local network gateway that the subnet extension uses. (Name of local network gateway)
Local Subnet Displays the name of the local subnet that the subnet extension uses. (Name of local subnet)
Remote Site Displays the name of the remote network gateway that the subnet extension uses. (Name of remote network gateway)
Connection Status Displays the status of the connection that is created by the subnet extension. Not Available status indicates that Prism Central is unable to ascertain the status. (Not Available, Connected, or Disconnected)
Interface Status Displays the status of the interface that is used by the subnet extension. (Connected or Down)

You can click the name of a subnet extension to open the subnet extension details page that presents the information about the subnet extension in widgets.

Subnet Extensions Details View

You can click the name of a subnet extension in the Subnet Extensions dashboard list to open the subnet extension details page that presents the information about the subnet extension in widgets.

The subnet extension details page displays the name of the subnet extension on the top left corner. It has two tabs - Summary , Address Table and Throughput .

Summary

The Summary tab provides the information about the subnet extension in widgets. It also allows you to take the following actions:

  • On the top right corner, the close button (X) allows you to close the details page.

  • The Update button opens the Update Subnet Extension page. See Updating an Extended Subnet for more information.

  • The Delete button allows you to delete the subnet extension. See Removing an Extended Subnet for more information.

Figure. Subnet Extensions Details View - Summary tab sample for VTEP extension Click to enlarge Displays the subnet extension details page, Summary that provides details of the subnet extension in one extended widget with three sections - Properties, IP Address Pools and Subnet Extension properties.

Table 1. Subnet Extension Details - Summary Tab Fields
Parameter Description Values
Properties
Type Displays the subnet type. (VLAN or Overlay)
VLAN ID (For VLAN subnets only) Displays the VLAN ID of the VLAN subnet that is extended. (VLAN ID number)
VPC (For Overlay subnets only) Displays the name of the VPC subnet that is extended. (Name of VPC)
Cluster (For VLAN subnets only) Displays the cluster that the VLAN subnet belongs to. (Name of cluster)
IP Address Prefix Displays the network IP address with prefix, of the VLAN subnet that is extended. (IP Address with prefix)
Virtual Switch (For VLAN subnets only) Displays the virtual switch on which the VLAN subnet is configured. (Virtual Switch name such as vs0 or vs1)
IP Address Pools
Pool Range Displays the range of IP addresses in the pool configured in the subnet that is extended. (IP address range)
(Interactive Graphic Pie Chart) Displays a dynamic pie chart that displays the statistic you hover on. Displays the following IP address statistics outside the pie chart, that you can hover on:
  • Total number of IP addresses available.
  • Used IP addresses in the subnets
  • Used IP addresses in the IP address pools
  • Free IP addresses in the subnets
  • Free IP addresses in the IP address pools
(IP Address statistics)
Subnet Extension
Subnet Extension (properties) - Common
Type Displays the subnet extension type. ( Across Availability Zones or To a Third Party Data Center )
Interface Status Displays the status of the interface that is used by the subnet extension. (Connected or Down)
Connection Status Displays the status of the connection that is created by the subnet extension. Not Available status indicates that Prism Central is unable to ascertain the status. (Not Available, Connected, or Disconnected)
Local IP Address Displays the IP address that you entered in the Local IP Address field while creating the subnet extension. (IP Address)
Local Subnet Displays the name of the local subnet that the subnet extension uses. (Name of local subnet)
Subnet Extension (properties) - (Only for Across Availability Zones type)
Local Availability Zone (Only for Across Availability Zones type) Displays the name of the local AZ that is hosting the subnet that is extended. (Name of the local Availability Zone)
Remote Availability Zone (Only for Across Availability Zones type) Displays the name of the remote AZ that the subnet is extended to. (Name of the remote Availability Zone)
Remote Subnet (Only for Across Availability Zones type) Displays the name of the remote subnet that the subnet extension connects to. (Name of remote subnet)
Remote IP Address (Only for Across Availability Zones type) Displays the IP address that you entered in the Remote IP Address field while creating the subnet extension. (IP Address)
Subnet Extension (properties) - (Only for To a Third Party Data Center type)
Local Gateway (Only for To a Third Party Data Center type) Displays the name of the local gateway used for the subnet extension. (Name of local gateway)
Remote Gateway (Only for To a Third Party Data Center type) Displays the name of the remote gateway used for the subnet extension. (Name of remote gateway)

Address Table

The Address Table tab provides MAC Address information only when the subnet extension uses VTEP service. The tab provides the following information in the table:

  • MAC Address : This provides the MAC addresses of devices connected to the remote VTEP endpoint in the subnet extension.
  • Remote VTEP Endpoint : This provides the IP address of the remote VTEP endpoint in the subnet extension.
Figure. Subnet Extensions Details View - Address Table tab sample for VTEP extension Click to enlarge Displays the subnet extension details page, Address Table tab that provides details of the MAC Addresses in the subnet extension

Figure. Subnet Extensions Details View - Address Table tab sample for VPN-based Extension Click to enlarge Displays the subnet extension details page, Address Table tab that provides details of the MAC Addresses in the subnet extension

Throughput

The Throughput tab provides a graphical representation of the throughput of the subnet extension.

Figure. Subnet Extensions Details View - Throughput tab sample for VTEP Extension Click to enlarge Displays the subnet extension details page, Throughput tab that provides graphical representation of the throughput of the subnet extension

Security Policies Summary View

To access the security policies dashboard, select Policies > Security Policies from the entities menu (see Entities Menu). The security policies dashboard allows you to view summary information about defined security policies.

Note: This section describes the information and options that appear in the security policies dashboard.
  • See Entity Exploring for instructions on how to view and organize that information in a variety of ways.
  • See Flow Microsegmentation Guide for information about how to create and apply security policies.
Figure. Security Policies Dashboard Click to enlarge Security policies view of the Explore dashboard

The following table describes the fields that appear in the security policies list. A dash (-) is displayed in a field when a value is not available or applicable.

Table 1. Security Policies List Fields
Parameter Description Values
Name Displays the policy name. The policy is one of three types: application, quarantine, or isolation. (name), Application, Quarantine, Isolation
Purpose Describes (briefly) the policy's purpose. (text string)
Policy Displays (high level) what the policy does. (boxed text)
Status Displays the current status of the policy (either applied currently or in monitoring mode). Applied, Monitoring
Last Modified Displays the date the policy was last modified (or the creation date if the policy has never been modified). (date)

You can filter the security polices list based on several parameter values. The following table describes the filter options available when you open the Security Policies view Filter pane. To apply a filter, select a parameter and check the box of the desired value (or multiple values) you want to use as a filter. You can apply filters across multiple parameters.

Table 2. Filter Pane Fields
Parameter Description Values
Name Filters on the item name. Select a condition from the pull-down list ( Contains , Doesn't contain , Starts with , Ends with , or Equal to ) and enter a string in the field. It will return a list of security policies that satisfy the name condition/string. (policy name string)
Type Filters on the policy type. Check the box for one or more of the policy types (application, quarantine, isolation). It will limit the list to just those policy types. Application, Quarantine, Isolation
Status Filters on the policy status. Check the box for applied or monitoring. Applied, Monitoring

The security policies dashboard includes a Create Security Policy action button with a drop-down list to Secure an Application or Isolation Environments .

The Actions menu appears when one or more policies are selected. It includes options to update, apply, monitor, and delete. The available actions appear in bold; other actions are grayed out. (For grayed out options, a tool tip explaining the reason is provided.)

Security Policy Details View

This Guide does not provide detailed information about Security Policies . It only describes the Summary and Details views for the same. For detailed information about Security Policies , see Flow Microsegmentation Guide

To access the details page for a security policy, click on the desired security policy name in the list (see Security Policies Summary View). The Security Policy details page includes the following:

  • The policy name appears in the upper left. You can switch from one policy to another by selecting the policy name from the pull-down list.
  • The rule status appears below the name and indicates whether the policy is being applied currently or is in monitoring mode.
  • Three columns appear that specify the Inbound policy (on the left), the affected entities (in the middle), and the Outbound policy (on the right).
  • There are three action buttons (upper right).
    • Click the appropriate button to update, apply, monitor, or delete the policy (see Nutanix Security Guide for details). The available actions appear in bold; other actions are grayed out. (For grayed out options, a tool tip explaining the reason is provided.)
    • Click the question mark icon to open a help page in a separate tab or window.
    • Click the X icon to close the details page.
Figure. Security Policy Details View: Monitoring Rule Example Click to enlarge Security policies view of the Explore dashboard

Figure. Security Policy Details View: Applied Rule Example Click to enlarge Security policies view of the Explore dashboard

For more information about Security Policies, see Flow Microsegmentation Guide.

Virtual Private Cloud

A Virtual Private Cloud (VPC) is an independent and isolated IP address space that functions as a logically isolated virtual network. A VPC could be made up of one or more subnets that are connected through a logical or virtual router. The IP addresses within a VPC must be unique. However, IP addresses may overlap across VPCs. As VPCs are provisioned on top of another IP-based infrastructure (connecting AHV nodes), they are often referred to as the overlay networks. Tenants may spin up VMs and connect them to one or more subnets within a VPC.

Virtual Private Cloud (VPC) is a virtualized network of resources that are specifically isolated from the rest of the resource pool. VPC allows you to manage the isolated and secure virtual network with enhanced automation and scaling. The isolation is done using network namespace techniques like IP-based subnets or VLAN based networking.

AHV provides the framework to deploy VPC on on-premises clusters using the following.

  • Advanced Networking subnets and DHCP management
  • Multiple uplink and bridge management via virtual switch (VS)
  • Virtual Private Network (VPN) gateways and connections

Flow Virtual Networking simplifies the deployment and configuration of overlay-based VPCs. It allows you to quickly:

  • Create, update and delete VPCs.
  • Create, update and delete subnets within VPCs.
    Note: Create subnets as necessary when you create VPCs.
  • Add network security policies and services.
  • Configure hybrid cloud connectivity with VPNs.

This section covers the concepts and procedures necessary to implement VPCs in the network.

VM IP Address Management

Primary Address

The primary IP address is assigned to a VM during initialization when the cluster provides any virtual NIC (NIC) to a VM.

  • Select Assign Static IP as the Assignment Type to add a static IP address as primary IP address of the VM, when you attach a subnet to a VM.
  • Select Assign with DHCP as the Assignment Type to allow DHCP to dynamically assign an IP address to the VM.
  • Select No Private IP as the Assignment Type if you do not want to assign an IP address to the vNIC of the VM.

For more information about attaching a subnet to a VM, see Creating a VM through Prism Central (AHV) in the Prism Central Guide .

Secondary IP Addresses (Overlay Networks only)

For your deployment, you may need to configure multiple (static) IP addresses to a single NIC. These IP addresses (other than the primary IP address) are secondary IP addresses. A secondary IP address can be permanently associated with a specific NIC or be changed to any other NIC. The NIC ownership of a secondary IP address is important for security routing policies.

Note: You can configure secondary IP addresses only for VMs in an Overlay network.

You can configure secondary IP addresses to a NIC when you want to:

  • Associate multiple floating IP addresses with one VM without creating multiple NICs (each with one primary IP address) for the VM. You can assign one floating IP address to one secondary IP address that you create for the single NIC. For information about floating IP addresses, see Requesting Floating IPs.
  • Run appliances, such as load balancers, that have multiple IP addresses on each interface.
  • Host applications in a High Availability (HA) configuration where the ownership of IP address moves from the active entity to the standby entity when the active entity goes down.
  • Host applications in a clustered configuration where the ownership of IP address follows the leader.
  • Host Nutanix Files service in a VPC as a case of clustered application.
Note:

In applications that use secondary IP addresses as virtual IP addresses and the NIC ownership of the secondary IP address changes dynamically from one NIC to another, configure the application to incorporate the ownership change in its settings or configuration. If the applications do not incorporate these ownership changes, the VPCs configured for such applications fail.

For information about configuring secondary IP addresses, see Creating Secondary IP Addresses.

IP Address Information

You can view the IP addresses configured on a VM by clicking the See More link in the IP Address column in the VM details view to open the IP Address Information box.

Note: The See More link in the IP Address column in the VM details view and the IP Address Information box are available only if the VM has any secondary IP addresses configured.
Figure. IP Address Information Click to enlarge Displaying the IP Address Information box

Creating Secondary IP Addresses

You can assign multiple secondary IP addresses to a single vNIC.

About this task

You can add multiple secondary IP addresses to the vNIC configured on a VM. Add the secondary IP addresses to the vNIC in the Create VM or the Update VM page.

Procedure

  1. Go to the Networks section.
  2. Click the Edit icon for the subnet that you want to add the secondary IP addresses from.
    The Update NIC dialog box opens.
  3. Check the Add Secondary IPs check box in the Update NIC dialog box.
    Figure. Add Secondary IP Addresses Click to enlarge Displaying the Add Secondary IPs section in Update NIC page.

  4. Add a comma-separated list of the secondary IP addresses that you want to add to the vNIC of the VM.
    Note:

    Ensure that the secondary IP addresses are within the same subnet that the primary IP address of the NIC is from. The subnets are displayed in the Private IP Assignment section in the Update NIC dialog box.

    Ensure that the secondary IP address is not the same as the IP address provided in the Private IP Assignment field.

  5. Click Save .
  6. Click Next on the Resources and the Management tabs of the Update VM page.

    If you need to make any other changes on the Resources and the Management tabs for any configurations other than adding secondary IP addresses, make the changes and then click Next on these tabs.

  7. Click Launch VM on the Review tab after you review

What to do next

You can view the secondary IP addresses configured on the VM in the IP Address Information box.

Assigning Secondary IP Addresses to Interfaces

Assign the secondary IP addresses to interfaces or subinterfaces on the VM.

About this task

To assign the secondary IP addresses to virtual interfaces on the VM, do the following on the VM details page:

Procedure

  1. Click Console .
  2. Log in as a root user.
  3. Run the ifconfig command as follows:
    root@host$ ifconfig <interface> <secondary ip address> <network mask>

    Provide the following in the command:

Parameter Description
<interface> The interface of the VM such as eth0. You can provide subinterfaces such as eth0:1 and eth0:2.
<secondary IP address> The secondary IP address that you created and want to associate with the interface.
<network mask> The network mask that is an expansion of the network prefix of the network that the secondary IP address belongs to. For example, if the secondary IP address belongs to 10.0.0.0/24 then the network mask is 255.255.255.0.
  1. Repeat the aforementioned steps for all the secondary IP addresses you want to associate with interfaces on the VM.
  2. Exit from the Console.

Assigning Secondary IP Addresses to Floating IPs

Assign the secondary IP addresses to floating IP addresses on the VM.

About this task

After you assign secondary IP addresses to interfaces or subinterfaces on the VM, you can assign the secondary IP addresses to floating IP addresses that may be used for external connectivity.

Do one of the following:

Procedure

  • Assign floating IP addresses when you request floating IP addresses in the Assign Floating IPs section of the Request Floating IP dialog box.
    To assign floating IP addresses while requesting for them, you must have the secondary IP addresses configured and ready when you are requesting the floating IP addresses.
  • Select the floating IP address you want to assign, in the Floating IPs dashboard. Click the Update option in the Actions drop-down menu.
    Assign the secondary IP addresses you configured to the floating IP addresses you have.

VPC Workflow

A virtual private cloud (VPC) can be deployed on Nutanix cluster infrastructure to manage the internal and external networking requirements using Flow Virtual Networking. The workflow to create a complete network based on VPC is described below.

  1. Create a VPC—See Creating Virtual Private Cloud. See Updating Virtual Private Cloud to update a VPC you created.
  2. Add Subnets to the VPC—See Creating a Subnet to create a Subnet. See Updating a Subnet to update a subnet.
  3. Attach the Subnet to VMs—See Attaching a Subnet to a Virtual Machine.

VPC Management

This section provides information and procedures that you need to manage virtual private clouds using Flow Virtual Networking.

Creating Virtual Private Cloud

About this task

You can create VPCs on the Virtual Private Clouds page. Go to the Virtual Private Clouds page by clicking Virtual Infrastructure > Networking > Virtual Private Clouds .

To create a VPC, do the following.

Procedure

  1. On the VPC dashboard, click Create VPC .

    See Network and Security View for more information about the VPC dashboard.

    The Create Virtual Private Cloud (VPC) dialog box opens.
    Figure. Create Virtual Private Cloud Click to enlarge

  2. Provide the necessary values in respective fields in the Create Virtual Private Cloud (VPC) dialog box.
Fields Description and Values

Name

Provide a name for the VPC.

External Connectivity

This section takes you through configuration of the parameters necessary for connectivity to the Internet or clusters outside the VPC.

A subnet with external connectivity (External Subnet) is required if the VPC needs to send traffic to a destination outside of the VPC.

Note: You can add a maximum of two external subnets - one external subnet with NAT and one external subnet without NAT to a VPC. Both external subnets cannot be of the same type. For example, you cannot add two external subnets, both with NAT. You can update an existing VPC similarly.

Network address translation (NAT) Gateways perform the required IP-address translations required for external routing. You can also have external connectivity without NAT.

External Subnet

Select an external subnet from the drop down list. By associating the VPC with the external subnet you can provide external connectivity to the VPC.
Note:

Ensure that the externally routable IP addresses (subnets with external connectivity without NAT) for different VPCs do not overlap.

Configure the routes for the external connectivity subnets with next hop as the Router or SNAT IP address. Also configure the routes on the router for the return traffic to reach the VPC. See External Connectivity panel in VPC Details View.

Externally Routable IP Addresses Provide IP addresses that are externally routable. Externally routable IP addresses are IP addresses that within the VPC which can communicate externally without NAT. These IP addresses are used when an external subnet without NAT is used.

Domain Name Servers (DNS)

(Optional) DNS is advertised to Guest VMs via DHCP. This can be overridden in the subnet configuration.

Click + Server IP to add DNS server IPs under IP Address and click the check mark.

You can Edit or Delete an IP address you added using the options under Actions .

  1. Click Save .

Requesting Floating IPs

About this task

Each VPN gateway requires a floating IP. If you do not provide one during the VPN gateway creation, then Flow Virtual Networking automatically allocates a floating IP to a VPN gateway. To provide floating IP during the VPN gateway creation, you can request floating IPs and assign them to VMs.

You can view the allocated floating IPs on the Floating IPs page. Click Networking > > Floating IPs .

To request a floating IP, do the following.

Procedure

  1. Click the Request Floating IP button on the Floating IPs page.
  2. On the Request Floating IP dialog box, provide the information in the respective fields.
    Figure. Request and Assign Floating IPs Click to enlarge

    Note:

    Uncheck the Assign Floating IPs box if you want to assign the requested IP addresses after you receive it.

    See Floating IPs for more information.

Fields Description and Values
External Subnet Select a subnet that you configured with external connectivity.
Number of Floating IPs Enter the number of floating IPs you want. You can request a maximum of 5 floating IP addresses.
Assign Floating IPs

Select this check box if you want to assign the floating IPs to specific VMs in the table.

Based on the number you entered in the Number of Floating IPs field, the system provides an equivalent number of rows of Search VMs and IP Address in the table.

Under Search VMs , select the VM to which you want to assign a floating IP address. Under IP Address , select the IP address on the VM (primary or secondary IP address) to which you want to assign the floating IP.

You can assign multiple floating IP addresses to multiple secondary IP addresses that you can create on the NIC of the VM.

For information about configuring secondary IP addresses, see Creating Secondary IP Addresses.

Note:
  1. Click Save .

What to do next

When you receive the floating IP address you requested, you can see it, assign it (if not already assigned while requesting) or delete it in the Floating IPs view.

Creating a Subnet

About this task

You can create subnets on the Subnets page. Go to the Subnets page by clicking Virtual Infrastructure > Networking and open the Create Subnet dialog box.

You can also open the Create Subnet dialog box from the VPC details view by clicking the Add Subnet option.

To create a subnet, do the following.

Procedure

  1. Click Create Subnet .
    The Create Subnet dialog box opens. The following figure displays the Create Subnet dialog box with all the options. These options are displayed based on the values you select in the Type field.
    Figure. Create Subnet (With External Connectivity Disabled) Click to enlarge

    Figure. Create Subnet (With External Connectivity Enabled) Click to enlarge

Fields Description and Values
Name Provide a name for the subnet.
Type

Select the type of subnet you want to create.

You can create a VLAN subnet or an Overlay subnet.

VLAN ID

(VLAN subnet only) Enter the number of the VLAN .

Enter just the number in this field, for example 1 or 27. Enter 0 for the native VLAN. The value is displayed as vlan.1 or vlan.27 in the View pages.

Note: Provision any single VLAN ID either in the AHV network stack or in the Flow Virtual Networking (brAtlas) networking stack. Do not use the same VLAN ID in both the stacks.
IP Address management

(Mandatory for Overlay type subnets) This section provides the Network IP Prefix and Gateway IP fields for the subnet.

(Optional for VLAN type subnet) Check this box to display the Network IP Prefix and Gateway IP fields and configure the IP address details.

Unchecking this box hides these fields. In this case, it is assumed that this virtual LAN is managed outside the cluster.

Note:

The DHCP Settings option is only available for VLAN subnets if you select this option.

DHCP Settings

(Optional for both VLAN and Overlay subnets) Check this box to display fields for defining a domain.

Checking this box displays fields to specify DNS servers and domains. Unchecking this box hides those fields.

See Settings the DHCP Options for more information.

Cluster (VLAN subnet only) (VLAN subnet only) This option is available only for VLAN subnet configuration. Select the cluster that you want to assign to the subnet.
External Connectivity (VLAN subnet only) Turn on this toggle switch if you want use this VLAN subnet for external connectivity.
Note:

Ensure that the externally routable IP addresses (subnets with external connectivity without NAT) for different VPCs do not overlap.

Configure the routes for the external connectivity subnets with next hop as the Router or SNAT IP address. Also configure the routes on the router for the return traffic to reach the VPC. See External Connectivity panel in VPC Details View.

NAT (Option under External Connectivity ) If you turn on the External Connectivity toggle switch, then you can choose whether to connect to external networks with or without enabling NAT. Check the NAT check box to enable NAT for external connectivity for VPCs.

Virtual Switch (VLAN subnet only) Select the virtual switch that is configured for the VLAN you selected. The default value is the default virtual switch vs0. This option is displayed only if you add a VLAN ID in the VLAN ID field.
VPC (Overlay subnet only)

Select the Virtual Private Cloud (VPC) that you want to assign to the subnet from the drop down list.

You can create VPCs and assign them to Overlay subnets.

IP Address Pool

Defines a range of addresses for automatic assignment to virtual NICs.

This field is optional for both VLAN and Overlay . For VLAN , this field is displayed only if you select the IP Address Management option.

Note: Configure this field for VLAN or Overlay to complete the creation of the VPC, if you do not need external connectivity for this subnet. You must configure this field only if you need external connectivity for this subnet.

Click the Create Pool button and enter the following in the Add IP Pool page:

  • Enter the starting IP address of the range in the Start Address field.

  • Enter the ending IP address of the range in the End Address field.

  • Under Actions , click the check mark to submit the starting and ending IP addresses you entered.

    Click the X mark to remove the entries.

Override DHCP Server

(VLAN subnet only) To configure a DHCP server, check the Override DHCP Server box and enter an IP address in the DHCP Server IP Address field.

See Override DHCP Server (VLAN Only) in Settings the DHCP Options for information about this option.

  1. Click Save .

Settings the DHCP Options

About this task

Selecting the DHCP Settings checkbox in Create Subnet or Update Subnet allows you to configure the DHCP options for the VMs within the subnet. When DHCP settings are configured for a VM in a subnet and the VM is powered on, Flow Virtual Networking configures these options on the VM automatically. If you do not configure the DHCP settings, then these options are not available on the VM automatically when you power it on.

You can enable DHCP Settings when you create a subnet and configure the DHCP Settings for the new subnet. You could also update the DHCP Settings for an existing subnet.

DHCP Settings is common to and is available on both the Create Subnet and the Update Subnet dialog boxes.

To configure the DHCP Settings , do the following in the Create Subnet or the Update Subnet dialog box:

Procedure

  • Provide the information in the DHCP Settings fields.
    Figure. DHCP Settings Click to enlarge DHCP Settings display

Fields Description and Values
Domain Name Servers

Provide a comma-separated list of DNS IP addresses.

Example: 8.8.8.8, 9.9.9.9

Domain Search

Enter the VLAN domain name. Use only the domain name format.

Example: nutanix.com

TFTP Server Name

Enter a valid TFTP host server name of the TFTP server where you host the host boot file. The IP address of the TFTP server must be accessible to the virtual machines to download a boot file.

Example: tftp_vlan103

Boot File Name

The name of the boot file that the VMs need to download from the TFTP host server.

Example: boot_ahv2020xx

  • (Optional and for VLAN networks only) Check the Override DHCP Server dialog box and enter an IP address in the DHCP Server IP Address field.

    You can configure a DHCP server using the Override DHCP Server option only in case of VLAN networks.

    The DHCP Server IP address (reserved IP address for the Acropolis DHCP server) is visible only to VMs on this network and responds only to DHCP requests. If this box is not checked, the DHCP Server IP Address field is not displayed and the DHCP server IP address is generated automatically. The automatically generated address is network_IP_address_subnet.254 , or if the default gateway is using that address, network_IP_address_subnet.253 .

    Usually the default DHCP server IP is configured as the last usable IP in the subnet (For eg., its 10.0.0.254 for 10.0.0.0/24 subnet). If you want to use a different IP address in the subnet as the DHCP server IP, use the override option.

Attaching a Subnet to a Virtual Machine

About this task

To attach a subnet to a VM, go to the Virtual Infrastructure > VM > List view in Prism Central and do the following.

Procedure

  1. Select the VM you want to attach a subnet to. Click Actions > > Update .
  2. In the Update VM dialog box, click Add NIC .
    Figure. Click to enlarge

  3. Provide the necessary information in the indicated fields in the Create NIC dialog box.
    1. Select the Subnet Name from the drop down list.
    2. Select the Network Connection State as Connected or Disconnected .

      The Network Connection State selection defines the state of the connection after the NIC configuration is implemented.

    3. Select the Assignment Type .

      You can select Assign with DHCP to assign a DHCP based IP address to the VM.

      You can select Assign Static IP to assign a static IP address to the VM to reach the VM quickly from any endpoint in the network such as a laptop.

    4. Click Add .
  4. Click Save on the Update VM dialog box.

Creating a Policy

About this task

For Policy-based routing you need to create policies that route the traffic in the network.

When you create a VPC, there is one default policy that Flow Virtual Networking creates for the VPC. This policy is pre-configured with the Priority 1 and other default values to Deny traffic flow and service (see the table of field descriptions and values for this dialog box).
Note: You cannot update or delete the default policy.
  • Policies control the traffic flowing between subnets (inter-subnet traffic).

  • Policies control the traffic flowing in and out of the VPC.

  • Policies do not control the traffic within a subnet (intra-subnet traffic).

Figure. Policy Tab Click to enlarge

You can create a traffic policy using the Create Policy dialog box. You can open the Create Policy dialog box either from the VPC list view or the VPC list view.

  • On the VPC list view, select the VPC you want to update and click Create Policy in the Actions drop down menu.

  • On the VPC details view, click the Create Policy option in the More drop down menu.

To create a policy, do the following in the Create Policy dialog box.

Procedure

  1. Provide the necessary values in the respective fields.
    Figure. Create Policy Click to enlarge

Fields Description and Values Value in Default Policy
Priority The priority of the access list (ACL) determines which ACL is processed first. Priority is indicated by an integer number. A higher priority number indicates a higher priority.For example, if two ACLs have priority numbers 100 and 70 respectively, the ACL with priority 100 takes precedence over the ACl with priority 70.
Note:
  • Click the Understand Priorities link to see the Understand Priorities information box (see the image of this box below this table).
1
Source

The source indicates the source IP or subnet for which you want to manage traffic.

Source can be:

  • Any : Indicates any IP address.

  • External : Indicates an IP address that is outside the subnets configured for the VPC.

  • Custom : You can provide a specific Source Subnet IP with prefix.
Any
Source Subnet IP

Only required if you selected the Source as Custom . Provide the subnet IP and prefix that you want to designate as the source for the policy. Use the CIDR notation format to provide the subnet IP. For example, 10.10.10.0/24.

None
Destination

The destination is the destination IP or subnet for which you want to set the priority.

Destination can be:

  • Any : Indicates any IP address.

  • External : Indicates an IP address that is outside the subnets configured for the VPC.

  • Custom : You can provide a specific Destination Subnet IP with prefix.
Any
Destination Subnet IP

Only required if you selected the Destination as Custom .

None
Protocol You can also set the priority configure policy for certain protocols. You can select one of the following options:
  • Any : Indicates any IP address.

  • Protocol Number : Provide an integer number that indicates the protocol you want to prioritize.

    Provide the appropriate value in the Protocol Number field.
  • TCP
  • UDP
  • ICMP
Protocol Number

This field is displayed only if you select Protocol Number as the value in the Protocol field. The number you provide must be the IANA designated number that indicates respective protocol. See IANA Protocol Numbers .

None
Action

Assign the appropriate action for implementation of the policy.

  • Permit : Permits traffic and services based on the parameters set.

    If the Permit rule is set to override a Drop rule, then the Permit rule must be set in both the directions to allow bidirectional communication between the Source and Destination .

  • Deny : Denies traffic and service based on the parameters set.

  • Re-route :Sends matching traffic to the next-hop IP address specified by the Reroute IP . In case of reroute, you need to provide an IP address that the traffic needs to be re-routed to, in the Reroute IP field.
Permit
Figure. Understanding Priorities Click to enlarge Sample Understand Priorities information box.

  1. Click Save .

Creating Static Routes

About this task

You can create a static route using the Create Static Routes dialog box. You can open the Create Static Routes dialog box either from the VPC list view or the VPC details view.

  • On the VPC list view, select the VPC and click Create Static Routes in the Actions drop down menu.

  • On the VPC details view, click the Create Static Routes option in the More drop down menu.

Figure. Create Static Routes Click to enlarge

To create static route, do the following in the Create Static Routes dialog box:

Procedure

  1. Provide the necessary values in the respective fields.
Fields Description and Values
Destination Prefix Provide the IP address with prefix of the destination subnet.
Next Hop Link Select the next hop link from the drop down list. The next hop link is the IP address that the traffic must be sent for the static route you are configuring.
Add Prefix You can create multiple static routes using this option. Click this link to add another set of Destination Prefix and Next Hop Link to configure another static route.
  1. Click Save .

Updating Virtual Private Cloud

About this task

You can update a VPC using the Update Virtual Private Cloud (VPC) dialog box. You can open the Update Virtual Private Cloud (VPC) dialog box either from the VPC list view or the VPC details view.

  • On the VPC list view, select the VPC you want to update and click Update in the Actions drop down menu.

  • On the VPC details view, click the Update option.

The Update Virtual Private Cloud (VPC) dialog box is identical to the Create Virtual Private Cloud (VPC) dialog box.

Figure. Update VPC Click to enlarge Displaying Update VPC dialog box

For details about the parameters that you can update in the Update Virtual Private Cloud (VPC) dialog box, see Creating Virtual Private Cloud.

Procedure

  • Update the parameters in the Update Virtual Private Cloud (VPC) dialog box.
  • Click Save .

Updating a Subnet

About this task

You can update a subnet displayed on the Subnets page. Go to the Subnets page by clicking Virtual Infrastructure > Networking > Subnets and open the Update Subnet dialog box.

You can also open the Update Subnet dialog box from the VPC dashboard for a specific VPC. Click the Edit option for the subnet listed on the Subnets tab of the VPC dashboard.

The fields in the Update Subnet and the Create Subnet dialog boxes are the same.
Note: You cannot edit or update the subnet type. For example, if the subnet type is already configured as VLAN , you cannot modify it to an Overlay type subnet.

To update a subnets, do the following.

Procedure

  1. Select the subnet you want to update. Select Actions > Update Subnet .
  2. Update the necessary values in the respective fields in the Update Subnet dialog box.
    Figure. Update Subnet Click to enlarge

    The Update Subnet dialog box has the same fields as the Create Subnet dialog box. For details about the fields and the values that can be updated in the Update Subnet dialog box, see Creating a Subnet.

  3. Click Save to ensure that the updates are saved in the configuration.

Category Management

A category is a key-value pair that groups similar entities. Associating a policy with a category ensures that the policy applies to all the entities in the group regardless of how the group scales with time. For example, you can associate a group of VMs with the Department: Marketing category, where Department is a category that includes a value Marketing along with other values such as Engineering and Sales.

Currently, you can associate only VMs with a category. Categories are implemented in the same way on on-premises Prism Central instances and in Xi Cloud Services. For information about configuring categories, see the Prism Central Guide .

Updating a Policy

About this task

You can update a policy using the Update Policy dialog box. You can open the Update Policy dialog box in two ways in the VPC details view.

  • On the VPC details view, select the VPC you want to update and click the Update option in the top menu.
  • On the VPC details view, click the Edit option provided in the Actions menu for the selected VPC.
Note: You cannot update or delete the default policy.

The Update Policy dialog box has the same parameters as the Create Policy dialog box.

For details about the parameters that you can update in the Update Policy dialog box, see Creating a Policy.

Procedure

  • Update the parameters in the Update Policy dialog box.
  • Click Save .

Updating Static Routes

About this task

You can update a static route using the Update Static Routes dialog box. You can open the Update Static Routes dialog box either from the VPC list view or the VPC details view.

Note: You must configure the default route (0.0.0.0/0) to the external subnet as the next hop for connectivity outside the cluster (north-south connectivity).
  • On the VPC details view, select the VPC you want to update and click the Update option in the top menu.
  • On the VPC details view, click the Edit option provided in the Actions menu for the selected VPC.

The Update Static Routes dialog box has the same parameters as the Create Static Routes dialog box.

For details about the parameters that you can update in the Update Static Routes dialog box, see Creating Static Routes.

Procedure

  • Update the parameters in the Update Static Routes dialog box.
  • Click Save .

Deleting a Virtual Private Cloud

About this task

Prism Central does not allow you to delete a VPC if the VPC is associated with any subnets and/or VPNs. After you remove all the subnets or VPN associations from the VPC, delete the VPC.

You can delete a VPC from the VPC list view or the VPC details view.

Procedure

  • Do one of the following.
    • To delete a VPC from the VPC list view, select the VPC you want to delete and click Delete in the Actions drop down menu.
    • To delete a VPC from the VPC details view, click the VPC name to go to the VPC details view and click the Delete option in the More drop down menu.
  • In the confirmation dialog box, do the following.
    • Click Delete to delete the VPC.
    • Click Cancel to exit without deleting the VPC.

Deleting Subnets, Policies or Routes

About this task

You can delete VPC entities such as subnets, policies or routes from the VPC details page.

Note: You cannot update or delete the default policy.

Do the following.

Procedure

  1. Open the VPC details page and go to the respective tab like Subnets , Policies or Routes .
  2. Click the Delete option provided for the selected entity (subnet, policy or route respectively).
  3. In the confirmation dialog box, do the following.
    • Click Delete to delete the entity.
    • Click Cancel to exit without deleting the entity.

Connections Management

This section covers the management of network gateways, VPN connections and subnet extensions including operations like create, update and delete network gateways and VPN connections, and extending subnets.

Network Gateway Management

You can create, update or delete network gateways that host one of VPN or VTEP service for connections.

Creating a Network Gateway

About this task

VPN or s connect two networks together, and can be used in both VLAN and VPC networks on AHV. In other words, you can extend the routing domain of a VLAN network or that of a VPC using a VPN. Accordingly, VPN gateways can be configured using VLANs or VPCs. You need VPN gateways on clusters to provide a gateway to the traffic between on-premise clusters or remote sites.

You can create multiple VPN gateways for a VPC. Since a VPC is configured only on a PC, the VPC is available to all the clusters registered to that PC.

A VPN gateway may be defined as a Local gateway or a Remote gateway based on where the traffic needs to be routed.

To create a VPN gateway, do the following on the Networking & Security > Connectivity > Gateways page.

Procedure

  1. Select Local or Remote in the Create Gateway drop-down menu.
    If you select Local in the drop-down menu, the Create Local Gateway dialog box opens. If you select Remote in the drop-down menu, the Create Remote Gateway dialog box opens.

  2. Provide the necessary values in the respective fields as described in the table.
    For example, if you select Local in the drop-down menu, then the Create Local Gateway dialog box opens. Provide the necessary values in the respective fields as described in the table.
    Figure. Sample Create Local Gateway - VM Deployment Click to enlarge

    Figure. Sample Create Local Gateway - VPN Service Configuration Click to enlarge

    Figure. Sample Create Local Gateway - VTEP Service Configuration Click to enlarge

    Figure. Sample Create Remote Gateway - VPN Gateway Service Click to enlarge

    Figure. Sample Create Remote Gateway - VTEP Gateway Service Click to enlarge

Table 1. Local Gateway Fields
Fields Description Values
VM Deployment
Name Enter a name for the network gateway. (Name)
Gateway Attachments (for Local gateway type only) Select the gateway attachment as VPC or VLAN . The VPN VM is deployed on a VPC VM or a cluster that has the selected VLAN respectively.
  1. If you select VPC , then VPC Attachment is displayed. VPC is the default value for the Gateway Attachments field. The Gateway VM is deployed on the cluster and associated with the VPC selected in the VPC Attachment section.

    VPC attachment mode provides the options of eBGP and Static routing methods for external routing (configured in the External Routing Configuration section).

  2. If you select VLAN , then the VLAN Attachment is displayed. The Gateway VM is deployed on the cluster that has the VLAN and the subnet specified in the VLAN Attachment section.

    VLAN attachment mode provides only the eBGP routing method for external routing.

(VLAN or VPC)
Gateway VM Deployment - VPC Attachment
Cluster Select the cluster on which you want to deploy the Gateway VM on. (Name of the cluster)
VPC (If Gateway Attachment type is VPC) Select the VPC configured on the selected cluster that you want to use for the Gateway VM deployment. (Name of the VPC selected)
Floating IP (Optional)

Select a floating IP for the network gateway configuration. If you do not select a floating IP address then Prism Central allocates a floating IP automatically. This allocated floating IP is deleted when you delete the gateway.

To request floating IPs and allocate them to subnets, see Requesting Floating IPs

(IP address)
Gateway VM Deployment - VLAN Attachment
Cluster Select the Cluster, from the drop down list, on which you want to deploy the Gateway VM on.
Note: Only clusters with VLANs are available in the list.
(Name of the cluster)
Subnet Select the subnet you want to attach the Gateway VM to, from the drop down list.
Note: The list includes all the subnets you created on the selected cluster.
After you select the subnet, the details of the subnet are displayed in a box below the Subnet field. The details include: VLAN ID, IPAM type being Managed or Unmanaged, and Network Address with Prefix.
(Name of the VLAN subnet)
Static IP Address for VPN Gateway VM Enter the static IP address that the Gateway VM needs to use. (IP Address with Prefix)
Default Gateway IP Enter the default gateway IP of the subnet for the Gateway VM. (IP Address)
Service Configuration
Gateway Service Select the gateway service you want to use for the gateway. (VPN or VTEP)
VPN Service Configuration - External Routing Configuration (This section is available for VLAN and VPC attachment types)
Routing Protocol
  1. For VPC gateway attachments: Select Static for static routing.
    Note: You need to create static routes (see Creating Static Routes) for external routing and attach the route to the VPC selected in this configuration.
  2. Select eBGP for eBGP based external routing.
  3. For VLAN gateway attachments: External routing protocol is pre-set to eBGP . You cannot change the routing protocol.
(Static or eBGP)
Redistribute Connected Routes (Applicable only if VLAN type gateway attachment is selected) ( VLAN only) Select this checkbox to enable the redistribution of connected routes into the eBGP. (Check mark or blank)
ASN (Only available if eBGP routing protocol is selected)

(For eBGP only) Enter the ASN for your on-prem gateway. If you do not have a BGP environment in your on-prem site, you can choose any number. For example, you can choose a number in the 65000 range.

Note: Make sure that this ASN does not conflict with any of the other on-premises BGP ASNs.

ASN must be distinct in case of eBGP.

(Number)
eBGP Password (For eBGP in Local gateway type only) Enter the eBGP password for the eBGP route. (Password: The password must be between 1 and 80 characters.
  • Characters allowed for Pre-Shared Key for IPSec

    • a-z

    • A-Z

    • 0-9

    • ~ ! @ # % ^ & * ( ) _ - + = : ; { } [ ] | < > , . / ? $

    • Password length: Minimum 1 and maximum 64 characters.

  • Characters allowed for BGP passwords
    • a-z

    • A-Z

    • 0-9

    • ~ ! @ # % ^ & * ( ) _ - + = : ; { } [ ] | < > , . / ? $

    • Password length: Minimum 1 and maximum 80 characters.

)
VPN Service Configuration - Internal Routing Configuration (This section is available for VLAN attachment type only.)
Routing Protocol (Between On-prem Gateway and On-prem Router) Select the Routing Protocol to be used between on-premises Nutanix gateway and on-premises router.

You can select:

  • Static : Select this protocol to provide a static route configuration for the VLAN gateway.

  • OSPF : Select this protocol to provide an OSPF routing configuration for the VLAN gateway.

  • iBGP : Select this protocol to provide a iBGP route configuration for the VLAN gateway.
    Note: For iBGP, the ASN must be the same between the Gateway appliance and the peer iBGP, when iBGP is selected as the internal routing protocol.
(Static or OSPF or iBGP)
+Add Prefix (Applicable to Static routing)

(For Static routing selected in Routing Protocol ) Click this to enter a Local Prefix and click the check mark under Actions to add the prefix.

If you click the X mark under Actions , the local prefix you entered is not added.

The prefixes you add are advertised to all the connected peers via eBGP.

The prefix must be a valid IP address with the host bits not set.

You can add multiple local prefix IP addresses.

(prefix like /24)
Area ID (Applicable to OSPF protocol) (OSPF only) Enter the OSPF area id in the IPv4 address format.
Password Type (OSPF only) Select the password type you want to set for the OSPF route. The options are:
  1. MD5 : Select this option to encrypt the packets with MD5 hash that can be decrypted with the MD5 password at the destination.

  2. Plain Text : Select this option to set a clear-text password.

  3. None : Select this if you do to set an open route without password protection

Password

(OSPF only) Enter a password for the MD5 or Plain Text password type you select in the Password Type field.

  • For MD5 : The password must be 1-16 characters long.

    Characters allowed for OSPF passwords (MD5)

    • a-z

    • A-Z

    • 0-9

  • For Plain Text : The password must be 1-8 characters long.

    Characters allowed for OSPF passwords (Plain text): a-z.

Peer IP (for iBGP) Enter the IP Address of the On-prem router used to exchange routes with the network gateway. (IP Address)
Password Enter a password with 1-80 characters. (Password)
VTEP Service Configurations
VxLAN (UDP) Port The default value provided is 4789. Do not change this. (Number. Default value is 4789)
Table 2. Remote Gateway Fields
Fields Description Values
Name Enter a name for the network gateway. (Name)
Gateway Service Select the gateway service you want to use for the gateway. (VPN or VTEP)
VPN Service Configurations
Public IP Address Enter the public IP address of the remote endpoint. If a Floating IP is not selected, a new Floating IP is automatically allocated for the Gateway. These allocated IP addresses are deleted when the network gateway is deleted. (IP Address)
Vendor Select the vendor of the third party gateway appliance. (Name of Vendor)
External Routing
Protocol
  1. Select Static for static routing.
    Note: You need to create static routes (see Creating Static Routes) for external routing and attach the route to the VPC selected in this configuration.
  2. Select eBGP for eBGP based external routing.
(Static or eBGP)
eBGP ASN (Only available if eBGP routing protocol is selected)

(For eBGP only) Enter the ASN for your on-prem gateway. If you do not have a BGP environment in your on-prem site, you can choose any number. For example, you can choose a number in the 1-65000 range.

Note: Make sure that this ASN does not conflict with any of the other on-premises BGP ASNs.

ASN must be distinct in case of eBGP.

(Number)
VTEP Service Configurations
VTEP IP Address Enter VTEP IP Addresses of the remote endpoints that you want to create the gateway for. You can add IP addresses of multiple endpoints in one remote gateway. (Comma separated list of IP Addresses)
VxLAN (UDP) Port The default value provided is 4789. Do not change this. (Number. Default value is 4789)
  1. Click Save .

What to do next

The Gateway you create is displayed in the Gateways page.

Updating a Network Gateway

About this task

You can update a network gateway using the Update Gateway dialog box.

You can open the Update Gateway dialog box. The parameters in the Update Gateway dialog box are the same as those in the Create Local Gateway or Create Remote Gateway dialog box.

Procedure

  1. Select the gateway you want to update on Gateways .
  2. Click Update in the Actions menu.
  3. Update the required details in the Update Gateway dialog box.
    You cannot modify some information. Such fields are greyed and in-actionable. If you need to modify such information, consider creating a new gateway with the updated parameters and deleting the current gateway.
  4. Click Save .

Deleting a Network Gateway

About this task

If you want to delete a network gateway, you must first delete all the VPN connections associated with the gateway and only then you can delete the network gateway.

To delete a network gateway, do the following on the Gateway page.

Procedure

  1. Do one of the following.
    • Select the check box next to the name of the gateway and, in the Actions drop-down list, click Delete .
    • Click the name of the gateway and, in the details page, click Delete .
  2. In the confirmation dialog box, do the following.
    • Click Delete to delete the entity.
    • Click Cancel to exit without deleting the entity.

Virtual Network Connections

Virtual Private Network

You can use the Nutanix VPN solution to set up VPN between your on-prem clusters, which exist in distinct routing domains that are not directly connected. These distinct routing domains could either be VPCs within the same cluster or remote clusters or sites.

If you need to connect one Nutanix deployment in one site to another deployment in a different site, you can create a VPN endpoint in each of the sites. A VPN endpoint consists of a local VPN gateway, remote VPN gateway and VPN connection. Local VPN gateway can be instantiated in a VPC context or a legacy VLAN context. Launching the VPN gateway within a VPC allows stretching of the VPC. For example, in the figure, the Blue VPC is stretched between two sites with a VPN.

Figure. VPN Working Click to enlarge

VPN connections are useful in connecting two points. You can connect two VPCs in the same cluster using a VPN or VPCs in different clusters in the same site. However, VPN connection can connect only one endpoint to another endpoint. Flow virtual networking based VPN service allows you to only connect two endpoints that use Nutanix VPN based gateway service.

Virtual Tunnel End Points Based Network Extensions

To connect one endpoint to multiple endpoints or third party (non Nutanix) networks, use Virtual Tunnel End Point (VTEP) service based subnet extensions. For more information about VTEP, see Layer 2 Virtual Subnet Extension Over VTEP.

VPN Workflow

If you need to connect one Nutanix deployment in one site to another deployment in a different site, you can create a VPN endpoint in each of the sites. A VPN endpoint consists of a local VPN gateway, remote VPN gateway and VPN connection. You can configure multiple VPN endpoints for a site.

Each endpoint must have configurations for a local VPN gateway, remote VPN gateway (pointer information for the peer local VPN in the remote site endpoint) and a VPN connection (connecting the two endpoints). Then, based on the VPN connection configuration as initiator or acceptor, one endpoint initiates a tunnel and the endpoint at the other end accepts the tunnel connection and, thus, establishes the VPN tunnel.

  1. Gateways: Every VPN endpoint for each site consists of two VPN gateway configurations - Local and Remote.

    Local gateway is a VM that runs the VPN protocols (IKEv2, IPSec) and routing (BGP and OSPF). Remote gateway is a pointer - database entry - that provides information about the peer remote VPN endpoint. One of the key information contained in the remote gateway is the source IP of the remote VPN endpoint. For security reasons, the local VPN gateway will accept IKEv2 packets originating only from this Source IP.

    VPN gateways are of the following types:

    • On premises Nutanix VPN Gateway: Represents the VPN gateway appliance at your on-premises local or remote site if you are using the Nutanix VPN solution.

    • On premises Third Party Gateway: Represents the VPN gateway appliance at your on-prem site if you are using your own VPN solution (provided by a third-party vendor).

      To configure third party VPN Gateways, see the relevant third party documentation.

  2. VPN Connection: Represents the VPN IPSec tunnel established between local gateway and remote gateway. When you create a VPN connection, you need to select two gateways between which you want to create the VPN connection.

VPN appliances perform the following:

  1. Implementation of IKEv2 and IPSec protocols.
  2. Routing: Between remote sites, Flow virtual networking advertises prefixes using eBGP. Optionally it uses Static routing. Within a site, Flow virtual networking uses iBGP or OSPF to share prefixes between the Nutanix VPN appliance and the edge router.

Prerequisites for VPN Configurations

General Requirements

  • Ensure that you have enabled Flow virtual networking with microservices Infrastructure.

  • Ensure that you have floating IP addresses when you create VPN gateways.

    Flow virtual networking automatically allocates a floating IP to a VPN gateway if you do not provide one during the VPN gateway creation. To provide floating IP during the VPN gateway creation, you can request floating IPs. See Requesting Floating IPs.

  • Ensure that you have one of the following, depending on whether you are using iBGP or OSPF:

    • Peer IP (for iBGP): The IP address of the router to exchange routes with the VPN gateway VM.

    • Area ID (for OSPF): The OSPF area ID for the VPN gateway in the IP address format.

  • Ensure that you have the following details for the deployment of the VPN gateway VM:

    • Public IP address of the VPN Gateway Device: A public WAN IP address that you want the on-prem gateway to use to communicate with the Xi VPN gateway appliance.

    • Static IP Address: A static IP address that you want to allocate to the VPN gateway VM. Use a floating IP address requested as the static IP address.

    • IP Prefix Length: The subnet mask in CIDR format of the subnet on which you want to install the VPN gateway VM. You can use an overlay subnet used for a VPC and assigned to the VM that you are using for the VPN gateway.

    • Default Gateway IP: The gateway IP address for the on-premise VPN gateway appliance.

    • Gateway ASN: ASN must not be the same as any of your on-prem BGP ASNs. If you already have a BGP environment in your on-prem site, the customer gateway is the ASN for your organization. If you do not have a BGP environment in your on-prem site, you can choose any number. For example, you can choose a number in the 65000 range.

Ports and Protocols

Nutanix deploys a number of ports and protocols in its software. ports that must be open in the firewalls to enable Flow Virtual Networking to function. To see the ports and protocols used Flow Virtual Networking , see Port Reference.

Endpoints and Terminations

The following endpoints and terminations occur in the course of Flow virtual networking based connections. For information about creating, updating or deleting VPN connections, see Connections Management.

Note: In a VPN connection do not configure both the gateways (local gateway and remote gateway) in an endpoint as Initiators or as Acceptors. If you configure the local gateway as Initiator then configure the remote gateway as Acceptor in one endpoint and vice-versa in the (other) remote endpoint.
VPN Endpoint Behind a Network Address Translation or Firewall Device

In this scenario, the IPSec tunnel terminates behind a network address translation (NAT) or firewall device. For NAT to work, open UDP ports 500 and 4500 in both directions.

Figure. VPN Endpoint Behind NAT or Firewall Click to enlarge

Things to do in NAT Things to do in on-prem VPN GW
Open UDP ports 500 and 4500 on both directions

Enable the business application policies to Allow the commonly-used business application ports.

IPSec Terminates on the Firewall Device

In this scenario, you do not need to open the ports for NAT (500 and 4500).

However, enable the on-prem VPN gateway to allow the traffic from the PC subnet to the advertised load balancer route where the Source port is any and the Destination port may be in the range of 1024-1034.

The PC subnet refers to the subnet where your Prism Central is running.

Figure. Tunnel Terminates on NAT or Firewall Click to enlarge

Creating a VPN Connection

About this task

Create a VPN connection to establish a VPN IPSec tunnel between VPN gateways in your on-prem site. Select the gateways between which you want to create the VPN connection.

To create a VPN connection, do the following on the Networking > VPN Connections page.

Procedure

  1. Click the Create VPN Connection button on the VPN Connections page.
  2. In the Create VPN Connection dialog box, provide the values in the respective fields.
Fields Description and Values
Name Enter a name for the connection.
VPN Connection
IPSec Secret Enter a secret password for the IPSec connection. To see the password, click Show . To hide the password, click Hide .
Local Gateway Select the connection parameters on the local gateway as Initiator or Acceptor of VPN Tunnel connections.
VPN Gateway Select the appropriate VPN Gateway as the local gateway for the VPN connection
VTI Prefix - Local Gateway Enter a IPv4 Address with /<prefix>. Example: 10.25.25.2/30.

This is the VPN Tunnel Interface IP address with prefix for the local gateway. The subnet for this IP address must be a /30 subnet with two usable IP addresses. One of the IP addresses is used for Local Gateway. Use the other IP address for the Remote Gateway.

Connection Handshake This defines the type of handshake that the connection must use. There are two types of connection handshakes:
  1. Initiator : The local VPN gateway acts as the initiator of the connection and thus initializes the VPN tunnel.
  2. Acceptor : The local VPN gateway accepts or rejects incoming connection requests from other gateways.
Note: In a VPN connection do not configure both the gateways (local gateway and remote gateway) in an endpoint as Initiators or as Acceptors. If you configure the local gateway as Initiator then configure the remote gateway as Acceptor in one endpoint and vice-versa in the (other) remote endpoint.
Remote Gateway For a specific VPN connection, set the remote gateway as Initiator or Acceptor when you configure the VPN connection on the Remote Gateway.
VPN Gateway Select the appropriate VPN Gateway as the remote gateway for the VPN connection.
VTI Prefix - Remote Gateway The VPN Tunnel Interface IP address with prefix for the local gateway. Provide a IPv4 Address with /<prefix>. Example: 10.25.25.2/30.

This is the VPN Tunnel Interface IP address with prefix for the local gateway. The subnet for this IP address must be a /30 subnet with two usable IP addresses. One of the IP addresses is used for Local Gateway. Use the other IP address for the Remote Gateway.

Advanced Settings Set the traffic route priority for the VPN connection. The route priority uses Dynamic route priority because the priority is dependent on the routing protocol configured in the VPN gateway.
Route Priority - Dynamic Route Priority Set the route priority as an integer number. The greater the number, higher is the priority.
  1. Click Save .

What to do next

The VPN connection you create is displayed in the VPN Connections page.

Updating VPN Connection

About this task

You can update a VPN Connection using the Update VPN Connection dialog box.

You can open the Update VPN Connection dialog box. The parameters in the Update VPN Connection dialog box are the same as those in the Create VPN Connection dialog box.

Procedure

  1. Select the VPN Connection you want to update on the VPN Connection .
  2. Click Update in the Actions menu.
  3. Update the required details in the Update VPN Connection dialog box.
  4. Click Save .

Deleting a VPN Connection

About this task

To delete a VPN connection, do the following on the VPN Connection page.

Procedure

  1. Do one of the following.
    • Select the check box next to the name of the VPN connection and, in the Actions drop-down list, click Delete .
    • Click the name of the VPN connection and, in the details page, click Delete .
  2. In the confirmation dialog box, do the following.
    • Click Delete to delete the entity.
    • Click Cancel to exit without deleting the entity.

VPN Connection within Same Prism Central

You can connect two VPCs within the same Prism Central availability zone using a VPN connection.

About this task

Assume that you have created two VPCs named vpc-a and vpc-b with overlay subnets named subnet-a and subnet-b .

To connect the two VPCs within the same Prism Central using a VPN connection, do the following.

Procedure

  1. Do the following for local gateways:
    1. Create a local VPN gateway with dynamically assigned address for vpc-a , for example, named local-vpn-a . Note or write down the assigned IP address.
    2. Create a local VPN gateway with dynamically assigned address for vpc-b , for example, named local-vpn-b . Note or write down the assigned IP address.

    See Creating a Network Gateway for more information about creating a VPN gateway.

  2. Do the following for remote gateways:
    1. Create a remote VPN gateway with the IP address noted in 1.a for vpc-a , for example, named remote-vpn-a .
    2. create a local VPN gateway with the IP address noted in 1.b for vpc-b , for example, named remote-vpn-b .

    See Creating a Network Gateway for more information about creating a VPN gateway.

  3. Create a VPN connection between vpc-a and vpc-b named, for example, vpn-conn-a-to-b .
    Ensure that the VTI IP addresses for the local and remote gateways is unique with /30 prefix.
    Note: The VPN Tunnel Interface IP address with prefix for the local gateway. The subnet for this IP address must be a /30 subnet with two usable IP addresses. One of the IP addresses is used for Local Gateway. Use the other IP address for the Remote Gateway.

    Ensure that you select local-vpn-a as the local gateway with Connection Handshake set as Acceptor .

    Ensure that you select remote-vpn-b as the remote gateway.

  4. Create a VPN connection between vpc-b and vpc-a named, for example, vpn-conn-b-to-a .
    Ensure that the VTI IP addresses with /30 prefix for local and remote gateways are the reverse (vice versa) of what you configured for the VPN connection in previous step. For example, if in previous step you configured the VTI IP addresses as 10.20.20.5/30 for local and 10.20.20.6/30 for remote then for VPN connection in this step, configure 10.20.20.6/30 for local gateway and 10.20.20.5/30 for remote gateway respectively. These IP addresses do not need to be reachable anywhere else in the network. However, ensure that these IP addresses do not overlap with any other IP addresses assigned in the network.

    Ensure that you select local-vpn-b as the local gateway with Connection Handshake set as Initiator .

    Ensure that you select remote-vpn-a as the remote gateway.

Layer 2 Virtual Network Extension

You can extend a subnet between on-prem local and remote clusters or sites (Availability Zones or AZs) to support seamless application migration between these clusters or sites.

Note: One or more on-prem cluster or sites managed by one Prism Central instance is defined as an Availability Zone or AZ. In this section, Availability Zone or AZ refers to and must be understood as one or more on-prem clusters or sites managed by one Prism Central. Local AZ refers to local on-prem clusters or sites managed by a Prism Central instance and remote AZ refers to another on-prem cluster or site managed by another Prism Central instance.

With Layer 2 subnet extension, you can migrate a set of applications to the remote AZ while retaining their network bindings such as IP address, MAC address, and default gateway. Since the subnet extension mechanism allows VMs to communicate over the same broadcast domain, it eliminates the need to re-architect the network topology, which could otherwise result in downtime.

Layer 2 extension assumes that there are underlying existing layer 3 connectivity already available between the Availability Zones. You can extend a subnet from a remote AZ to the primary (Local) AZ (and other remote AZs in case of VTEP-based subnet extensions)

  • You can extend a Layer 2 subnet across two Nutanix AZs over either VPN or Virtual tunnel End Point (VTEP). SeeLayer 2 Virtual Subnet Extension Over VPN.
  • You can extend a Layer 2 subnet between a Nutanix AZ and one or more non-Nutanix datacenters only over VTEP. See Layer 2 Virtual Subnet Extension Over VTEP.

You can extend subnets for the following configurations.

  • IPAM Type. Managed and unmanaged networks.
  • Subnet Type. On-prem VLAN subnets and VPC subnets.
  • Traffic Type. IPv4 unicast traffic and ARP.
  • On-prem Hypervisor. AHV and ESXi
    Note: If your cluster is ESXi, use vCenter Server to manually configure the port group attached to the subnet you want to extend. Set the security settings, Promiscuous mode and Forged transmits to Accept on the vSwitch as shown in the following image.
    Figure. ESXi Host Port Group Configuration Click to enlarge ESXi port group settings

Prerequisites for Setting Up Subnet Extension

Ensure the following before you configure Layer 2 subnet extension between your on-prem AZs.

  • Ensure that the Prism Central versions support Layer 2 virtual subnet extension as specified in the Release Notes. See AOS Family Release Notes and Release Notes | Prism Central as applicable.

    See the Prism Central Upgrade and Installation Guidelines and Requirements section of the Acropolis Upgrade Guide for instructions about how to upgrade a Prism Central instance through the Prism Central web console.

  • Ensure that you pair the Prism Central at the local AZ with the Prism Central at the remote AZ to use Create Subnet Extension wizard to extend a subnet across the AZs and facilitate bidirectional communication between these clusters or sites. Using paired availability zones it is possible to configure both VXLAN over VPN and VTEP based subnet extension. You can also extend subnets using the manual gateway and connection workflows instead of pairing the AZs.

    See the Pairing AZs (Nutanix Disaster Recovery) for instructions about how to pair the local and remote AZs.

  • Ensure that you set up a default static route with 0.0.0.0/0 prefix and the External Network next hop for the VPC you use for any subnet extension. This allows NTP and DNS access for the Network Gateway appliance.

Best Practices for Subnet Extension

Nutanix recommends the following configurations to allow IP address retention for VMs on extended subnets.

  • When using Nutanix IPAM ensure the address ranges in the paired subnets are unique to avoid conflict between VM IP addresses across extended subnets.
  • If the source and target sites use third-party IPAM, ensure that there are no conflicting IP address assignments across the two sites.
    Note: If the source and target sites use Nutanix IPAM, the Prism Central web console displays a message that indicates an IP address conflict if one exists.
  • If connectivity between sites already provides encryption, consider using VTEP only subnet extension to reduce encryption overhead.
  • Use the Subnet Extension to a Third Party Data-Center workflow in the following scenarios
    • To extend a subnet to more than one other AZ. This is also known as point to multi-point.
    • To extend subnets between clusters managed by the same Prism Central.

Subnet Extension Workflow

You can manage Layer 2 subnet extension on the Subnet Extensions tab of the Connectivity page. Open the Subnet Extensions by clicking the hamburger icon in the top-left corner of the Dashboard and then clicking Connectivity .

  • You can create point-to-point Layer 2 subnet extensions between two AZs over VPN or VTEP by opening the Create Subnet Extension Across Availability Zones dialog box. See Extending a Subnet Over VPN for VPN-based extensions. See Extending a Subnet Across Availability Zones Over VTEP for VTEP-based extensions.

  • You can create point-to-point or point-to-multipoint Layer 2 subnet extensions to third party datacenters over VTEP by opening the Create Subnet Extension To A Third Party Data-Center dialog box. See Extending a Subnet to Third Party Datacenters Over VTEP.

  • You can update a subnet extension that extends across AZs using the Update Subnet Extension Across Availability Zones dialog box. The Update Subnet Extension Across Availability Zones has the same parameters and fields as the Create Subnet Extension Across Availability Zones dialog box. You can open the Update Subnet Extension Across Availability Zones dialog box by:

    • Selecting the subnet extended across AZs in the Subnet Extensions and clicking the Update button.

    • Clicking the subnet extended across AZs in the Subnet Extensions and clicking the Update button on the Summary tab.

You can update a subnet extension that extends to multiple AZs or third party datacenters using the Update Subnet Extension To A Third Party Data-Center dialog box. Update Subnet Extension To A Third Party Data-Center dialog box has the same parameters and fields as the Create Subnet Extension To A Third Party Data-Center dialog box. You can open the Update Subnet Extension To A Third Party Data-Center dialog box by:

  • Selecting the subnet extended to third datacenters in the Subnet Extensions and clicking the Update button.

  • Clicking the subnet extended to third datacenters in the Subnet Extensions and clicking the Update button on the Summary tab.

See Updating an Extended Subnet.

Layer 2 Virtual Subnet Extension Over VPN

Subnet extension using VPN allows seamless, secure migration to a new datacenter or for disaster recovery. VPN based Layer 2 extension provides secure point to point connection to migrate workloads between Availability Zones. Consider VTEP-only Subnet Extension without VPN when encryption is not required.

Subnet extension using VPN is useful:

  • When the two Availability Zones (where the subnets to be extended belong) do not have any underlying secure connectivity. For example, when connecting over the Internet, VPN (IPSec) provides the necessary connectivity and encryption (security).
  • Sometimes when you need to move (lift-and-shift) workloads from a VLAN subnet to a VPC subnet retaining the same VM IP addresses . You need connectivity from other subnets to workloads that have already migrated to VPC. In such cases, VPN provides the Layer 3 connectivity and encryption between the VPC segment of extended subnet to other VLAN subnets.

Prerequisites for Setting Up Subnet Extension Over VPN

  • See Layer 2 Virtual Network Extension for general prerequisites to extend subnets.

  • Set up VPN gateway services and a VPN connection between local AZ and the remote AZ. The subnet extension feature supports only the Nutanix VPN solution (not a third-party VPN solution) at the both the local and remote AZs. See the Virtual Network Connections for instructions about how to upgrade the VPN gateway VM at the local and remote clusters or sites.
    Note: Ensure that the VPN gateway version is 5.0 or higher. See the Updating a Network Gateway section for instructions about how to upgrade the network gateway at the local and remote sites.
  • Configure subnets with the same IP CIDR prefix at the source and target sites. For example, if the IP prefix at one site is 30.0.0.0/24, the IP prefix at the other site must also be 30.0.0.0/24. The network and mask must match at both AZs.
  • Configure distinct DHCP pools for the source and target sites with no IP address overlap. Separate DHCP pools ensure no IP address conflicts occur for dynamically assigned IP addresses between the two AZs.
  • Procure two free IP addresses, one from each subnet, for the Network Gateway in the subnets to be extended. These IP addresses are configured as local IP address and remote IP address for the subnet extension in the Subnet Extension wizard. These two free IP addresses are the externally accessible IP addresses for the local gateway, and the remote gateway. Those two usable IP addresses are already contained inside the VPN connection and must not conflict with the following:
    • DHCP pools on any of the Availability Zones.
    • Gateway IP address on any of the Availability Zones.
    • IP addresses allocated to existing user VMs on any of the Availability Zones.
    • IP addresses used by Network Gateway Management NIC subnet (IP pool 100.64.1.0/24)

Limitation

To use subnet extension over a VPN, both sites must use the VPN service of the Nutanix Network Gateway. Consider VTEP-only subnet extension to connect to non-Nutanix third party sites.

Pairing AZs (Nutanix Disaster Recovery)

To replicate entities (protection policies, recovery plans, and recovery points) to different on-prem AZs (AZs) bidirectionally, pair the AZs with each other. To replicate entities to different Nutanix clusters at the same AZ bidirectionally, you need not pair the AZs because the primary and the recovery Nutanix clusters are registered to the same AZ (Prism Central). Without pairing the AZs, you cannot perform DR to a different AZ.

About this task

To pair an on-prem AZ with another on-prem AZ, perform the following procedure at either of the on-prem AZs.

Procedure

  1. Log on to the Prism Central web console.
  2. Click the hamburger icon at the top-left corner of the window. Go to Administration > AZs in the left pane.
    Figure. Pairing AZ
    Click to enlarge Pairing AZ

  3. Click Connect to AZ .
    Specify the following information in the Connect to Availability Zone window.
    Figure. Connect to AZ
    Click to enlarge Connect to AZ

    1. AZ Type : Select Physical Location from the drop-down list.
      A physical location is an on-prem AZ (AZ). To pair the on-prem AZ with Xi Cloud Services, select XI from the drop-down list, and enter the credentials of your Xi Cloud Services account in step c and set d.
    2. IP Address for Remote PC : Enter the IP address of the recovery AZ Prism Central.
    3. Username : Enter the username of your recovery AZ Prism Central.
    4. Password : Enter the password of your recovery AZ Prism Central.
  4. Click Connect .
    Both the on-prem AZs are paired to each other.

Extending a Subnet Over VPN

The subnet extension allows VMs to communicate over the same broadcast domain to a remote site or Availability Zone (AZ).

Before you begin

See Layer 2 Virtual Network Extension and Layer 2 Virtual Subnet Extension Over VPN for information on prerequisites and best practices for extending a subnet.

About this task

Perform the following procedure to extend a subnet from the on-prem site.

Procedure

  1. Click the hamburger icon in the top-left corner of the Dashboard > Networking & Security > Connectivity > Subnet Extension .
  2. On the Subnet Extension page, select Create Subnet Extension > Across Availability Zones .
  3. In the Create Subnet Extension Across Availability Zones dialog box, enter the necessary details as described in the table.
    Figure. Create Subnet Extension Across Availability Zones Click to enlarge Create Subnet Extension Across Availability Zones using VPN service

Fields Description Values
Extend Subnet over a Select the gateway service you want to use for the subnet extension. (VPN or VTEP)
Note: Configure the following fields for the Local and the Remote sides of the dialog box.
Availability Zone (For Local) Local AZ is pre-selected default.

(For Remote) Select the appropriate AZ from the drop-down list of AZs.

(Local: Local AZ)

(Remote: Dropdown list of AZs.)

Subnet Type Select the type of subnet that you want to extend. (VLAN or Overlay)
Cluster Displayed if your selected VLAN subnet. Select the cluster from the dropdown list of clusters. (Name of cluster selected from dropdown list)
VPC Displayed if your selected Overlay subnet. Select the appropriate VPC from the dropdown list of VPCs. (Name of VPC selected from dropdown list)
Subnet Select the subnet that needs to be extended. (Name of subnet selected from dropdown list)
(Network Information frame) Displays the details of the VLAN or Overlay network that you selected in the preceding fields. (Network information)
Gateway IP Address/Prefix Displays the gateway IP address for the subnet. This field is already populated based on the subnet selected. (IP Address)
(Local or Remote) IP Address Enter a unique and available IP address that are externally accessible IP addresses in Local IP Address and Remote IP Address . (IP Address)
VPN Connection Select the appropriate VPN Connection from the dropdown list that Flow virtual networking must use for the subnet extension. See Creating a VPN Connection for instructions to create VPN connection. (Name of VPN connection selected from the dropdown list)
  1. Click Save .

    A successful subnet extension is listed on the Subnet Extension dashboard. See .

Layer 2 Virtual Subnet Extension Over VTEP

Subnet extension using Virtual tunnel End Point (VTEP) allows seamless migration to new datacenters or for disaster recovery. VTEP based Layer 2 extension provides point-to-multipoint connections to migrate workloads from one Availability Zone to multiple Availability Zones without encryption. If you need security and encryption, consider using Subnet Extension over VPN.

Subnet extension using VTEP is useful:

  • When both subnets that need to be stretched are Nutanix subnets (managed or unmanaged). VTEP provides an optimized workflow to stretch the two subnets.
  • When both subnets are connected over an existing private and secure link that does not need additional encryption.
  • When one Nutanix subnet needs to be stretched across one or more non-Nutanix networks, sites, or datacenters. Subnet Extension with third-party VTEPs provides point-to-multipoint connectivity to third party datacenters assuming that there is underlying layer 3 connectivity between these VTEPs.

VTEP-based Layer 2 Subnet Extension provides the following advantages:

  • Layer 2 subnet extension from one AZ to multiple AZs.
  • Layer 2 subnet extension between Nutanix AZs and non-Nutanix third party VTEP-based AZs.
  • The Remote VTEP Gateway is a set of endpoint IP addresses. You can add endpoint IP addresses to an existing operational Remote VTEP Gateway without stopping the subnet extension services. This on-the-fly addition enables you to extend the subnets to more AZs than originally planned, or perform maintenance, without disrupting the running services or configuring new remote VTEP gateways.

Prerequisite for Setting Up Subnet Extension Over VTEP

  • See Layer 2 Virtual Network Extension for general prerequisites to extend subnets.

  • Set up VTEP local and remote gateway services on local and remote AZs. In case of point-to-multipoint extension, ensure that you create local and remote VTEP gateways on all the remote AZs that the subnet needs to be extended to.

  • For each extended subnet within the same Network Gateway appliance ensure that you have unique VxLAN Network Identifiers (VNIs) that you can use for the VTEP subnet extensions. VNI may be any number between 0 and 16777215.

Extending a Subnet Across Availability Zones Over VTEP

The subnet extension over VTEP allows VMs to communicate two Availability Zones (AZ) without a VPN connection.

Before you begin

See Layer 2 Virtual Network Extension and Layer 2 Virtual Subnet Extension Over VTEP for information on prerequisites and best practices for extending a subnet.

About this task

To extend a subnet over VTEP across two availability zones (AZs), do the following.

Procedure

  1. Open the Create Subnet Extension Across Availability Zones in one of the following ways:
    • On the Subnet Extensions tab, click > Create Subnet Extension > Across Availability Zones > .

    • In the Subnets dashboard, select the subnet you want to extend and click Actions > Extend > Across Availability Zones

    • In the Subnets dashboard, click the subnet you want to extend. On the subnet details page, click Extend > Across Availability Zones .

    Figure. Example of Create VTEP Extension Across AZs with VLAN Subnet Click to enlarge Displaying example of Create Subnet Extension Across Availability Zones for VLAN Subnet over VTEP

  2. For Extend Subnet over a , select VTEP .
  3. Enter or select the necessary values for the parameters in the Local and Remote (AZ) sections as described in the table.
Parameters Description and Value
Availability Zone Displays the name of the paired availability zone at the local AZ.
Subnet Type Select the type of the subnet - VLAN or Overlay that you are extending.
Cluster Select the name of the cluster in the local AZ that the subnet is configured for.
Subnet Select the name of the subnet at the local AZ for network. The VLAN ID and the IPAM - managed or unmanaged are displayed in the box below the Subnet field.
Gateway IP Address. Enter the gateway IP address of the subnet you want to extend. Ensure that you provide the IP address in <IP-address/network-prefix> format. for example the gateway IP is 10.20.20.1 in a /24 subnet then provide the gateway IP address as 10.20.20.1/24 .
Note: For an unmanaged network, enter the gateway IP address of the created subnet.
Local IP Address Enter a unique and available (unused) IP address from the subnet provided in Subnet for the Network Gateway appliance.
Remote IP Address Enter a unique and available (unused) IP address from the subnet provided in Subnet for the remote Network Gateway appliance.
Local VTEP Gateway Select the local VTEP gateway you created on the local AZ. See Creating a Network Gateway for information about creating VTEP gateways.
Remote VTEP Gateway Select the VTEP gateway you created on the remote AZ. See Creating a Network Gateway for information about creating VTEP gateways.
Connection Properties
VxLAN Network Identifier (VNI) Enter a unique number from the range 0-16777215 as VNI. Ensure that this number is not reused anywhere in the local or remote VTEP Gateways.
MTU The default MTU is 1392 to account for 108 bytes of overhead and the standard physical MTU of 1500 bytes. VPC Geneve encapsulation requires 58 bytes and VXLAN encapsulation requires 50. However, you can enter any valid MTU value for the network, taking this overhead into account. For example, if the physical network MTU and vs0 MTU are 1600 bytes, the Network Gateway MTU can be set to 1492 to account for 108 bytes of overhead. Ensure that the MTU value does not exceed the MTU of the AHV Host interface and all the network interfaces between the local and remote AZs.
  1. Click Save .
    After the subnet is extended, the extension appears in the Subnet Extensions list view.

Extending a Subnet to Third Party Datacenters Over VTEP

The subnet extension over VTEP allows VMs to communicate with multiple remote sites or Availability Zones (AZ) that may be third party (non-Nutanix) networks, or datacenters. It also provides the flexibility of adding more remote AZs to the same VTEP-based extended Layer 2 subnet. Examples of compatible VTEP gateways are switches from Cisco, Juniper, Arista, and others that support plain VXLAN VTEP termination.

About this task

To extend a subnet over VTEP across multiple availability zones (AZs) or third party datacenters, do the following.

Procedure

  1. Open the Create Subnet Extension To A Third Party Data-Center in one of the following ways:
    • On the Subnet Extensions tab, click > Create Subnet Extension > To A Third Party Data-Center

    • In the Subnets dashboard, select the subnet you want to extend and click Actions > Extend > To A Third Party Data-Center

    • In the Subnets dashboard, click the subnet you want to extend. On the subnet details page, click Extend > To A Third Party Data-Center .

    Figure. Example of Create VTEP Extension To A Third Party Data-Center with VLAN Subnet Click to enlarge Displaying example of Create Subnet Extension To A Third Party Data-Center for VLAN subnet over VTEP

  2. Enter or select the necessary values for the parameters in the Local , Remote (AZ), and Connection Properties sections as described in the table.
Parameters Description and Value
Local
Availability Zone Displays the name of the paired availability zone at the local AZ.
Subnet Type Select the type of the subnet - VLAN or Overlay that you are extending.
Cluster Select the name of the cluster in the local AZ that the subnet is configured for.
Subnet Select the name of the subnet at the local AZ for network. The VLAN ID and the IPAM - managed or unmanaged are displayed in the box below the Subnet field.
Gateway IP Address Enter the gateway IP address of the subnet you want to extend. Ensure that you provide the IP address in <IP-address/network-prefix> format. for example the gateway IP is 10.20.20.1 in a .24 subnet then provide the gatewway IP address as 10.20.20.1/24 .
Note: For unmanaged network, enter the gateway IP address of the created subnet.
Local IP Address Enter a unique and available (unused) IP address from the subnet provided in Subnet .
Local VTEP Gateway Select the local VTEP gateway you created on the local AZ. See Creating a Network Gateway for more information about creating a local VTEP gateway.
Remote
Remote VTEP Gateway Select the remote VTEP gateway you created on the local AZ. See Creating a Network Gateway for more information about creating a remote VTEP gateway.
Connection Properties
VxLAN Network Identifier (VNI) Enter a unique number from the range 0-16777215 as VNI. Ensure that this number is not reused anywhere in the networks that the Prism Central and Cluster are a part of.
MTU

The default MTU is 1392 to account for 108 bytes of overhead and the standard physical MTU of 1500 bytes. VPC Geneve encapsulation requires 58 bytes and VXLAN encapsulation requires 50. However, you can enter any valid MTU value for the network, taking this overhead into account. For example, if the physical network MTU and vs0 MTU are 1600 bytes, the Network Gateway MTU can be set to 1492 to account for 108 bytes of overhead. Ensure that the MTU value does not exceed the MTU of the AHV Host interface and all the network interfaces between the local and remote AZs.

  1. Click Save .
    After the subnet is extended, the extension appears in the Subnet Extensions list view.

Updating an Extended Subnet

The Update Subnet Extension Across Availability Zones dialog box has the same parameters and fields as the Create Subnet Extension Across Availability Zones dialog box.

About this task

You can update a subnet extension that extends across AZs using the Update Subnet Extension Across Availability Zones or the Update Subnet Extension To A Third Party data center dialog box. The Update Subnet Extension Across Availability Zones or the Update Subnet Extension To A Third Party data center dialog box has the same parameters and fields as the Create Subnet Extension Across Availability Zones or the Create Subnet Extension To A Third Party data center dialog box, respectively.

Based on the type of the subnet extension that you want to modify, refer to the following:

Procedure

  • Extending a Subnet Over VPN
  • Extending a Subnet Across Availability Zones Over VTEP
  • Extending a Subnet to Third Party Datacenters Over VTEP

Removing an Extended Subnet

Perform this procedure to remove the subnet extension.

About this task

This procedure deletes the extended subnet between the two Availability Zones (AZs) or between one Nutanix AZ and one or more third party subnets. Deleting the subnet extension does not automatically remove the network gateways or VPN connections that may have automatically been created by the Subnet Extension wizard. You need to separately delete these entities created automatically when the subnet was extended.

Note: Removing an extended subnet from a cluster or AZ (either source or target AZs) automatically deletes the extended subnet from the corresponding source or target AZs.

Procedure

  1. Click the hamburger icon in the top-left corner of the Dashboard .
    The main feature list appears.
  2. Click Network & Security > Connectivity > > Subnet Extensions .
    The Subnet Extensions tab displays a list of the extended subnets.
    Figure. Sample Subnet Extensions dashboard Click to enlarge Displaying the Delete action button for selected subnet extension

  3. Select the subnet extension you want to remove.
  4. Click Actions > Delete
    The confirmation dialog box is displayed.
  5. Click Remove .
    Click Cancel to close the dialog box without removing the subnet extension.

What to do next

Check the list in the Subnet Extensions tab to confirm that the subnet extension is removed.
Flow Networking Guide

Flow Virtual Networking pc.2022.1

Product Release Date: 2022-02-24

Last updated: 2022-12-09

Purpose

This Flow Networking Guide describes how to enable and deploy Nutanix Flow Networking on Prism Central.

Upgrading from EA Versions

If you have enabled the early access (EA) version of Flow Networking, disable it before upgrading the Prism Central and enabling the general availability (GA) version of Flow Networking.

Related Documentation

Links to Nutanix Support Portal software and documentation.

The Nutanix Support Portal provides software download pages, documentation, compatibility, and other information/

Documentation Description
Release Notes | Flow Networking Flow Networking Release Notes
Port Reference Port Reference: See this page for details of ports that must be open in the firewalls to enable Flow Networking to function.
Nutanix Security Guide Prism Element and Prism Central security, cluster hardening, and authentication.
AOS guides and release notes Covers AOS Administration, Hyper-V Administration for Acropolis, Command Reference, Powershell Cmdlets Reference, AOS Family Release Notes, and AOS release-specific Release Notes
Acropolis Upgrade Guide How to upgrade core and other Nutanix software.
AHV guides and release notes Administration and release information about AHV.
Prism Central and Web Console guides and release notes Administration and release information about Prism Central and Prism Element.

Flow Networking Overview

Enabled and administered from Prism Central, Flow Networking powers network virtualization to offer a seamless network experience with enhanced security. It is disabled by default.

To enable and use Flow Networking, ensure that you log on to Prism Central as a local account user with Prism Admin role. If you log on to Prism Central as a non-local account (IDP-based) user or without Prism Admin role privileges, then Prism Central does not allow you to enable or use Flow Networking. The task is reported as Failed with a User Denied Access message.

Note:

Nutanix deploys a number of ports and protocols in its software. ports that must be open in the firewalls to enable Flow Networking to function. To see the ports and protocols used Flow Networking, see Port Reference.

It is a software-defined network virtualization solution providing overlay capabilities for the on-prem AHV clusters. It integrates tools to deploy networking features like Virtual Private Cloud (VPC) and Virtual Private Network (VPN) to support flexible app-driven networking that focuses on VMs and applications instead of virtual LANs and network addresses.

After you enable it on Prism Central, Flow Networking delivers the following.

  • A simplified, Prism Central-based workflow that deploys the application-driven network virtualization feature.
  • A secure multi-tenancy solution allowing per-tenant isolation using VPC-based network segmentation and namespace isolation.
  • A secure VPN-based connectivity solution for multiple sites, with automated VPN bundle upgrades.
  • NAT-based secure egress to external networks, with IP address retention and policy-based routing.
  • Self-serve networking services using REST APIs.
  • Enhanced networking features for more effective disaster recovery.
    Note: You can enable network segmentation on a Layer 2 extended virtual subnet that does not have a gateway. For more information about Layer 2 subnet extensions, see Layer 2 Virtual Network Extension. For information about network segmentation of an extended layer 2 subnet, see Segmenting a Stretched L2 Network for Disaster Recovery in the Securing Traffic through Network Segmentation section of the Security Guide .

Deployment Workflow

You can enable Flow Networking using a simple Prism Central driven workflow, which installs the network controller. The network controller is a collection of containerized services that run directly on the Prism Central VM(s). The network controller orchestrates all the virtual networking operations.

  • Ensure that microservices infrastructure is enabled in Prism Central Settings > Prism Central Management . See Prism Central Guide for information about enabling microservices infrastructure.
  • Enable Flow Networking in Prism Central Settings > Advanced Networking . It is disabled by default. See Enabling Flow Networking

  • You can opt out of Flow networking by disabling the Advanced Networking option subject to prerequisites to disable advanced networking. See Disabling Flow Networking.

  • You can deploy Flow Networking in a dark site (a site that does not have Internet access) environment. See the Deploying Flow Networking at a Dark Site topic for more information.

  • You can upgrade the Flow networking controller. Nutanix releases an upgrade for the Flow networking controller with AOS and Prism Central releases. See Upgrading Flow Networking.

    See the AOS Family Release Notes and Release Notes | Prism Central .

  • Flow networking allows you to create and manage virtual private clouds (VPCs) and overlay subnets to leverage the underlying physical networks that connect clusters and datacenters. See Virtual Private Cloud.

  • You can upgrade the network gateway version. Network gateway is used to create VPN or VTEP gateways to connect subnets using VPN connections, or Layer 2 subnet extensions over VPN or VTEP.

Flow Networking Architecture

The Flow Networking architecture uses a three-plane approach to simplify network virtualization.

Prism Central provides the management plane, the network controller itself acts as the control plane while the AHV nodes provide the data plane. This architecture provides a strong foundation for Flow Networking. This architecture is depicted in the following chart.

Figure. Flow Networking Architecture Click to enlarge Flow Networking Architecture diagram

Deployment Scale

Flow Networking supports the following scale:

Entities Scale

Virtual Private Clouds

500

Subnets

5,000

Ports

50,000

Floating IPs

2,000 per networking controller-enabled Prism Central.

Routing Policies

1,000 per Virtual Private Cloud.

10,000 per networking controller-enabled Prism Central.

Essential Concepts

VPC

A Virtual Private Cloud (VPC) is an independent and isolated IP address space that functions as a logically isolated virtual network. A VPC could be made up of one or more subnets that are connected through a logical or virtual router. The IP addresses within a VPC must be unique. However, IP addresses may overlap across VPCs. As VPCs are provisioned on top of another IP-based infrastructure (connecting AHV nodes), they are often referred to as the overlay networks. Tenants may spin up VMs and connect them to one or more subnets within a VPC. Virtual Private Cloud (VPC) is a virtualized network of resources that are specifically isolated from the rest of the resource pool. VPC allows you to manage the isolated and secure virtual network with enhanced automation and scaling. The isolation is done using network namespace techniques like IP-based subnets or VLAN based networking.

VPC Subnets

You can use IP address-based subnets to network virtual machines within a VPC. A VPC may use multiple subnets. VPC subnets use private IP address ranges. IP addresses within a single VPC must be unique, in other words, IP addresses inside the same VPC cannot be repeated. However, IP addresses can overlap across multiple VPCs. The following figure shows two VPCs named Blue and Green. Each VPC has two subnets, 192.168.1.0/24 and 192.168.2.0/24, that are connected by a logical router. Each subnet has a VM with an IP address assigned. The subnets and VM IP addresses overlap between the two VPCs.

Figure. VPC Subnet Click to enlarge Displaying an illustration of VPC networks

The communication between VMs in the same subnets or different subnets in the same VPC (also called East-West communication) is enabled using GEneric NEtwork Virtualization Encapsulation (GENEVE). If a Prism Central manages multiple clusters, then the VMs that belong to the same VPC could be deployed across different clusters. The virtual switch on the AHV nodes provide distributed virtual switching and distributed virtual routing for all VPCs.

The communication from a VM in a VPC to an endpoint outside the VPC (called external communication or North-South communication) is enabled by an external network connection. Such a connection may be secured using VPN. The following figure shows the logical connectivity of the VPCs to the external network, and subsequently to the Internet.
Note: You must configure the default route (0.0.0.0/0) to the external subnet as the next hop for connectivity outside the cluster (north-south connectivity).
Figure. External Communication Click to enlarge

External Subnets

Subnets outside a VPC are external subnets. External subnets may be subnets within the deployment but not included in a specific VPC. External subnets may also be subnets that connect to the endpoints outside the deployment such as another deployment or site.

External subnets can be deployed with NAT or without NAT. You can add a maximum of two external subnets - one external subnet with NAT and one external subnet without NAT to a VPC. Both external subnets cannot be of the same type. For example, you cannot add two external subnets, both with NAT. You can update an existing VPC similarly.

Primary and Secondary IP Addresses for VMs
See VM IP Address Management.
SNAT and Floating IP Address

SNAT and Floating IP addresses are used only when you use NAT for an external subnet.

In Source Network Address Translation (SNAT), the NAT router modifies the IP address of the sender in IP packets. SNAT is commonly used to enable hosts with private addresses to communicate with servers on the public Internet.

For VMs within the VPC to communicate with the rest of the deployment, the VPC must be associated with an external network. In such a case, the VPC is assigned a unique IP address, called the SNAT IP, from the subnet prefix of the external network. When the traffic from a VM needs to be transmitted outside the VPC, the source IP address of the VM, which is a private IP address, is translated to the SNAT IP address. The reverse translation from SNAT IP to private IP address occurs for the return traffic. Since the SNAT IP is shared by multiple VMs within a VPC, only the VMs within the VPC can initiate connections to endpoints outside the VPC. The NAT gateway allows the return traffic for these connections only. Endpoints outside the VPC cannot initiate connections to VMs within a VPC.

In addition to the SNAT IP address, you can also request a Floating IP address — an IP from the external subnet prefix that is assigned to a VM via the VPC that manages the network of the VM. Unless the floating IP address is assigned to the private IP address (primary or secondary IP address) of the VM, the floating IP address is not reachable. When the VM transmits packets outside the VPC, the private IP of the VM is modified to the Floating IP. The reverse translation occurs on the return traffic. As the VM uses the Floating IP address, an endpoint outside the VPC can also initiate a connection to the VM with the floating IP address.

The translation of the private IP addresses to Floating IP or SNAT IP address, and vice versa, is performed in the hypervisor virtual switch. Therefore, the VM is not aware of this translation. Floating IP translation may be performed on the hypervisor that hosts the VM to which the floating IP is assigned to. However, SNAT translation is typically performed in a centralized manner on a specific host.

NAT Gateway

NAT Gateways are used only when you use NAT for an external subnet.

Network Address Translation (NAT) is a process for modifying the source or destination addresses in the headers of an IP packet while the packet is in transit. In general, the sender and receiver applications are not aware that the IP packets are being manipulated.

A NAT Gateway provides the entities inside an internal network with connectivity to the Internet without exposing the internal network and its entities.

A NAT Gateway is:

  • A node or a AHV host. You need a host or a node to implement a NAT Gateway because NAT gateways require operations like load balancing and routing that are automatically performed by Flow Networking.
  • Connected to the internal network with an internal subnet based IP address and to the external network with an externally-routable IP address.

    The externally-routable IP address may be an IP address from a private IP address space or an RFC1918 address that is used as a NAT gateway. The NAT Gateway IP address could be a static IP address or a DHCP assigned IP address.

Table 1. NAT Gateway Failover Time
Event Failover Time
Network controller stops on AHV Up to 45 seconds.
Node reboot Up to 45 seconds.
Node power off:

When NAT Gateway and network controller MSP worker VMs are not on the same node.

Up to 45 seconds.
Node power off:

When NAT Gateway and network controller MSP worker VMs are on the same node.

Up to 300 seconds (5 minutes).
Static IP Address

A static IP address is a fixed IP address that is manually assigned to an interface in a network. Static IP addresses provide stable routes that do not have to be updated frequently in the routing table since the static routes generated using static IP addresses do not need to be updated.

Usually in a large IP-based network (a network that uses IP addresses), a Dynamic Host Configuration Protocol or DHCP server assigns IP addresses to interfaces of an entity (using DHCP client service on the entity). However, some entities may require a static IP address that can be reached (manual remote access or via VPN) quickly. A static IP address can be reached quickly because the IP address is fixed, assigned manually and is stored in the routing table for a long duration. For example, a printer in an internal network would need a static IP address so that it can be connected reliably. Static IP addresses can be used to generate static routes which remain unchanged in routing tables, thus providing stable long-term connectivity to the entity that has the static IP address assigned.

Static Route

Static routes are fixed routes that are created manually by the network administrator. Static routes are more suited for small networks or subnets. Irrespective of the size of a network, static routes may be required in a variety of cases. For example, in VPCs where you use virtual private networks (VPNs) or Virtual Tunnel End Point (VTEP) over VxLAN transport connections to manage secure connections, you could use static routes for specific connections such as site-to-site connections for disaster recovery. In such a case it is necessary to have a known reliable route over which the disaster recovery operations can be performed smoothly. Static routes are primarily used for:

  • Facilitating the easy maintenance of the routing table in small networks that are not expected to grow.
  • Routing to and from other internal route or stub networks. A stub network or an internal route network is a network accessed using a single route and the router has only one neighbor.
  • Use as a default or backup route. Such a route is not expected to specifically match any other route in the routing table.

In a network that is not constantly changing, static routes can provide faster and more reliable services by avoiding the network overheads like route advertisement and routing table updates for specific routes.

Overlay networks

You can create an IP-based Overlay subnet for a VPC. An Overlay network is a virtualized network that is configured on top of an underlying virtual or physical network. A special purpose multicast network can be created as an Overlay network within an existing network. A peer-to-peer network or a VPN are also examples of Overlay networks. An important assumption for an Overlay network is that the underlying network is fully connected. Nutanix provides the capability to create Overlay network-based VPCs.

Comparing Overlay with VLAN

See how overlay networks compare with VLAN networks. A virtual local area network or VLAN network is a Layer 2 network that provides virtualized network segmentation solution. VLANs route and balance traffic in a network based on MAC addresses, Protocols such as Ethernet, ports or specific subnets. A VLAN creates a virtual Layer 3 network using Layer 2 addressing by separating broadcast domains virtually or logically. A VLAN configured network behaves as if the network is segmented using a physical layer 2 switch without implementing a layer 3 IP based subnet for the segmentation. VLAN traffic usually cannot traverse outside the VLAN.

The main advantage that VLAN networks provide is that VLAN networks require only layer 2 (L2) connectivity. VLANs do not require any of the layer 3 (L3) Flow Networking features.

Overlay networks can be laid on underlying physical network connections including VLAN networks. Overlay networks provide the following advantages and constraints:

  • IP address namespace is decoupled from the physical network.
  • You can create, update or delete overlay networks without requiring any configurations on the physical network and powering down the systems.
  • You can create overlay networks that can span across multiple clusters.
  • VLAN networks are necessary for Bootstrapping of Flow Networking.
    Note: Nutanix recommends using VLAN0 as the default untagged (also called native) VLAN for a CVM and AHV host. You can create VLANs for user VMs using the Network Configuration page. You can use the Create Virtual Switch dialog box from the Network Configuration page to create virtual switches for the user VM VLANs.
  • AHV Networking VLAN and Flow Networking VLAN: VLAN backed subnets for external connectivity are managed by the Flow Networking control plane. Traditional AHV VLAN IPAM networks are managed by Acropolis. Do not configure the same VLAN as both a Flow Networking external network and an AHV IPAM network, as this can lead to IP address conflicts.

Traffic Behavior

Broadcast Traffic

When all the guest VMs belonging to a subnet are in the same AHV: Flow Networking broadcasts the traffic to all guest VMs in the same subnet.

When some VMs belonging to a subnet are in other AHVs: Flow Networking tunnels the traffic to only those AHVs which have endpoints in the same subnet.

In other words, Flow Networking broadcasts traffic to all the guest VMs in the same subnet.

Unicast Traffic

Unicast traffic is traffic transmitted on a one-to-one basis between IP addresses and ports. There is only one sender and one receiver for the traffic. Unicast traffic is usually the most used form of traffic in any LAN network using Ethernet or IP networking. Flow Networking transmits unicast traffic based on the networking policies set.

Unknown Unicast Traffic

Flow Networking always drops unknown unicast traffic. It is not transmitted to any guest VM within or outside the source AHV.

Multicast Traffic

Flow Networking transmits the traffic to the VMs in the multicast group within the same subnet. If the VM is on another AHV, the destination AHV must have an endpoint in the subnet.

Multicast Group

A multicast group is defined by an IP address (called a multicast IP address, usually a Class D IP address) and a port number. Once a host has group membership, the host will receive any data packets that are sent to that group defined by an IP address/port number.

Prerequisites for Enabling Flow Networking

Make sure you meet these prerequisites before you enable Flow networking on Prism Central.

Requirements

Important: Prism Central protection and recovery does not protect or recover Flow networking services.

You must have the following fulfilled to enable Flow networking:

  • Ensure that you log on to Prism Central as a local account user with Prism Admin role. If you log on to Prism Central as a non-local account (IDP-based) user or without Prism Admin role privileges, then Prism Central does not allow you to enable or use Flow Networking. The task is reported as Failed with a User Denied Access message.

  • Ensure that the Prism Central running Flow networking is hosted on an AOS cluster running AHV.

    The network controller has a dependency only on the AHV version.

  • Ensure that microservices infrastructure on Prism Central is enabled. See Prism Central Guide for information about microservices infrastructure.
  • Choose the x-large PC VM size for Flow networking deployments. Small or large PC VMs are not supported for Flow Networking.

    If you are running a small or large Prism Central VMs, upgrade the Prism Central VM resources to x-large PC VM. See Acropolis Upgrade Guide for procedure to install an x-large Prism Central deployment.

  • Although Flow networking may be enabled on a single-node PC, Nutanix strongly recommends that you deploy a three-node scale-out Prism Central for production deployments. The availability of Flow networking service in Prism Central is critical for performing operations on VMs that are connected to overlay networks. A three-node scale-out Prism Central ensures that Flow networking continues to run even if one of the nodes with a PCVM fails.

  • Prism Central VM registration. You cannot unregister the Prism Element cluster that is hosting the Prism Central deployment where you have enabled Flow Networking. You can unregister other clusters being managed by this Prism Central deployment.

  • Ensure that Microservices Infrastructure (CMSP) is enabled on Prism Central before you enable Flow Networking. See the Prism Central Guide for more information.

    For the procedure to enable Microservices Infrastructure (including enable in dark site), see Enabling Micro Services Infrastructure section in the Prism Central Guide .

    Note: When you configure microservices infrastructure, ensure that the DNS name you configure for CMSP does not end with test . Flow networking does not support test as a top level domain. For example, the following are valid domain configurations:
    • my.cluster.domain
    • my.test.cluster.test.domain
    However, the following are examples of domains that Flow networking does not support:
    • my.cluster.test
    • my.cluster.domain.test
  • Ensure that you have created a virtual IP address (VIP) for Prism Central. The Acropolis Upgrade Guide describes how to set the VIP for the Prism Central VM. Once set, do not change this address.

  • Ensure connectivity:

    • Between Prism Central and its managed Prism Element clusters.

    • To the Internet for connectivity (not required for dark site) to:

      • ECR for Docker images
      • S3 storage for LCM portal
      Note: For dark site deployments, Nutanix provides a dark site bundle, which has the Docker images (normally hosted on ECR) and the network controller package (normally hosted on LCM portal). These dark site bundles can be downloaded using an internet-connected system outside the dark site.
  • Prism Central backup, restore, and migration. You cannot perform these operations on MSP-enabled Prism Central.
  • Nutanix recommends increasing the MTU to 9000 bytes on the virtual switch vs0 and ensure that the physical networking infrastructure supports higher MTU values (jumbo frame support). The recommended MTU range is 1600-9000 bytes.

    Nutanix CVMs use the standard Ethernet MTU (maximum transmission unit) of 1,500 bytes for all the network interfaces by default. The system advertises the MTU of 1442 bytes to guest VMs using DHCP to account for the extra 58 bytes used by Generic Network Virtualization Encapsulation (Geneve). However, some VMs ignore the MTU advertisements in the DHCP response. Therefore, to ensure that Flow networking functions properly with such VMs, enable jumbo frame support on the physical network and the default virtual switch vs0.

    If you cannot increase the MTU of the physical network, decrease the MTU of every VM in a VPC to 1442 bytes in the guest VM console.

    Note: Do not change the MTU of the CVM.
    Figure. Sample Configurations with and without Higher MTU - VS0, CVM and UVMs Click to enlarge

Requirements for Upgrades

The following applies to upgrades of Flow networking network controller ( Advanced Networking in Prism Control Settings ):

  • Ensure that the Prism Central host is running an AHV version compatible with the networking controller upgrade version. If necessary, upgrade the AHV version using LCM to the version compatible with the network controller upgrade version.
    Note:

    See Compatibility and Interoperability Matrix on the Nutanix Support portal for AOS and Prism Central compatibility.

  • Ensure that all the AHV hosts in the AOS cluster are running the version compatible with the network controller upgrade version.

    The network controller upgrade fails if any of the AHV hosts is running an incompatible version.

Limitations

Limitations for Flow networking are as follows.
  • Flow networking does not support Flow security for guest VMs.

    You cannot configure rules for Flow security if a guest VM has any NICs connected to VPCs.

  • Flow networking is supported only on AHV clusters. It is not supported on ESXi or Hyper-V clusters.

  • Flow networking is not enabled on the new PE cluster registering with the Flow networking-enable Prism Central if the Prism Element cluster has an incompatible AHV version.

  • Flow networking does not support updating a VLAN-backed subnet as an external subnet.

    You cannot enable the external connectivity option in the Update Subnet dialog box. Therefore, you cannot modify an existing VLAN-backed subnet to add external connectivity.

    VLAN backed subnets for external connectivity are managed by the Flow networking control plane. Traditional AHV VLAN IPAM networks are managed by acropolis.

    Note: Do not configure the same VLAN as both a Flow networking external network and an AHV IPAM network, as this can lead to IP address conflicts.
  • Flow networking cannot be disabled if any external subnets and VPCs are in use. Delete the external subnets and VPCs and then disable Flow Networking.

  • Disaster Recovery backup and migration: CMSP-enabled Prism Central does not support disaster recovery backup and migration operations both as a source and target host.

Flow Networking Configurations

Enabling Flow Networking

Before you begin

Ensure tha microservices infrastructure is enabled on Prism Central. See Enabling Micro Services Infrastructure section in the Prism Central Guide .

About this task

Before you proceed to enable Flow Networking by enabling the Advanced Networking option, see Prerequisites for Enabling Flow Networking.

To enable Advanced Networking, go to Prism Central Settings > Advanced Networking and do the following.

Procedure

  • In the Advanced Networking pane, click Enable .

    Ensure that the prerequisites specified on the pane are fulfilled.

    Figure. Enabling Flow Networking Click to enlarge Displaying the Advanced Networking page.

    Prism Central displays the deployment in-progress.
    Figure. Deployment Progress Click to enlarge Displaying the Deployment Progress.

  • Flow Networking is enabled.
    Figure. Flow Networking Status Click to enlarge Displaying the enabled status of Flow Networking.

Disabling Flow Networking

About this task

You can disable Flow Networking. However, the network controller cannot be disabled if any external subnets and VPCs are in use. Delete the subnets and VPCs before you disable advanced networking.

Note:

Flow Networking cannot be disabled if any external subnets and VPCs are in use. Delete the external subnets and VPCs and then disable Flow Networking.

To disable Flow Networking, do the following.

Procedure

  1. On the Advanced Networking page, click Disable .
    Figure. Click to enlarge Displaying the highlighted Disable Advanced Networking link.

  2. On the confirmation message box, click Confirm to confirm disablement.

    To exit without disabling the Advanced Networking controller, click Cancel .

Unregistering a PE from the PC

Before unregistering a Prism Element from PC, disable Flow Networking on that Prism Element using network controller CLI (or atlas_cli).

About this task

When Flow Networking is enabled on a Prism Central, it propagates the capability to participate in VPC networking to all the registered Prism Elements that are running the required AHV version.

In cases where there are VMs on the Prism Element attached to the VPC network, or if the Prism Element is used to host one or more of the external VLAN networks attached to a VPC, Prism Central alerts you with a prompt. When being alerted about the aforementioned conditions, close the CLI and make adequate configuration to resolve the condition (for example, select a different cluster for the external VLAN network and delete the VMs attached to the VPC network running on the Prism Element). After making such configurations, execute the network controller CLI to disable Flow Networking. If the command goes through successfully, it is safe to unregister the Prism Element.

For example, in a deployment of three Prism Elements - PE1, PE2 and PE3 - registered to the Flow Networking-enabled PC, you want to unregister PE3 from the PC. You must first disable Flow Networking using the following steps:

Procedure

  1. SSH to PE3.
  2. Run the ncli cluster info or ncli cluster get-params command to get the cluster parameters.
    Copy the cluster UUID (For example: 017457d3-1012-465c-9c54-aa145f2da7d9) from the displayed cluster parameters.
  3. SSH to the Prism Central VM.
  4. Open the network controller console by executing the atlas_cli command.
    nutanix@cvm$ atlas_cli
    <atlas> 
  5. Execute the config.add_to_excluded_clusters <cluster uuid> command, providing the cluster UUID that you copied earlier.

    An example of the PC alert, for the condition that PE3 VM is attached to an external network, is as follows:

    <atlas> config.add_to_excluded_clusters 0005bf8d-2a7f-3b2e-0310-d8e34995511e 
    Cluster 0005bf8d-2a7f-3b2e-0310-d8e34995511e has 1 external subnet, 
    which will lose connectivity. Are you sure? (yes/no)
    Note: To enable Flow Networking on the cluster, execute the config.remove_from_excluded_clusters <cluster uuid> command, providing the cluster UUID.

What to do next

To verify if Flow Networking is disabled or enabled, SSH to PE3 and run the acli atlas_config.get command.

The output displays the enable_atlas_networking parameter as False if Flow Networking is disabled and as True if Flow Networking is enabled on the Prism Element.

nutanix@cvm$ acli atlas_config.get
config {
  anc_domain_name_server_list: “10.10.10.10”
  enable_atlas_networking: False
  logical_timestamp: 19
  minimum_ahv_version: “20190916.101588"
  ovn_cacert_path: “/home/certs/OvnController/ca.pem”
  ovn_certificate_path: “/home/certs/OvnController/OvnController.crt”
  ovn_privkey_path: “/home/certs/OvnController/OvnController.key”
  ovn_remote_address: “ssl:anc-ovn-external.default.anc.aj.domain:6652"
}

You can now unregister the PE from the PC.

Upgrading Flow Networking

You can upgrade the Flow networking controller ( Advanced Networking Controller in Prism Central Settings ) using Life Cycle Manager (LCM) on Prism Central.

Before you begin

See Prerequisites for Enabling Flow Networking.

In case of upgrading the Flow networking controller in a dark site, ensure that LCM is configured to reach the local web server that hosts the dark site upgrade bundles.

Note:

The network controller upgrade fails to start after the pre-check if one or more clusters have Flow Networking enabled and are running an AHV version incompatible with the new network controller upgrade version.

About this task

To upgrade the network controller using LCM, do the following.

Procedure

  1. Choose one of the following ways to reach the LCM page:
    • Go to Administration > LCM > Inventory
    • Click Check for Updates on the Advanced Networking page.

    Figure. Check for Updates Click to enlarge Displaying Check for Updates link on the Advanced Networking page.

  2. Click Perform Inventory .

    When you click Perform Inventory , the system scans the registered Prism Central cluster for software versions that are running currently. Then it checks for any available upgrades and displays the information on the LCM page under Software .

  3. Go to Updates > Software . Select the Advanced Networking Controller version you want to upgrade to and click Update .
    Figure. Networking Controller version Click to enlarge Displaying sample LCM dashboard with the available Advanced Networking Controller upgrade available

Deploying Flow Networking at a Dark Site

About this task

Dark sites are primarily on-premises installations which do not have access to the internet. Such sites are disconnected from the internet for a range of reasons including security. To deploy Flow networking at such dark sites, you need to deploy the dark site bundle at the site.

This dark site deployment procedure includes downloading and deploying MSP and the network controller bundles.

Before you begin

  • See Prerequisites for Enabling Flow Networking.

  • You need access to the Nutanix Portal from an Internet-connected device to download the following dark site bundles:

    Note: For dark site deployments, Nutanix provides a dark site bundle, which has the Docker images (normally hosted on ECR) and the network controller package (normally hosted on LCM portal). These dark site bundles can be downloaded using an internet-connected system outside the dark site.
    • MSP dark site bundle: https://portal.nutanix.com/page/downloads/list > Microservices Platform (MSP)
    • Flow Networking network controller dark site bundle: See the Flow Networking Release Notes for the link to download the dark site bundle.
    • Network Gateway bundle: See the Flow Networking Release Notes for the link to download the dark site bundle with checksum text file. Also, see KB-12393 .

To deploy Flow Networking at a dark site, do the following.

Procedure

  1. Start a web server to host the dark site bundles and act as a source for the LCM downloads, if one is not already created.

    The web server can be a virtual machine on a cluster at the dark site. All the Prism Central VMs at the dark site must have access to this web server. This web server is used when you deploy any dark site bundle including the network controller darksite bundle.

    For more information about the server installation, see:

    • Linux web server

    • Windows web server

  2. In Prism Central, go to Administration > LCM > Inventory .

    Alternatively, SSH into the Prim Central VM as an admin user and run the following command.

    admin@pcvm$ mspctl controller airgap enable --url=http://<LCM-web-server-ip>/release

    Where <LCM-web-server-ip> is the IP address of the LCM web server and release is the name of the directory where the packages were extracted.

    For example, admin@pcvm$ mspctl controller airgap enable --url=http://10.48.111.33/release . Here, 10.48.111.33 is the IP address of the LCM web server and release is the name of the directory where the packages were extracted.

  3. Verify the configuration by running the following command:
    nutanix@cvm$ mspctl controller airgap get
  4. From a device that has public Internet access, click the Nutanix Compatibility Bundle link and down the bundle. Transfer this bundle to the LCM web server and extract the contents.
  5. From a device that has public Internet access, Nutanix recommends that you download and extract the latest MSP dark site bundle, transfer it to the LCM web server, and extract the contents.
  6. From a device that has public Internet access, download the Flow networking dark site bundle (see Release Notes | Flow Networking for download links). Transfer the bundle to the LCM web server.
  7. Extract or unpack the Flow networking dark site bundle on the LCM web server.

    After unpacking, check if the system shows a directory path that includes the following as per the example: http://<LCM-web-server-ip>/release/builds/msp-builds/msp-services/464585393164.dkr.ecr.us-west-2.amazonaws.com/nutanix-msp/atlas-hermes/ .

  8. Run the following command after unpacking to ensure that the file permissions are not disrupted during the unpacking:
    • Linux.
      chmod -R +r builds
    • Windows NTFS.
      
      $> takeown / R / F *
      $> icacls <Build-file-path> /t /grant:F 
      .
  9. Enable microservices infrastructure.

    See the Enabling Microservices Infrastructure section in the Prism Central Guide for details.

  10. Enable Flow Networking. See Enabling Flow Networking.

Troubleshooting Tips

This section provides information to assist troubleshooting of Flow Networking deployments. This is in addition to the information that the "Prism Central Guide" provides.

Audit Logs

Prism Central generates audit logs for all the flow networking activities like it does for other activities on Prism Central. See Audit Summary View in the Prism Central Guide , for more information about Audit log.

Support Bundle Collection

To support troubleshooting for Flow Networking, you can collect logs.

To collect the logs, run the following commands on the Prism Central VM console:

nutanix@cvm$ logbay collect -t msp,anc

An example of the command is as follows:

nutanix@cvm$ logbay collect -t msp,anc -O msp_pod=true,msp_systemd=true,kubectl_cmds=true,persistent=true --duration=-48h0m0s

Where:

  • -t flag indicates the tags to collect

    • msp tag will collect logs from the services running on MSP pods and persistent log volumes (application-level logs)

    • anc tag will collect the support bundle, which includes database dumps and OVN state

  • -O flag adds tag-level options

    • msp_pod=true collects logs from MSP service pods

      On the PC, these logs can be found under /var/log/containers .

    • persistent=true collects persistent log volumes (application-level logs for ANC)

      On the PC, these can be found under /var/log/ctrlog

    • kubectl_cmds=true runs kubectl commands to get the Kubernetes resource state

  • --duration sets the duration from the present to collect

The command run generates a zip file at a location, for example: /home/nutanix/data/logbay/bundles/<filename>.zip

Unzip the bundle and you'll find the anc logs under a directory specific to your MSP cluster, the worker VM where the pod is running, and the logging persistent volume of that pod. For example:

./msp/f9684be8-b4e8-4524-74b4-076ed53ca1fd/10.48.128.185__worker_master_etcd/persistent/default/ovn/anc-ovn_StatefulSet/

For more information about the task run, see the text file that the command generates at a location, for example: /home/nutanix/data/logbay/taskdata/<taskID>/collection_result.txt

For more information about the logbay collect command, see the Logbay Log Collection (Command Line) topic in the Nutanix Cluster Check Guide (NCC Guide).

Layer 2 Virtual Subnet Extension Alert

The L2StretchLocalIfConflict alert (Alert with Check ID - 801109) may occur while performing Layer 2 virtual subnet extensions. See KB-10395 for more information about its resolution.

Network Gateway Upgrades

Nutanix deployment can detect and install upgrades for the onprem Nutanix Gateways.

For information about identifying the current Nutanix Gateway version, see Identifying the Gateway Version.

For onprem Nutanix Gateways, the upgrades need to be detected and installed on the respective PC on which each Nutanix Gateway is installed.

For more information, see Detecting Upgrades for Gateways.

When PC detects the upgrades, it displays a banner on the Gateways tab of the Connectivity page. The banner notifies you that a Gateway upgrade is available after you have run LCM inventory. The table on the Gateways tab also displays an alert (exclamation mark) icon for the network gateways that the upgrade applies to. The hover message for the icon informs you that an upgrade is available for that Gateway.

Figure. Upgrade Banner Click to enlarge Displaying sample VPN Gateway tab.

For more information about the upgrade procedure, see Upgrading the PC-managed Onprem Nutanix VPN Gateways.

Identifying the Gateway Version

About this task

To identify the current Nutanix Gateway version, do the following:

Procedure

  • Click the hamburger icon and Networking & Security > Connectivity .
  • On the Gateways tab, click the Gateway name link text to open the Gateway details page.

    In the Gateway table, the VPN Gateway name is a clickable link text.

    The Gateway Version is listed in the Properties widget.

    Figure. Gateway Version Click to enlarge Displays sample VPN Gateway details page with clickable version number.

Detecting Upgrades for Gateways

About this task

Prism Central can detect whether new Gateway upgrades are available, or not, for Nutanix Gateways using LCM. You can then install the upgrade.

Procedure

  • Click the hamburger icon of Dashboard .
  • Click Administration > LCM > Inventory .
  • Click Perform Inventory .
    Note:

    Nutanix recommends that you select Enable LCM Auto Inventory in the LCM page in Prism Central to continuously detect new Gateway upgrades as soon as they are available.

    The upgrade notification banner is displayed on the Gateways page.

Upgrading the PC-managed Onprem Nutanix VPN Gateways

About this task

Perform upgrades of PC-managed Nutanix Gateways using the respective PC on which the Gateway is created.

To upgrade the on-prem Nutanix Gateways, do the following:

Procedure

  1. Log on to the Prism Central as the admin user and click the gear icon.
  2. Go to Administration > LCM > Inventory .
  3. Click Perform Inventory .

    When you click Perform Inventory , the system scans the registered Prism Central cluster for software versions that are running currently. Then it checks for any available upgrades and displays the information on the LCM page under Software .

    Note:

    Skip this step if you have enabled auto-inventory in the LCM page in Prism Central.

  4. Go to Updates > Software . Select the Gateway version you want to upgrade to and click Update .

    LCM upgrades the Gateway version. This process takes sometime.

Network and Security View

The Network and Security category in the Entities Menu expands on-click to display the following networking and security entities that are configured for the registered clusters:

  1. Subnets : This dashboard displays the subnets and the operations that you can perform on subnets.

  2. Virtual Private Clouds : This dashboard displays the VPCs and the operations that you can perform on VPCs.

  3. Floating IPs : This dashboard displays a list of floating IP addresses that you are using in the network. It allows you to request for floating IP addresses from the free pool of I addresses available to the clusters managed by the Prism Central instance.

  4. Connectivity : This dashboard allows you to manage the following networking capabilities:

    • Gateways : This tab provides a list of network gateways that you have created and configured, and the operations you can perform on the network gateways. You can check and upgrade the Gateway bundle in Administration > LCM > Inventory .

    • VPN Connections : This tab provides a list of VPN connections that you have created and configured, and the operations you can perform on VPN connections.

    • Subnet Extensions : This tab provides a list of subnets that you have extended at the Layer 2 level using VPN (point-to-point over Nutanix VPN) or VTEP (point-to-multi-point including third party).

  5. Security Policies : This dashboard provides a list of security policies you configured using Flow Segmentation. For more information about Security Policies, see the Flow Microsegmentation Guide.

See "Network Connections" section for information on how to configure network connections.

Subnets (Overlay IP subnets), Virtual private clouds, floating IPs, and Connectivity are Flow Networking features. These features support flexible app-driven networking that focuses on VMs and applications instead of virtual LANs and network addresses. Flow Networking powers network virtualization to offer a seamless network experience with enhanced security. It is disabled by default. It is a software-defined network virtualization solution providing overlay capabilities for the on-premises AHV clusters.

Security policies drives the Flow Segmentation features for secure communications. See Flow Microsegmentation Guide.

Subnets

Manage subnets in the List view of Subnets dashboard in the Network and Security section.

To access the Subnets dashboard, select Subnets from the entities menu in Prism Central. The Subnets dashboard allows you to view information about the subnets configured for the registered clusters.

Note: This section describes the information and options that appear in the Network and Security dashboard. See Entity Exploring for instructions on how to view and organize that information in a variety of ways.
Figure. Subnets Dashboard Click to enlarge sample Subnets dashboard

The following table describes the fields that appear in the subnets list. A dash (-) is displayed in a field when a value is not available or applicable.

Table 1. Subnets Dashboard Fields
Parameter Description Values
Name Displays the subnet name. (subnet name)
External Connectivity Displays whether or not the subnet has external connectivity configured. (Yes/No)
Type Displays the subnet type. VLAN
VLAN ID Displays the VLAN identification number. (ID number)
VPC Displays the name of the VPC that the Subnet is used in. (Name of VPC)
Virtual Switch Displays the virtual switch that is configured for the VLAN you selected. The default value is the default virtual switch vs0 .
Note: The virtual switch name is displayed only if you add a VLAN ID in the VLAN ID field.
(virtual switch name)
IP Prefix Displays the IPv4 Address of the network with the prefix. (IPv4 Address/Prefix)
Cluster Displays the name of the cluster for which this subnet is configured. (cluster name)
Hypervisor Displays the hypervisor that the subnet is hosted on. (Hypervisor)

To filter the list by network name, enter a string in the filter field. (Ignore the Filters pane as it is blank.)

To view focused fields in the List, select the focus parameter from the Focus drop down list. You can create your own customised focus parameters by selecting Add custom from the drop down list and selecting the necessary fields after providing a Name , in the Subnet Columns .

There is a Network Config action button to configure a new network (see Configuring Network Connections

The Actions menu appears when one or more networks are selected and includes a Manage Categories option (see Assigning a Category ).

Go to the Subnets list view by clicking Network and Security > Subnets on the left side-bar.

Figure. Subnets Page Click to enlarge

To view or select actions you can perform on a subnet, select the subnet and click the Actions dropdown.

Figure. Subnet Actions Click to enlarge

Table 2. Subnet Actions
Action Description
Update Click this action to update the selected subnet. see Updating a Subnet in the Flow Networking Guide.
Manage Extension Click this action to create a subnet extension. A subnet extension allows VMs to communicate over the same broadcast domain to a remote Xi availability zone (in case of Xi-Leap based disaster recovery) via the extension.
Manage Categories Click this action to associate the subnet with a category or change the categories that the subnet is associated with.
Delete Click this action to delete the selected subnet. See Deleting Subnets, Policies, or Routes in the Flow Networking Guide .

You can also filter the list of subnets by clicking the Filters option and selecting the filtering parameters.

Subnet Summary View

View the details of a subnet listed on the Subnets page.

To view the details of a subnet, click the name of the subnet on the subnet list view.

Figure. Subnet Summary Page Click to enlarge Displaying sample subnets Summary view

The Summary page provides buttons for the actions you can perform on the subnet, at the top of the page. Buttons for the following actions are available: Update , Extend , Manage Categories , and Delete .

The subnet Summary page has the following widgets:

Widget Name Information provided
Subnet Details Provides the following:
  • Type — Displays the type of network like VLAN or Overlay.
  • VLAN ID — Displays the VLAN ID. This parameter is displayed only for VLAN networks.
  • VPC — Displays the VPC name. This parameter is displayed only for Overlay networks.
  • Cluster — Displays the cluster that the VLAN network is configured on. This parameter is displayed only for VLAN networks.
  • IP Prefix — Displays the IP address prefix configured for the network. This parameter is displayed for both VLAN and Overlay networks.
IP Pool Provides the IP address Pool Range assigned to the network.
External Connectivity Provides the following:
  • NAT — Displays whether NAT is enabled or disabled for VPCs connecting to the network. When you hover on the Enabled / Disabled status, the hover message displays details of VPCs connected to the external subnet.
  • Associated VPCs — Displays the VPCs associated with this external subnet.

Virtual Private Clouds

You can manage Virtual Private Clouds (VPCs) on the Virtual Private Clouds dashboard.

Go to the Virtual Private Clouds dashboard by clicking Network and Security > Virtual Private Clouds on the left side-bar.

Figure. Virtual Private Clouds dashboard Click to enlarge

You can configure the table columns for the VPC list table. The available column list includes Externally Routable IP Addresses that provides address space within the VPC that is reachable externally without NAT.. For the list of columns that you can add to the list table, see Customizing the VPC List View.

Note:

Ensure that the externally routable IP addresses (subnets with external connectivity without NAT) for different VPCs do not overlap.

Configure the routes for the external connectivity subnets with next hop as the Router or SNAT IP address. Also configure the routes on the router for the return traffic to reach the VPC. See External Connectivity panel in VPC Details View.

To view or select actions you can perform on a VPC, select the VPC and click the Actions drop down.

You can also filter the list of VPC by clicking the Filters option and selecting the filtering parameters.

Customizing the VPC List View

About this task

You can customize the columns in the table. Click the View by drop down and select + Add custom .

In the Virtual Network Columns dialog box, do the following.

Procedure

  1. Enter a name for the view.
  2. Select the columns you want displayed in the table.

    During the column selection, the columns you select are moved under the Selected Columns list. The Name (of the VPC) column is the default column already selected. You can add a maximum of 10 columns (including the Name column) to the Selected Column list.

    Figure. Customizing Columns in VPC View Click to enlarge

    To arrange the order of the selected columns, hover on the column name and click the up or down arrow button as appropriate.

  3. Click Save .

VPC Details View

To view the details of a VPC, click the name of the VPC on the VPC list view.

The VPC details view has the following tabs:

  • Summary
    Figure. Summary Tab Click to enlarge Displaying the Summary tab in the VPC dashboard

    The Summary tab provides the following panes:

    • DNS Servers —Provides more information about the DNS Servers used by the VPC.
    • External Connectivity —Provides the name of the external subnet, NAT Gateway host details, router/SNAT IP address and the IP address spaces or ranges configured for the VPC.
    • Floating IP Addresses —Provides details of the floating IP addresses that the VPC uses.
  • Subnets
    Figure. Subnet Tab Click to enlarge Displaying the Subnet tab in the VPC dashboard

    The Subnet tab provides the following information for the subnets:

    • Name —Displays the name of the subnet.
    • IP Range —Displays the IP address range configured for the subnet.
    • DHCP IP Pool —Displays the DHCP IP address pool configured for the subnet.
    • Default Gateway IP —Displays the IP address used as the default gateway by the entities in the subnet.
    • Actions —Displays the actionable links to Edit or Delete the subnet.
  • Policies
    Figure. Policies Tab Click to enlarge Displaying the Policy tab in the VPC dashboard

    The Policies tab maps the following information about the security-based traffic shaping policies you configure:

    • Priority —The traffic priority.
    • Rule —The Allow or Deny rule set for the priority.
    • Traffic —The traffic type that the priority and rule should be applied to.
    • Actions —Actions you can take on the policy. You can perform three actions: Clear counters , Edit the policy or Delete the policy.
  • Routes
    Figure. Routes Tab Click to enlarge Displaying the Router tab in the VPC dashboard

    The Routes tab provides the following information about the routes:

The VPC details view has the following configuration options for the VPC:

  • Update : Use this option to update the VPC. For more information, see Updating Virtual Private Cloud.
  • Add Subnet : Use this option to add a subnet to the VPC. For more information, see Creating a Subnet.
  • Create Static Routes : Use this option to create a static route. For more information, Creating Static Routes.
  • Update Static Routes : Use this option to update static route configurations that you already created. For more information, see Updating Static Routes.
  • Create Policy : Use this option to create traffic policies in addition to the pre-configured default policy. When you create a VPC, there is one default policy that Advanced Networking creates for the VPC. This policy is pre-configured and cannot be edited. For more information, see Creating a Policy.
  • Clear All Counters : Allows you to clear all the counters for the VPC.
  • Delete : Allows you to delete the VPC. For more information, see Deleting a Virtual Private Cloud.

Floating IPs

You can access floating IPs on the Floating IPs dashboard or list view in the Network and Security section.

For information about floating IP addresses and their role in Flow Networking, see SNAT and Floating IP Address.

Go to the Floating IPs dashboard by clicking Network and Security > Floating IPs on the left side-bar.

Figure. Floating IPs dashboard Click to enlarge Displaying the Floating IP dashboard

To view or select actions you can perform on a floating IP address assigned, select the floating IP address and click the Actions drop down. The following actions are available for a selected floating IP address:

  • Update—Assign or change the assignment of the floating IP address. You can assign the floating IP address to a IP address such as a private IP address in a VPC or the primary IP address of a VM or a secondary IP address created on a VM.
  • Delete—Delete the floating IP address. The deleted IP address returns to the IP address pool as unused. Before you delete, ensure that it is not assigned to a private IP address or a VM. Change the assignment to None if it is already assigned, using the Update action.
Note: Floating IP addresses are not reachable (Pings fail) unless you associate them to primary or secondary IP addresses of VMs. For more information about assigning floating IP addresses to secondary IP addresses of VMs, see Assigning Secondary IP Addresses to Floating IPs .

To filter the list of floating IP address assignments, click the Filters option and select the appropriate filtering parameters.

To request floating IP addresses, see Requesting Floating IPs.

Connectivity

You can access network Gateways, VPN connections and subnet extensions on the Connectivity dashboard.

Click Network & Security > Connectivity to see the Connectivity dashboard.

The Connectivity dashboard opens on the Gateways tab. To see the VPN connections, click the VPN Connections tab. To see the subnets extended across AZs, click the Subnet Extensions tab.

Gateways Summary View

The Connectivity dashboard opens on the Gateways dashboard or summary view.

The Gateway dashboard provides a list of gateways created for the clusters managed by the Prism Central.

The Gateways dashboard provides a Create Gateway dropdown menu that lets you create a Local or a Remote gateway. You can create a local or remote gateway with VPN or VTEP service. For more information, see Creating a Network Gateway.

You can select a gateway from the list (select the checkbox provided for the gateway) and then perform an action provided in the Actions dropdown list. The Actions dropdown list allows you to Update or Delete the selected gateway.

Figure. Gateways dashboard Click to enlarge Displaying the Connectivity dashboard with the Gateways dashboard

The Gateway summary list view provides the following details about the gateway.

Table 1. Gateway List Fields
Parameter Description Values
Name Displays the name of the gateway. (Name of gateway)
Type Displays the gateway type. (Local or Remote)
Service Displays the service that the gateway uses. (VPN or VTEP)
Service IP Displays the IP address used by the service. (IP address)
Status Displays the operational status of the gateway. (Up or Down)
Attachment Type/Vendor Displays the type of subnet associated with the gateway. (VLAN or Overlay-VPC name)
Connections Displays the number of service connections (such as VPN connections) configured and operational on the gateway. (number)

You can click the name of a gateway to open the gateway details page that presents the information about the gateway in widgets.

Gateway Details View

You can click the name of a gateway in the Gateway dashboard list to open the gateway details page that presents the information about the gateway in widgets.

The gateway details page displays the name of the gateway on the top left corner.

  • On the top right corner, the close button (X) allows you to close the details page.

  • The Update button opens the Update Gateway page. See Updating Gateways for more information.

  • The Delete button allows you to delete the gateway. See Deleting Gateways for more information.

Figure. Gateway Details View Click to enlarge Displays the gateway details page that provides details of the gateway in two widgets - Properties and Service configuration

The details about the gateway are organized in widgets as follows:

Table 1. Gateway Details
Parameter Description Values
Properties widget
Type Displays the gateway type. (Local or Remote)
Attachment Type Displays the network entity like VLAN or VPC that the gateway is attached to. (VLAN or VPC)
VPC or Subnet (VLAN) Displays the name of the attached VPC or VLAN subnet. (Name of VLAN or VPC)
Floating or Private IP Address Displays the Floating (for VPC) or Private (for VLAN) IP address assigned to the gateway. (IP Address)
Status Displays the operational status of the gateway. (Up or Down)
Gateway Version Displays the version of the Nutanix gateway appliance deployed. (Version)
Cluster Displays the name of the cluster on which the gateway is created. (Cluster name)
Gateway VM Displays the name of the VM on which the gateway is created. (Name of VM - actionable link. Click the name-link to open the VM details page of the gateway VM.)
Service Configuration
Service Displays the service used by the gateway. (VPN or VTEP)
External Routing Displays the type of routing associated with the gateway for external traffic routing. (Static or eBGP with ASN)
Internal Routing Displays the type of routing associated with the gateway for internal traffic routing. (Static or eBGP with ASN)
VPN Connections Displays the total number of VPN connections associated with the gateway. (Number - actionable link. Click the link to open the VPN connection details page for the associated VPN connection.)
View VPN Connections Click this link to open the VPN Connections tab. -

VPN Connections Summary View

The Connectivity dashboard allows you to open the VPN Connections dashboard or summary view.

VPN Connection: Represents the VPN IPSec tunnel established between local gateway and remote gateway. When you create a VPN connection, you need to select two gateways between which you want to create the VPN connection.

The VPN Connections dashboard provides a list of VPN connections created for the clusters managed by the Prism Central.

The VPN Connections dashboard provides a Create VPN Connection button that opens the Create VPN Connection . For more information, see Creating a VPN Connection.

You can select a VPN connection from the list (select the checkbox provided for the VPN connection) and then perform an action provided in the Actions dropdown list. The Actions dropdown list allows you to Update or Delete the selected VPN connection.

The VPN Connections summary list view provides the following details about the VPN connection.

Figure. VPN Connections dashboard Click to enlarge Displaying the VPN Connections dashboard.

Table 1. VPN Connections List Fields
Parameter Description Values
Name Displays the name of the connection. (gateway name)
IPSec Status Displays the connection status of IPSec tunnel. (Connected or Not Connected)
EBGP Status Displays the status of the EBGP gateway connection. (Established or Not Established)
Local Gateway Displays the name of the local gateway used for the connection. (Name of local gateway)
Remote Gateway Displays the name of the remote gateway used for the connection. (Name of remote gateway)
Dynamic Routing Priority Displays the dynamic routing priority assigned to the connection for throughput management. You can assign any value in the range of 100-1000. Flow networking assigns the first VPN connection the value 500 by default. Thereafter, subsequent VPN connections are assigned values decremented by 50. For example, the first connections is assigned 500, then the second connection is assigned 450, the third one 400 and so on. (Number in the range of 100-1000. User assigned.)

VPN Connections Details View

You can click the name of a VPN connection in the VPN Connections dashboard list to open the VPN connection details page that presents the information about the VPN connection in widgets.

The VPN connection details page displays the name of the VPN connection on the top left corner.

  • On the top right corner, the close button (X) allows you to close the details page.

  • The Update button opens the Update VPN Connection page. For more information, see Updating a VPN Connection.

  • The Delete button allows you to delete the VPN connection. For more information, see Deleting a VPN Connection.

Figure. VPN Connection Details Click to enlarge Displaying the detailed view of the selected VPN connection with the information organized in widgets.

The details about the VPN connection are organized in widgets as follows:

  • Summary tab—See the VPN Connection Summary Tab Details table below.
  • Throughput tab—See the VPN Connection Throughput Tab Details table below.
  • IPSec Logging tab—Provides logs for the IPSec tunnel.
  • Routing Protocol Logging tab—Provides logs for the routing protocol used in the VPN connection.
Table 1. VPN Connection Summary Tab Details
Parameter Description Values
VPN Connection widget
IPSec Status Displays the connection status of IPSec tunnel. (Connected or Not Connected)
EBGP Status Displays the status of the EBGP gateway connection. (Established or Not Established)
Dynamic Routing Priority Displays the dynamic routing priority assigned to the connection for throughput management. You can assign any value in the range of 100-1000. Flow networking assigns the first VPN connection the value 500 by default. Thereafter, subsequent VPN connections are assigned values decremented by 50. For example, the first connections is assigned 500, then the second connection is assigned 450, the third one 400 and so on. (Number in the range of 100-1000. User assigned.)
Local Gateway Properties
Gateway Name Displays the name of the local gateway used for the connection. (Name of local gateway)
Type Displays the type of gateway. (Local)
Attachment Type Displays the network entity like VLAN or VPC that the gateway is attached to. (VLAN or VPC)
VPC or Subnet (VLAN) Displays the name of the attached VPC or VLAN subnet. (Name of VLAN or VPC)
Tunnel IP Displays the Tunnel IP address of the local gateway. (IP Address)
Connection Type Displays the connection type you selected while creating the VPN connection. The connection type may be Initiator or Acceptor of a VPN connection between the local and remote gateways. T (Initiator or Acceptor)
External Routing Displays the type of routing associated with the gateway for external traffic routing. (Static or eBGP with ASN)
Internal Routing Displays the type of routing associated with the gateway for internal traffic routing. (Static or eBGP with ASN)
Floating or Private IP Address Displays the Floating (for VPC) or Private (for VLAN) IP address assigned to the gateway. (IP Address that you assigned to the local gateway with /30 prefix when you configured the VPN connection.)
Status Displays the operational status of the gateway. (Up or Down)
Cluster Displays the name of the cluster on which the gateway is created. (Cluster name)
Gateway VM Displays the name of the VM on which the gateway is created. (Name of VM - actionable link. Click the name-link to open the VM details page of the gateway VM.)
Remote Gateway Properties
Gateway Name Displays the name of the remote gateway used for the connection. (Name of remote gateway)
Type Displays the type of gateway. (Remote)
Tunnel IP Displays the Tunnel IP address of the remote gateway. (IP Address)
Connection Type Displays the connection type you selected while creating the VPN connection. The connection type may be Initiator or Acceptor of a VPN connection between the local and remote gateways. T (Initiator or Acceptor)
External Routing Displays the type of routing associated with the gateway for external traffic routing. (Static or eBGP with ASN)
ASN Displays the ASN of the EBGP route. This information is only displayed if you configured EBGP as the External Routing protocol. (Number)
Vendor Displays the name of the vendor of the gateway appliance at the remote site. (Name of vendor of gateway appliance)
External IP Displays the IP address assigned to remote the gateway. (IP Address that you assigned to the remote gateway with /30 prefix when you configured the VPN connection.)
Status Displays the operational status of the gateway. -
Protocol Details
Service Displays the service used by the gateway. (VPN or VTEP)
Gateway Routes Displays the status of the routes used by the gateways. (Sent)

Subnet Extensions Summary View

The Connectivity dashboard opens on the Subnet Extensions dashboard or summary view.

The Subnet Extensions dashboard provides a list of subnet extensions created for the clusters managed by the Prism Central.

The Subnet Extensions dashboard provides a Create Subnet Extension dropdown menu that lets you extend a subnet Across Availability Zones or To a Third Party Data Center . You can extend a subnet using VPN or VTEP service. See Layer 2 Virtual Network Extension for more information.

You can select a subnet extension from the list (select the checkbox provided for the subnet extension) and then perform an action provided in the Actions dropdown list. The Actions dropdown list allows you to Update or Delete the selected subnet extension.

Figure. Subnet Extensions dashboard Click to enlarge Displaying the Subnet Extension dashboard.

The Subnet Extensions summary list view provides the following details about the gateway.

Table 1. Subnet Extensions List Fields
Parameter Description Values
Name Displays the name of the subnet extension. (Name of subnet extension)
Type Displays the subnet extension type. ( Across Availability Zones or To a Third Party Data Center )
Extension Over Displays the service that the subnet extension uses. (VPN or VTEP)
Extension Uses Displays the name of the local network gateway that the subnet extension uses. (Name of local network gateway)
Local Subnet Displays the name of the local subnet that the subnet extension uses. (Name of local subnet)
Remote Site Displays the name of the remote network gateway that the subnet extension uses. (Name of remote network gateway)
Connection Status Displays the status of the connection that is created by the subnet extension. Not Available status indicates that Prism Central is unable to ascertain the status. (Not Available, Connected, or Disconnected)
Interface Status Displays the status of the interface that is used by the subnet extension. (Connected or Down)

You can click the name of a subnet extension to open the subnet extension details page that presents the information about the subnet extension in widgets.

Subnet Extensions Details View

You can click the name of a subnet extension in the Subnet Extensions dashboard list to open the subnet extension details page that presents the information about the subnet extension in widgets.

The subnet extension details page displays the name of the subnet extension on the top left corner. It has two tabs - Summary and Address Table . The Summary tab provides the information about the subnet extension in widgets. The Address Table tab provides MAC Address information only when the subnet extension uses VTEP service.

  • On the top right corner, the close button (X) allows you to close the details page.

  • The Update button opens the Update Subnet Extension page. See Updating an Extended Subnet for more information.

  • The Delete button allows you to delete the subnet extension. See Removing an Extended Subnet for more information.

Figure. Subnet Extensions Details View - Summary Tab Click to enlarge Displays the subnet extension details page, Summary that provides details of the subnet extension in one extended widget with three sections - Properties, IP Address Pools and Subnet Extension properties.

Figure. Subnet Extensions Details View - Address Table Tab for VPN-based Extension Click to enlarge Displays the subnet extension details page, Address Table tab that provides details of the MAC Addresses in the subnet extension

Figure. Subnet Extensions Details View - Address Table Tab for VTEP-based Extension Click to enlarge Displays the subnet extension details page, Address Table tab that provides details of the MAC Addresses in the subnet extension

The details about the subnet extension are organized in two tabs. The Summary tab organizes the subnet extension details in the extended widget as provided in the table. The Address Table tab provides details about the MAC addresses in a list.

Table 1. Subnet Extension Details - Summary Tab Fields
Parameter Description Values
Properties
Type Displays the subnet type. (VLAN or Overlay)
VLAN ID (For VLAN subnets only) Displays the VLAN ID of the VLAN subnet that is extended. (VLAN ID number)
VPC (For Overlay subnets only) Displays the name of the VPC subnet that is extended. (Name of VPC)
Cluster (For VLAN subnets only) Displays the cluster that the VLAN subnet belongs to. (Name of cluster)
IP Address Prefix Displays the network IP address with prefix, of the VLAN subnet that is extended. (IP Address with prefix)
Virtual Switch (For VLAN subnets only) Displays the virtual switch on which the VLAN subnet is configured. (Virtual Switch name such as vs0 or vs1)
IP Address Pools
Pool Range Displays the range of IP addresses in the pool configured in the subnet that is extended. (IP address range)
(Interactive Graphic Pie Chart) Displays a dynamic pie chart that displays the statistic you hover on. Displays the following IP address statistics outside the pie chart, that you can hover on:
  • Total number of IP addresses available.
  • Used IP addresses in the subnets
  • Used IP addresses in the IP address pools
  • Free IP addresses in the subnets
  • Free IP addresses in the IP address pools
(IP Address statistics)
Subnet Extension
Subnet Extension (properties) - Common
Type Displays the subnet extension type. ( Across Availability Zones or To a Third Party Data Center )
Interface Status Displays the status of the interface that is used by the subnet extension. (Connected or Down)
Connection Status Displays the status of the connection that is created by the subnet extension. Not Available status indicates that Prism Central is unable to ascertain the status. (Not Available, Connected, or Disconnected)
Local IP Address Displays the IP address that you entered in the Local IP Address field while creating the subnet extension. (IP Address)
Local Subnet Displays the name of the local subnet that the subnet extension uses. (Name of local subnet)
Subnet Extension (properties) - (Only for Across Availability Zones type)
Local Availability Zone (Only for Across Availability Zones type) Displays the name of the local AZ that is hosting the subnet that is extended. (Name of the local Availability Zone)
Remote Availability Zone (Only for Across Availability Zones type) Displays the name of the remote AZ that the subnet is extended to. (Name of the remote Availability Zone)
Remote Subnet (Only for Across Availability Zones type) Displays the name of the remote subnet that the subnet extension connects to. (Name of remote subnet)
Remote IP Address (Only for Across Availability Zones type) Displays the IP address that you entered in the Remote IP Address field while creating the subnet extension. (IP Address)
Subnet Extension (properties) - (Only for To a Third Party Data Center type)
Local Gateway (Only for To a Third Party Data Center type) Displays the name of the local gateway used for the subnet extension. (Name of local gateway)
Remote Gateway (Only for To a Third Party Data Center type) Displays the name of the remote gateway used for the subnet extension. (Name of remote gateway)

Security Policies Summary View

To access the security policies dashboard, select Policies > Security Policies from the entities menu (see Entities Menu). The security policies dashboard allows you to view summary information about defined security policies.

Note: This section describes the information and options that appear in the security policies dashboard.
  • See Entity Exploring for instructions on how to view and organize that information in a variety of ways.
  • See Flow Microsegmentation Guide for information about how to create and apply security policies.
Figure. Security Policies Dashboard Click to enlarge Security policies view of the Explore dashboard

The following table describes the fields that appear in the security policies list. A dash (-) is displayed in a field when a value is not available or applicable.

Table 1. Security Policies List Fields
Parameter Description Values
Name Displays the policy name. The policy is one of three types: application, quarantine, or isolation. (name), Application, Quarantine, Isolation
Purpose Describes (briefly) the policy's purpose. (text string)
Policy Displays (high level) what the policy does. (boxed text)
Status Displays the current status of the policy (either applied currently or in monitoring mode). Applied, Monitoring
Last Modified Displays the date the policy was last modified (or the creation date if the policy has never been modified). (date)

You can filter the security polices list based on several parameter values. The following table describes the filter options available when you open the Security Policies view Filter pane. To apply a filter, select a parameter and check the box of the desired value (or multiple values) you want to use as a filter. You can apply filters across multiple parameters.

Table 2. Filter Pane Fields
Parameter Description Values
Name Filters on the item name. Select a condition from the pull-down list ( Contains , Doesn't contain , Starts with , Ends with , or Equal to ) and enter a string in the field. It will return a list of security policies that satisfy the name condition/string. (policy name string)
Type Filters on the policy type. Check the box for one or more of the policy types (application, quarantine, isolation). It will limit the list to just those policy types. Application, Quarantine, Isolation
Status Filters on the policy status. Check the box for applied or monitoring. Applied, Monitoring

The security policies dashboard includes a Create Security Policy action button with a drop-down list to Secure an Application or Isolation Environments .

The Actions menu appears when one or more policies are selected. It includes options to update, apply, monitor, and delete. The available actions appear in bold; other actions are grayed out. (For grayed out options, a tool tip explaining the reason is provided.)

Security Policy Details View

To access the details page for a security policy, click on the desired security policy name in the list (see Security Policies Summary View). The Security Policy details page includes the following:

  • The policy name appears in the upper left. You can switch from one policy to another by selecting the policy name from the pull-down list.
  • The rule status appears below the name and indicates whether the policy is being applied currently or is in monitoring mode.
  • Three columns appear that specify the Inbound policy (on the left), the affected entities (in the middle), and the Outbound policy (on the right).
  • There are three action buttons (upper right).
    • Click the appropriate button to update, apply, monitor, or delete the policy (see Nutanix Security Guide for details). The available actions appear in bold; other actions are grayed out. (For grayed out options, a tool tip explaining the reason is provided.)
    • Click the question mark icon to open a help page in a separate tab or window.
    • Click the X icon to close the details page.
Figure. Security Policy Details View: Monitoring Rule Example Click to enlarge Security policies view of the Explore dashboard

Figure. Security Policy Details View: Applied Rule Example Click to enlarge Security policies view of the Explore dashboard

For more information about Security Policies, see Flow Microsegmentation Guide.

Virtual Private Cloud

A Virtual Private Cloud (VPC) is an independent and isolated IP address space that functions as a logically isolated virtual network. A VPC could be made up of one or more subnets that are connected through a logical or virtual router. The IP addresses within a VPC must be unique. However, IP addresses may overlap across VPCs. As VPCs are provisioned on top of another IP-based infrastructure (connecting AHV nodes), they are often referred to as the overlay networks. Tenants may spin up VMs and connect them to one or more subnets within a VPC.

Virtual Private Cloud (VPC) is a virtualized network of resources that are specifically isolated from the rest of the resource pool. VPC allows you to manage the isolated and secure virtual network with enhanced automation and scaling. The isolation is done using network namespace techniques like IP-based subnets or VLAN based networking.

AHV provides the framework to deploy VPC on on-premises clusters using the following.

  • Advanced Networking subnets and DHCP management
  • Multiple uplink and bridge management via virtual switch (VS)
  • Virtual Private Network (VPN) gateways and connections

Flow Networking simplifies the deployment and configuration of overlay-based VPCs. It allows you to quickly:

  • Create, update and delete VPCs.
  • Create, update and delete subnets within VPCs.
    Note: Create subnets as necessary when you create VPCs.
  • Add network security policies and services.
  • Configure hybrid cloud connectivity with VPNs.

This section covers the concepts and procedures necessary to implement VPCs in the network.

VM IP Address Management

Primary Address

The primary IP address is assigned to a VM during initialization when the cluster provides any virtual NIC (NIC) to a VM.

  • Select Assign Static IP as the Assignment Type to add a static IP address as primary IP address of the VM, when you attach a subnet to a VM.
  • Select Assign with DHCP as the Assignment Type to allow DHCP to dynamically assign an IP address to the VM.
  • Select No Private IP as the Assignment Type if you do not want to assign an IP address to the vNIC of the VM.

For more information about attaching a subnet to a VM, see Creating a VM through Prism Central (AHV) in the Prism Central Guide .

Secondary IP Addresses (Overlay Networks only)

For your deployment, you may need to configure multiple (static) IP addresses to a single NIC. These IP addresses (other than the primary IP address) are secondary IP addresses. A secondary IP address can be permanently associated with a specific NIC or be changed to any other NIC. The NIC ownership of a secondary IP address is important for security routing policies.

Note: You can configure secondary IP addresses only for VMs in an Overlay network.

You can configure secondary IP addresses to a NIC when you want to:

  • Associate multiple floating IP addresses with one VM without creating multiple NICs (each with one primary IP address) for the VM. You can assign one floating IP address to one secondary IP address that you create for the single NIC. For information about floating IP addresses, see Requesting Floating IPs.
  • Run appliances, such as load balancers, that have multiple IP addresses on each interface.
  • Host applications in a High Availability (HA) configuration where the ownership of IP address moves from the active entity to the standby entity when the active entity goes down.
  • Host applications in a clustered configuration where the ownership of IP address follows the leader.
  • Host Nutanix Files service in a VPC as a case of clustered application.
Note:

In applications that use secondary IP addresses as virtual IP addresses and the NIC ownership of the secondary IP address changes dynamically from one NIC to another, configure the application to incorporate the ownership change in its settings or configuration. If the applications do not incorporate these ownership changes, the VPCs configured for such applications fail.

For information about configuring secondary IP addresses, see Creating Secondary IP Addresses.

IP Address Information

You can view the IP addresses configured on a VM by clicking the See More link in the IP Address column in the VM details view to open the IP Address Information box.

Note: The See More link in the IP Address column in the VM details view and the IP Address Information box are available only if the VM has any secondary IP addresses configured.
Figure. IP Address Information Click to enlarge Displaying the IP Address Information box

Creating Secondary IP Addresses

You can assign multiple secondary IP addresses to a single vNIC.

About this task

You can add multiple secondary IP addresses to the vNIC configured on a VM. Add the secondary IP addresses to the vNIC in the Create VM or the Update VM page.

Procedure

  1. Go to the Networks section.
  2. Click the Edit icon for the subnet that you want to add the secondary IP addresses from.
    The Update NIC dialog box opens.
  3. Check the Add Secondary IPs check box in the Update NIC dialog box.
    Figure. Add Secondary IP Addresses Click to enlarge Displaying the Add Secondary IPs section in Update NIC page.

  4. Add a comma-separated list of the secondary IP addresses that you want to add to the vNIC of the VM.
    Note:

    Ensure that the secondary IP addresses are within the same subnet that the primary IP address of the NIC is from. The subnets are displayed in the Private IP Assignment section in the Update NIC dialog box.

    Ensure that the secondary IP address is not the same as the IP address provided in the Private IP Assignment field.

  5. Click Save .
  6. Click Next on the Resources and the Management tabs of the Update VM page.

    If you need to make any other changes on the Resources and the Management tabs for any configurations other than adding secondary IP addresses, make the changes and then click Next on these tabs.

  7. Click Launch VM on the Review tab after you review

What to do next

You can view the secondary IP addresses configured on the VM in the IP Address Information box.

Assigning Secondary IP Addresses to Interfaces

Assign the secondary IP addresses to interfaces or subinterfaces on the VM.

About this task

To assign the secondary IP addresses to virtual interfaces on the VM, do the following on the VM details page:

Procedure

  1. Click Console .
  2. Log in as a root user.
  3. Run the ifconfig command as follows:
    root@host$ ifconfig <interface> <secondary ip address> <network mask>

    Provide the following in the command:

Parameter Description
<interface> The interface of the VM such as eth0. You can provide subinterfaces such as eth0:1 and eth0:2.
<secondary IP address> The secondary IP address that you created and want to associate with the interface.
<network mask> The network mask that is an expansion of the network prefix of the network that the secondary IP address belongs to. For example, if the secondary IP address belongs to 10.0.0.0/24 then the network mask is 255.255.255.0.
  1. Repeat the aforementioned steps for all the secondary IP addresses you want to associate with interfaces on the VM.
  2. Exit from the Console.

Assigning Secondary IP Addresses to Floating IPs

Assign the secondary IP addresses to floating IP addresses on the VM.

About this task

After you assign secondary IP addresses to interfaces or subinterfaces on the VM, you can assign the secondary IP addresses to floating IP addresses that may be used for external connectivity.

Do one of the following:

Procedure

  • Assign floating IP addresses when you request floating IP addresses in the Assign Floating IPs section of the Request Floating IP dialog box.
    To assign floating IP addresses while requesting for them, you must have the secondary IP addresses configured and ready when you are requesting the floating IP addresses.
  • Select the floating IP address you want to assign, in the Floating IPs dashboard. Click the Update option in the Actions drop-down menu.
    Assign the secondary IP addresses you configured to the floating IP addresses you have.

VPC Workflow

A virtual private cloud (VPC) can be deployed on Nutanix cluster infrastructure to manage the internal and external networking requirements using Flow Networking. The workflow to create a complete network based on VPC is described below.

  1. Create a VPC—See Creating Virtual Private Cloud. See Updating Virtual Private Cloud to update a VPC you created.
  2. Add Subnets to the VPC—See Creating a Subnet to create a Subnet. See Updating a Subnet to update a subnet.
  3. Attach the Subnet to VMs—See Attaching a Subnet to a Virtual Machine.

VPC Management

This section provides information and procedures that you need to manage virtual private clouds using Flow networking.

Creating Virtual Private Cloud

About this task

You can create VPCs on the Virtual Private Clouds page. Go to the Virtual Private Clouds page by clicking Virtual Infrastructure > Networking > Virtual Private Clouds .

To create a VPC, do the following.

Procedure

  1. On the VPC dashboard, click Create VPC .

    See Network and Security View for more information about the VPC dashboard.

    The Create Virtual Private Cloud (VPC) dialog box opens.
    Figure. Create Virtual Private Cloud Click to enlarge

  2. Provide the necessary values in respective fields in the Create Virtual Private Cloud (VPC) dialog box.
Fields Description and Values

Name

Provide a name for the VPC.

External Connectivity

This section takes you through configuration of the parameters necessary for connectivity to the Internet or clusters outside the VPC.

A subnet with external connectivity (External Subnet) is required if the VPC needs to send traffic to a destination outside of the VPC.

Note: You can add a maximum of two external subnets - one external subnet with NAT and one external subnet without NAT to a VPC. Both external subnets cannot be of the same type. For example, you cannot add two external subnets, both with NAT. You can update an existing VPC similarly.

Network address translation (NAT) Gateways perform the required IP-address translations required for external routing. You can also have external connectivity without NAT.

External Subnet

Select an external subnet from the drop down list. By associating the VPC with the external subnet you can provide external connectivity to the VPC.
Note:

Ensure that the externally routable IP addresses (subnets with external connectivity without NAT) for different VPCs do not overlap.

Configure the routes for the external connectivity subnets with next hop as the Router or SNAT IP address. Also configure the routes on the router for the return traffic to reach the VPC. See External Connectivity panel in VPC Details View.

Externally Routable IP Addresses Provide IP addresses that are externally routable. Externally routable IP addresses are IP addresses that within the VPC which can communicate externally without NAT. These IP addresses are used when an external subnet without NAT is used.

Domain Name Servers (DNS)

(Optional) DNS is advertised to Guest VMs via DHCP. This can be overridden in the subnet configuration.

Click + Server IP to add DNS server IPs under IP Address and click the check mark.

You can Edit or Delete an IP address you added using the options under Actions .

  1. Click Save .

Requesting Floating IPs

About this task

Each VPN gateway requires a floating IP. If you do not provide one during the VPN gateway creation, then Flow Networking automatically allocates a floating IP to a VPN gateway. To provide floating IP during the VPN gateway creation, you can request floating IPs and assign them to VMs.

You can view the allocated floating IPs on the Floating IPs page. Click Networking > > Floating IPs .

To request a floating IP, do the following.

Procedure

  1. Click the Request Floating IP button on the Floating IPs page.
  2. On the Request Floating IP dialog box, provide the information in the respective fields.
    Figure. Request and Assign Floating IPs Click to enlarge

    Note:

    Uncheck the Assign Floating IPs box if you want to assign the requested IP addresses after you receive it.

    See Floating IPs for more information.

Fields Description and Values
External Subnet Select a subnet that you configured with external connectivity.
Number of Floating IPs Enter the number of floating IPs you want. You can request a maximum of 5 floating IP addresses.
Assign Floating IPs

Select this check box if you want to assign the floating IPs to specific VMs in the table.

Based on the number you entered in the Number of Floating IPs field, the system provides an equivalent number of rows of Search VMs and IP Address in the table.

Under Search VMs , select the VM to which you want to assign a floating IP address. Under IP Address , select the IP address on the VM (primary or secondary IP address) to which you want to assign the floating IP.

You can assign multiple floating IP addresses to multiple secondary IP addresses that you can create on the NIC of the VM.

For information about configuring secondary IP addresses, see Creating Secondary IP Addresses.

Note:
  1. Click Save .

What to do next

When you receive the floating IP address you requested, you can see it, assign it (if not already assigned while requesting) or delete it in the Floating IPs view.

Creating a Subnet

About this task

You can create subnets on the Subnets page. Go to the Subnets page by clicking Virtual Infrastructure > Networking and open the Create Subnet dialog box.

You can also open the Create Subnet dialog box from the VPC details view by clicking the Add Subnet option.

To create a subnet, do the following.

Procedure

  1. Click Create Subnet .
    The Create Subnet dialog box opens. The following figure displays the Create Subnet dialog box with all the options. These options are displayed based on the values you select in the Type field.
    Figure. Create Subnet (With External Connectivity Disabled) Click to enlarge

    Figure. Create Subnet (With External Connectivity Enabled) Click to enlarge

Fields Description and Values
Name Provide a name for the subnet.
Type

Select the type of subnet you want to create.

You can create a VLAN subnet or an Overlay subnet.

VLAN ID

(VLAN subnet only) Enter the number of the VLAN .

Enter just the number in this field, for example 1 or 27. Enter 0 for the native VLAN. The value is displayed as vlan.1 or vlan.27 in the View pages.

Note: Provision any single VLAN ID either in the AHV network stack or in the Flow Networking (brAtlas) networking stack. Do not use the same VLAN ID in both the stacks.
IP Address management

(Mandatory for Overlay type subnets) This section provides the Network IP Prefix and Gateway IP fields for the subnet.

(Optional for VLAN type subnet) Check this box to display the Network IP Prefix and Gateway IP fields and configure the IP address details.

Unchecking this box hides these fields. In this case, it is assumed that this virtual LAN is managed outside the cluster.

Note:

The DHCP Settings option is only available for VLAN subnets if you select this option.

DHCP Settings

(Optional for both VLAN and Overlay subnets) Check this box to display fields for defining a domain.

Checking this box displays fields to specify DNS servers and domains. Unchecking this box hides those fields.

See Settings the DHCP Options for more information.

Cluster (VLAN subnet only) (VLAN subnet only) This option is available only for VLAN subnet configuration. Select the cluster that you want to assign to the subnet.
External Connectivity (VLAN subnet only) Turn on this toggle switch if you want use this VLAN subnet for external connectivity.
Note:

Ensure that the externally routable IP addresses (subnets with external connectivity without NAT) for different VPCs do not overlap.

Configure the routes for the external connectivity subnets with next hop as the Router or SNAT IP address. Also configure the routes on the router for the return traffic to reach the VPC. See External Connectivity panel in VPC Details View.

NAT (Option under External Connectivity ) If you turn on the External Connectivity toggle switch, then you can choose whether to connect to external networks with or without enabling NAT. Check the NAT check box to enable NAT for external connectivity for VPCs.

Virtual Switch (VLAN subnet only) Select the virtual switch that is configured for the VLAN you selected. The default value is the default virtual switch vs0. This option is displayed only if you add a VLAN ID in the VLAN ID field.
VPC (Overlay subnet only)

Select the Virtual Private Cloud (VPC) that you want to assign to the subnet from the drop down list.

You can create VPCs and assign them to Overlay subnets.

IP Address Pool

Defines a range of addresses for automatic assignment to virtual NICs.

This field is optional for both VLAN and Overlay . For VLAN , this field is displayed only if you select the IP Address Management option.

Note: Configure this field for VLAN or Overlay to complete the creation of the VPC, if you do not need external connectivity for this subnet. You must configure this field only if you need external connectivity for this subnet.

Click the Create Pool button and enter the following in the Add IP Pool page:

  • Enter the starting IP address of the range in the Start Address field.

  • Enter the ending IP address of the range in the End Address field.

  • Under Actions , click the check mark to submit the starting and ending IP addresses you entered.

    Click the X mark to remove the entries.

Override DHCP Server

(VLAN subnet only) To configure a DHCP server, check the Override DHCP Server box and enter an IP address in the DHCP Server IP Address field.

See Override DHCP Server (VLAN Only) in Settings the DHCP Options for information about this option.

  1. Click Save .

Settings the DHCP Options

About this task

Selecting the DHCP Settings checkbox in Create Subnet or Update Subnet allows you to configure the DHCP options for the VMs within the subnet. When DHCP settings are configured for a VM in a subnet and the VM is powered on, Flow Networking configures these options on the VM automatically. If you do not configure the DHCP settings, then these options are not available on the VM automatically when you power it on.

You can enable DHCP Settings when you create a subnet and configure the DHCP Settings for the new subnet. You could also update the DHCP Settings for an existing subnet.

DHCP Settings is common to and is available on both the Create Subnet and the Update Subnet dialog boxes.

To configure the DHCP Settings , do the following in the Create Subnet or the Update Subnet dialog box:

Procedure

  • Provide the information in the DHCP Settings fields.
    Figure. DHCP Settings Click to enlarge DHCP Settings display

Fields Description and Values
Domain Name Servers

Provide a comma-separated list of DNS IP addresses.

Example: 8.8.8.8, 9.9.9.9

Domain Search

Enter the VLAN domain name. Use only the domain name format.

Example: nutanix.com

TFTP Server Name

Enter a valid TFTP host server name of the TFTP server where you host the host boot file. The IP address of the TFTP server must be accessible to the virtual machines to download a boot file.

Example: tftp_vlan103

Boot File Name

The name of the boot file that the VMs need to download from the TFTP host server.

Example: boot_ahv2020xx

  • (Optional and for VLAN networks only) Check the Override DHCP Server dialog box and enter an IP address in the DHCP Server IP Address field.

    You can configure a DHCP server using the Override DHCP Server option only in case of VLAN networks.

    The DHCP Server IP address (reserved IP address for the Acropolis DHCP server) is visible only to VMs on this network and responds only to DHCP requests. If this box is not checked, the DHCP Server IP Address field is not displayed and the DHCP server IP address is generated automatically. The automatically generated address is network_IP_address_subnet.254 , or if the default gateway is using that address, network_IP_address_subnet.253 .

    Usually the default DHCP server IP is configured as the last usable IP in the subnet (For eg., its 10.0.0.254 for 10.0.0.0/24 subnet). If you want to use a different IP address in the subnet as the DHCP server IP, use the override option.

Attaching a Subnet to a Virtual Machine

About this task

To attach a subnet to a VM, go to the Virtual Infrastructure > VM > List view in Prism Central and do the following.

Procedure

  1. Select the VM you want to attach a subnet to. Click Actions > > Update .
  2. In the Update VM dialog box, click Add NIC .
    Figure. Click to enlarge

  3. Provide the necessary information in the indicated fields in the Create NIC dialog box.
    1. Select the Subnet Name from the drop down list.
    2. Select the Network Connection State as Connected or Disconnected .

      The Network Connection State selection defines the state of the connection after the NIC configuration is implemented.

    3. Select the Assignment Type .

      You can select Assign with DHCP to assign a DHCP based IP address to the VM.

      You can select Assign Static IP to assign a static IP address to the VM to reach the VM quickly from any endpoint in the network such as a laptop.

    4. Click Add .
  4. Click Save on the Update VM dialog box.

Creating a Policy

About this task

For Policy-based routing you need to create policies that route the traffic in the network.

When you create a VPC, there is one default policy that Flow Networking creates for the VPC. This policy is pre-configured with the Priority 1 and other default values to Deny traffic flow and service (see the table of field descriptions and values for this dialog box).
Note: You cannot update or delete the default policy.
  • Policies control the traffic flowing between subnets (inter-subnet traffic).

  • Policies control the traffic flowing in and out of the VPC.

  • Policies do not control the traffic within a subnet (intra-subnet traffic).

Figure. Policy Tab Click to enlarge

You can create a traffic policy using the Create Policy dialog box. You can open the Create Policy dialog box either from the VPC list view or the VPC list view.

  • On the VPC list view, select the VPC you want to update and click Create Policy in the Actions drop down menu.

  • On the VPC details view, click the Create Policy option in the More drop down menu.

To create a policy, do the following in the Create Policy dialog box.

Procedure

  1. Provide the necessary values in the respective fields.
    Figure. Create Policy Click to enlarge

Fields Description and Values Value in Default Policy
Priority The priority of the access list (ACL) determines which ACL is processed first. Priority is indicated by an integer number. A higher priority number indicates a higher priority.For example, if two ACLs have priority numbers 100 and 70 respectively, the ACL with priority 100 takes precedence over the ACl with priority 70.
Note:
  • Click the Understand Priorities link to see the Understand Priorities information box (see the image of this box below this table).
1
Source

The source indicates the source IP or subnet for which you want to manage traffic.

Source can be:

  • Any : Indicates any IP address.

  • External : Indicates an IP address that is outside the subnets configured for the VPC.

  • Custom : You can provide a specific Source Subnet IP with prefix.
Any
Source Subnet IP

Only required if you selected the Source as Custom . Provide the subnet IP and prefix that you want to designate as the source for the policy. Use the CIDR notation format to provide the subnet IP. For example, 10.10.10.0/24.

None
Destination

The destination is the destination IP or subnet for which you want to set the priority.

Destination can be:

  • Any : Indicates any IP address.

  • External : Indicates an IP address that is outside the subnets configured for the VPC.

  • Custom : You can provide a specific Destination Subnet IP with prefix.
Any
Destination Subnet IP

Only required if you selected the Destination as Custom .

None
Protocol You can also set the priority configure policy for certain protocols. You can select one of the following options:
  • Any : Indicates any IP address.

  • Protocol Number : Provide an integer number that indicates the protocol you want to prioritize.

    Provide the appropriate value in the Protocol Number field.
  • TCP
  • UDP
  • ICMP
Protocol Number

This field is displayed only if you select Protocol Number as the value in the Protocol field. The number you provide must be the IANA designated number that indicates respective protocol. See IANA Protocol Numbers .

None
Action

Assign the appropriate action for implementation of the policy.

  • Permit : Permits traffic and services based on the parameters set.

    If the Permit rule is set to override a Drop rule, then the Permit rule must be set in both the directions to allow bidirectional communication between the Source and Destination .

  • Deny : Denies traffic and service based on the parameters set.

  • Re-route :Sends matching traffic to the next-hop IP address specified by the Reroute IP . In case of reroute, you need to provide an IP address that the traffic needs to be re-routed to, in the Reroute IP field.
Permit
Figure. Understanding Priorities Click to enlarge Sample Understand Priorities information box.

  1. Click Save .

Creating Static Routes

About this task

You can create a static route using the Create Static Routes dialog box. You can open the Create Static Routes dialog box either from the VPC list view or the VPC details view.

  • On the VPC list view, select the VPC and click Create Static Routes in the Actions drop down menu.

  • On the VPC details view, click the Create Static Routes option in the More drop down menu.

Figure. Create Static Routes Click to enlarge

To create static route, do the following in the Create Static Routes dialog box:

Procedure

  1. Provide the necessary values in the respective fields.
Fields Description and Values
Destination Prefix Provide the IP address with prefix of the destination subnet.
Next Hop Link Select the next hop link from the drop down list. The next hop link is the IP address that the traffic must be sent for the static route you are configuring.
Add Prefix You can create multiple static routes using this option. Click this link to add another set of Destination Prefix and Next Hop Link to configure another static route.
  1. Click Save .

Updating Virtual Private Cloud

About this task

You can update a VPC using the Update Virtual Private Cloud (VPC) dialog box. You can open the Update Virtual Private Cloud (VPC) dialog box either from the VPC list view or the VPC details view.

  • On the VPC list view, select the VPC you want to update and click Update in the Actions drop down menu.

  • On the VPC details view, click the Update option.

The Update Virtual Private Cloud (VPC) dialog box is identical to the Create Virtual Private Cloud (VPC) dialog box.

Figure. Update VPC Click to enlarge Displaying Update VPC dialog box

For details about the parameters that you can update in the Update Virtual Private Cloud (VPC) dialog box, see Creating Virtual Private Cloud.

Procedure

  • Update the parameters in the Update Virtual Private Cloud (VPC) dialog box.
  • Click Save .

Updating a Subnet

About this task

You can update a subnet displayed on the Subnets page. Go to the Subnets page by clicking Virtual Infrastructure > Networking > Subnets and open the Update Subnet dialog box.

You can also open the Update Subnet dialog box from the VPC dashboard for a specific VPC. Click the Edit option for the subnet listed on the Subnets tab of the VPC dashboard.

The fields in the Update Subnet and the Create Subnet dialog boxes are the same.
Note: You cannot edit or update the subnet type. For example, if the subnet type is already configured as VLAN , you cannot modify it to an Overlay type subnet.

To update a subnets, do the following.

Procedure

  1. Select the subnet you want to update. Select Actions > Update Subnet .
  2. Update the necessary values in the respective fields in the Update Subnet dialog box.
    Figure. Update Subnet Click to enlarge

    The Update Subnet dialog box has the same fields as the Create Subnet dialog box. For details about the fields and the values that can be updated in the Update Subnet dialog box, see Creating a Subnet.

  3. Click Save to ensure that the updates are saved in the configuration.

Category Management

A category is a key-value pair that groups similar entities. Associating a policy with a category ensures that the policy applies to all the entities in the group regardless of how the group scales with time. For example, you can associate a group of VMs with the Department: Marketing category, where Department is a category that includes a value Marketing along with other values such as Engineering and Sales.

Currently, you can associate only VMs with a category. Categories are implemented in the same way on on-premises Prism Central instances and in Xi Cloud Services. For information about configuring categories, see the Prism Central Guide .

Updating a Policy

About this task

You can update a policy using the Update Policy dialog box. You can open the Update Policy dialog box in two ways in the VPC details view.

  • On the VPC details view, select the VPC you want to update and click the Update option in the top menu.
  • On the VPC details view, click the Edit option provided in the Actions menu for the selected VPC.
Note: You cannot update or delete the default policy.

The Update Policy dialog box has the same parameters as the Create Policy dialog box.

For details about the parameters that you can update in the Update Policy dialog box, see Creating a Policy.

Procedure

  • Update the parameters in the Update Policy dialog box.
  • Click Save .

Updating Static Routes

About this task

You can update a static route using the Update Static Routes dialog box. You can open the Update Static Routes dialog box either from the VPC list view or the VPC details view.

Note: You must configure the default route (0.0.0.0/0) to the external subnet as the next hop for connectivity outside the cluster (north-south connectivity).
  • On the VPC details view, select the VPC you want to update and click the Update option in the top menu.
  • On the VPC details view, click the Edit option provided in the Actions menu for the selected VPC.

The Update Static Routes dialog box has the same parameters as the Create Static Routes dialog box.

For details about the parameters that you can update in the Update Static Routes dialog box, see Creating Static Routes.

Procedure

  • Update the parameters in the Update Static Routes dialog box.
  • Click Save .

Deleting a Virtual Private Cloud

About this task

Prism Central does not allow you to delete a VPC if the VPC is associated with any subnets and/or VPNs. After you remove all the subnets or VPN associations from the VPC, delete the VPC.

You can delete a VPC from the VPC list view or the VPC details view.

Procedure

  • Do one of the following.
    • To delete a VPC from the VPC list view, select the VPC you want to delete and click Delete in the Actions drop down menu.
    • To delete a VPC from the VPC details view, click the VPC name to go to the VPC details view and click the Delete option in the More drop down menu.
  • In the confirmation dialog box, do the following.
    • Click Delete to delete the VPC.
    • Click Cancel to exit without deleting the VPC.

Deleting Subnets, Policies or Routes

About this task

You can delete VPC entities such as subnets, policies or routes from the VPC details page.

Note: You cannot update or delete the default policy.

Do the following.

Procedure

  1. Open the VPC details page and go to the respective tab like Subnets , Policies or Routes .
  2. Click the Delete option provided for the selected entity (subnet, policy or route respectively).
  3. In the confirmation dialog box, do the following.
    • Click Delete to delete the entity.
    • Click Cancel to exit without deleting the entity.

Connections Management

This section covers the management of network gateways, VPN connections and Subnet Extensions including operations like create, update and delete network gateways and VPN connections, and extending subnets.

Network Gateway Management

You can create, update or delete network gateways that host one of VPN or VTEP service for connections.

Creating a Network Gateway

About this task

VPN or s connect two networks together, and can be used in both VLAN and VPC networks on AHV. In other words, you can extend the routing domain of a VLAN network or that of a VPC using a VPN. Accordingly, VPN gateways can be configured using VLANs or VPCs. You need VPN gateways on clusters to provide a gateway to the traffic between on-premise clusters or remote sites.

You can create multiple VPN gateways for a VPC. Since a VPC is configured only on a PC, the VPC is available to all the clusters registered to that PC.

A VPN gateway may be defined as a Local gateway or a Remote gateway based on where the traffic needs to be routed.

To create a VPN gateway, do the following on the Networking & Security > Connectivity > Gateways page.

Procedure

  1. Select Local or Remote in the Create Gateway drop-down menu.
    If you select Local in the drop-down menu, the Create Local Gateway dialog box opens. If you select Remote in the drop-down menu, the Create Remote Gateway dialog box opens.

  2. Provide the necessary values in the respective fields as described in the table.
    For example, if you select Local in the drop-down menu, then the Create Local Gateway dialog box opens. Provide the necessary values in the respective fields as described in the table.
    Figure. Sample Create Local Gateway - VM Deployment Click to enlarge

    Figure. Sample Create Local Gateway - VPN Service Configuration Click to enlarge

    Figure. Sample Create Local Gateway - VTEP Service Configuration Click to enlarge

    Figure. Sample Create Remote Gateway - VPN Gateway Service Click to enlarge

    Figure. Sample Create Remote Gateway - VTEP Gateway Service Click to enlarge

Table 1. Local Gateway Fields
Fields Description Values
VM Deployment
Name Enter a name for the network gateway. (Name)
Gateway Attachments (for Local gateway type only) Select the gateway attachment as VPC or VLAN . The VPN VM is deployed on a VPC VM or a cluster that has the selected VLAN respectively.
  1. If you select VPC , then VPC Attachment is displayed. VPC is the default value for the Gateway Attachments field. The Gateway VM is deployed on the cluster and associated with the VPC selected in the VPC Attachment section.

    VPC attachment mode provides the options of eBGP and Static routing methods for external routing (configured in the External Routing Configuration section).

  2. If you select VLAN , then the VLAN Attachment is displayed. The Gateway VM is deployed on the cluster that has the VLAN and the subnet specified in the VLAN Attachment section.

    VLAN attachment mode provides only the eBGP routing method for external routing.

(VLAN or VPC)
Gateway VM Deployment - VPC Attachment
Cluster Select the cluster on which you want to deploy the Gateway VM on. (Name of the cluster)
VPC (If Gateway Attachment type is VPC) Select the VPC configured on the selected cluster that you want to use for the Gateway VM deployment. (Name of the VPC selected)
Floating IP (Optional)

Select a floating IP for the network gateway configuration. If you do not select a floating IP address then Prism Central allocates a floating IP automatically. This allocated floating IP is deleted when you delete the gateway.

To request floating IPs and allocate them to subnets, see Requesting Floating IPs

(IP address)
Gateway VM Deployment - VLAN Attachment
Cluster Select the Cluster, from the drop down list, on which you want to deploy the Gateway VM on.
Note: Only clusters with VLANs are available in the list.
(Name of the cluster)
Subnet Select the subnet you want to attach the Gateway VM to, from the drop down list.
Note: The list includes all the subnets you created on the selected cluster.
After you select the subnet, the details of the subnet are displayed in a box below the Subnet field. The details include: VLAN ID, IPAM type being Managed or Unmanaged, and Network Address with Prefix.
(Name of the VLAN subnet)
Static IP Address for VPN Gateway VM Enter the static IP address that the Gateway VM needs to use. (IP Address with Prefix)
Default Gateway IP Enter the default gateway IP of the subnet for the Gateway VM. (IP Address)
Service Configuration
Gateway Service Select the gateway service you want to use for the gateway. (VPN or VTEP)
VPN Service Configuration - External Routing Configuration (This section is available for VLAN and VPC attachment types)
Routing Protocol
  1. For VPC gateway attachments: Select Static for static routing.
    Note: You need to create static routes (see Creating Static Routes) for external routing and attach the route to the VPC selected in this configuration.
  2. Select eBGP for eBGP based external routing.
  3. For VLAN gateway attachments: External routing protocol is pre-set to eBGP . You cannot change the routing protocol.
(Static or eBGP)
Redistribute Connected Routes (Applicable only if VLAN type gateway attachment is selected) ( VLAN only) Select this checkbox to enable the redistribution of connected routes into the eBGP. (Check mark or blank)
ASN (Only available if eBGP routing protocol is selected)

(For eBGP only) Enter the ASN for your on-prem gateway. If you do not have a BGP environment in your on-prem site, you can choose any number. For example, you can choose a number in the 65000 range.

Note: Make sure that this ASN does not conflict with any of the other on-premises BGP ASNs.

ASN must be distinct in case of eBGP.

(Number)
eBGP Password (For eBGP in Local gateway type only) Enter the eBGP password for the eBGP route. (Password: The password must be between 1 and 80 characters.
  • Characters allowed for Pre-Shared Key for IPSec

    • a-z

    • A-Z

    • 0-9

    • ~ ! @ # % ^ & * ( ) _ - + = : ; { } [ ] | < > , . / ? $

    • Password length: Minimum 1 and maximum 64 characters.

  • Characters allowed for BGP passwords
    • a-z

    • A-Z

    • 0-9

    • ~ ! @ # % ^ & * ( ) _ - + = : ; { } [ ] | < > , . / ? $

    • Password length: Minimum 1 and maximum 80 characters.

)
VPN Service Configuration - Internal Routing Configuration (This section is available for VLAN attachment type only.)
Routing Protocol (Between On-prem Gateway and On-prem Router) Select the Routing Protocol to be used between on-premises Nutanix gateway and on-premises router.

You can select:

  • Static : Select this protocol to provide a static route configuration for the VLAN gateway.

  • OSPF : Select this protocol to provide an OSPF routing configuration for the VLAN gateway.

  • iBGP : Select this protocol to provide a iBGP route configuration for the VLAN gateway.
    Note: For iBGP, the ASN must be the same between the Gateway appliance and the peer iBGP, when iBGP is selected as the internal routing protocol.
(Static or OSPF or iBGP)
+Add Prefix (Applicable to Static routing)

(For Static routing selected in Routing Protocol ) Click this to enter a Local Prefix and click the check mark under Actions to add the prefix.

If you click the X mark under Actions , the local prefix you entered is not added.

The prefixes you add are advertised to all the connected peers via eBGP.

The prefix must be a valid IP address with the host bits not set.

You can add multiple local prefix IP addresses.

(prefix like /24)
Area ID (Applicable to OSPF protocol) (OSPF only) Enter the OSPF area id in the IPv4 address format.
Password Type (OSPF only) Select the password type you want to set for the OSPF route. The options are:
  1. MD5 : Select this option to encrypt the packets with MD5 hash that can be decrypted with the MD5 password at the destination.

  2. Plain Text : Select this option to set a clear-text password.

  3. None : Select this if you do to set an open route without password protection

Password

(OSPF only) Enter a password for the MD5 or Plain Text password type you select in the Password Type field.

  • For MD5 : The password must be 1-16 characters long.

    Characters allowed for OSPF passwords (MD5)

    • a-z

    • A-Z

    • 0-9

  • For Plain Text : The password must be 1-8 characters long.

    Characters allowed for OSPF passwords (Plain text): a-z.

Peer IP (for iBGP) Enter the IP Address of the On-prem router used to exchange routes with the network gateway. (IP Address)
Password Enter a password with 1-80 characters. (Password)
VTEP Service Configurations
VxLAN (UDP) Port The default value provided is 4789. Do not change this. (Number. Default value is 4789)
Table 2. Remote Gateway Fields
Fields Description Values
Name Enter a name for the network gateway. (Name)
Gateway Service Select the gateway service you want to use for the gateway. (VPN or VTEP)
VPN Service Configurations
Public IP Address Enter the public IP address of the remote endpoint. If a Floating IP is not selected, a new Floating IP is automatically allocated for the Gateway. These allocated IP addresses are deleted when the network gateway is deleted. (IP Address)
Vendor Select the vendor of the third party gateway appliance. (Name of Vendor)
External Routing
Protocol
  1. Select Static for static routing.
    Note: You need to create static routes (see Creating Static Routes) for external routing and attach the route to the VPC selected in this configuration.
  2. Select eBGP for eBGP based external routing.
(Static or eBGP)
eBGP ASN (Only available if eBGP routing protocol is selected)

(For eBGP only) Enter the ASN for your on-prem gateway. If you do not have a BGP environment in your on-prem site, you can choose any number. For example, you can choose a number in the 1-65000 range.

Note: Make sure that this ASN does not conflict with any of the other on-premises BGP ASNs.

ASN must be distinct in case of eBGP.

(Number)
VTEP Service Configurations
VTEP IP Address Enter VTEP IP Addresses of the remote endpoints that you want to create the gateway for. You can add IP addresses of multiple endpoints in one remote gateway. (Comma separated list of IP Addresses)
VxLAN (UDP) Port The default value provided is 4789. Do not change this. (Number. Default value is 4789)
  1. Click Save .

What to do next

The Gateway you create is displayed in the Gateways page.

Updating a Network Gateway

About this task

You can update a network gateway using the Update Gateway dialog box.

You can open the Update Gateway dialog box. The parameters in the Update Gateway dialog box are the same as those in the Create Local Gateway or Create Remote Gateway dialog box.

Procedure

  1. Select the gateway you want to update on Gateways .
  2. Click Update in the Actions menu.
  3. Update the required details in the Update Gateway dialog box.
    You cannot modify some information. Such fields are greyed and in-actionable. If you need to modify such information, consider creating a new gateway with the updated parameters and deleting the current gateway.
  4. Click Save .

Deleting a Network Gateway

About this task

If you want to delete a network gateway, you must first delete all the VPN connections associated with the gateway and only then you can delete the network gateway.

To delete a network gateway, do the following on the Gateway page.

Procedure

  1. Do one of the following.
    • Select the check box next to the name of the gateway and, in the Actions drop-down list, click Delete .
    • Click the name of the gateway and, in the details page, click Delete .
  2. In the confirmation dialog box, do the following.
    • Click Delete to delete the entity.
    • Click Cancel to exit without deleting the entity.

Virtual Network Connections

Virtual Private Network

You can use the Nutanix VPN solution to set up VPN between your on-prem clusters, which exist in distinct routing domains that are not directly connected. These distinct routing domains could either be VPCs within the same cluster or remote clusters or sites.

If you need to connect one Nutanix deployment in one site to another deployment in a different site, you can create a VPN endpoint in each of the sites. A VPN endpoint consists of a local VPN gateway, remote VPN gateway and VPN connection. Local VPN gateway can be instantiated in a VPC context or a legacy VLAN context. Launching the VPN gateway within a VPC allows stretching of the VPC. For example, in the figure, the Blue VPC is stretched between two sites with a VPN.

Figure. VPN Working Click to enlarge

VPN connections are useful in connecting two points. You can connect two VPCs in the same cluster using a VPN or VPCs in different clusters in the same site. However, VPN connection can connect only one endpoint to another endpoint. Flow networking based VPN service allows you to only connect two endpoints that use Nutanix VPN based gateway service.

Virtual Tunnel End Points Based Network Extensions

To connect one endpoint to multiple endpoints or third party (non Nutanix) networks, use Virtual Tunnel End Point (VTEP) service based subnet extensions. For more information about VTEP, see .

VPN Workflow

If you need to connect one Nutanix deployment in one site to another deployment in a different site, you can create a VPN endpoint in each of the sites. A VPN endpoint consists of a local VPN gateway, remote VPN gateway and VPN connection. You can configure multiple VPN endpoints for a site.

Each endpoint must have configurations for a local VPN gateway, remote VPN gateway (pointer information for the peer local VPN in the remote site endpoint) and a VPN connection (connecting the two endpoints). Then, based on the VPN connection configuration as initiator or acceptor, one endpoint initiates a tunnel and the endpoint at the other end accepts the tunnel connection and, thus, establishes the VPN tunnel.

  1. Gateways: Every VPN endpoint for each site consists of two VPN gateway configurations - Local and Remote.

    Local gateway is a VM that runs the VPN protocols (IKEv2, IPSec) and routing (BGP and OSPF). Remote gateway is a pointer - database entry - that provides information about the peer remote VPN endpoint. One of the key information contained in the remote gateway is the source IP of the remote VPN endpoint. For security reasons, the local VPN gateway will accept IKEv2 packets originating only from this Source IP.

    VPN gateways are of the following types:

    • On premises Nutanix VPN Gateway: Represents the VPN gateway appliance at your on-premises local or remote site if you are using the Nutanix VPN solution.

    • On premises Third Party Gateway: Represents the VPN gateway appliance at your on-prem site if you are using your own VPN solution (provided by a third-party vendor).

      To configure third party VPN Gateways, see the relevant third party documentation.

  2. VPN Connection: Represents the VPN IPSec tunnel established between local gateway and remote gateway. When you create a VPN connection, you need to select two gateways between which you want to create the VPN connection.

VPN appliances perform the following:

  1. Implementation of IKEv2 and IPSec protocols.
  2. Routing: Between remote sites, Flow Networking advertises prefixes using eBGP. Optionally it uses Static routing. Within a site, Flow Networking uses iBGP or OSPF to share prefixes between the Nutanix VPN appliance and the edge router.

Prerequisites for VPN Configurations

General Requirements

  • Ensure that you have enabled Flow Networking with microservices Infrastructure.

  • Ensure that you have floating IP addresses when you create VPN gateways.

    Flow Networking automatically allocates a floating IP to a VPN gateway if you do not provide one during the VPN gateway creation. To provide floating IP during the VPN gateway creation, you can request floating IPs. See Requesting Floating IPs.

  • Ensure that you have one of the following, depending on whether you are using iBGP or OSPF:

    • Peer IP (for iBGP): The IP address of the router to exchange routes with the VPN gateway VM.

    • Area ID (for OSPF): The OSPF area ID for the VPN gateway in the IP address format.

  • Ensure that you have the following details for the deployment of the VPN gateway VM:

    • Public IP address of the VPN Gateway Device: A public WAN IP address that you want the on-prem gateway to use to communicate with the Xi VPN gateway appliance.

    • Static IP Address: A static IP address that you want to allocate to the VPN gateway VM. Use a floating IP address requested as the static IP address.

    • IP Prefix Length: The subnet mask in CIDR format of the subnet on which you want to install the VPN gateway VM. You can use an overlay subnet used for a VPC and assigned to the VM that you are using for the VPN gateway.

    • Default Gateway IP: The gateway IP address for the on-premise VPN gateway appliance.

    • Gateway ASN: ASN must not be the same as any of your on-prem BGP ASNs. If you already have a BGP environment in your on-prem site, the customer gateway is the ASN for your organization. If you do not have a BGP environment in your on-prem site, you can choose any number. For example, you can choose a number in the 65000 range.

Ports and Protocols

Nutanix deploys a number of ports and protocols in its software. ports that must be open in the firewalls to enable Flow Networking to function. To see the ports and protocols used Flow Networking, see Port Reference.

Endpoints and Terminations

The following endpoints and terminations occur in the course of Flow networking based connections. For information about creating, updating or deleting VPN connections, see Connections Management.

Note: In a VPN connection do not configure both the gateways (local gateway and remote gateway) in an endpoint as Initiators or as Acceptors. If you configure the local gateway as Initiator then configure the remote gateway as Acceptor in one endpoint and vice-versa in the (other) remote endpoint.
VPN Endpoint Behind a Network Address Translation or Firewall Device

In this scenario, the IPSec tunnel terminates behind a network address translation (NAT) or firewall device. For NAT to work, open UDP ports 500 and 4500 in both directions.

Figure. VPN Endpoint Behind NAT or Firewall Click to enlarge

Things to do in NAT Things to do in on-prem VPN GW
Open UDP ports 500 and 4500 on both directions

Enable the business application policies to Allow the commonly-used business application ports.

IPSec Terminates on the Firewall Device

In this scenario, you do not need to open the ports for NAT (500 and 4500).

However, enable the on-prem VPN gateway to allow the traffic from the PC subnet to the advertised load balancer route where the Source port is any and the Destination port may be in the range of 1024-1034.

The PC subnet refers to the subnet where your Prism Central is running.

Figure. Tunnel Terminates on NAT or Firewall Click to enlarge

Creating a VPN Connection

About this task

Create a VPN connection to establish a VPN IPSec tunnel between VPN gateways in your on-prem site. Select the gateways between which you want to create the VPN connection.

To create a VPN connection, do the following on the Networking > VPN Connections page.

Procedure

  1. Click the Create VPN Connection button on the VPN Connections page.
  2. In the Create VPN Connection dialog box, provide the values in the respective fields.
Fields Description and Values
Name Enter a name for the connection.
VPN Connection
IPSec Secret Enter a secret password for the IPSec connection. To see the password, click Show . To hide the password, click Hide .
Local Gateway Select the connection parameters on the local gateway as Initiator or Acceptor of VPN Tunnel connections.
VPN Gateway Select the appropriate VPN Gateway as the local gateway for the VPN connection
VTI Prefix - Local Gateway Enter a IPv4 Address with /<prefix>. Example: 10.25.25.2/30.

This is the VPN Tunnel Interface IP address with prefix for the local gateway. The subnet for this IP address must be a /30 subnet with two usable IP addresses. One of the IP addresses is used for Local Gateway. Use the other IP address for the Remote Gateway.

Connection Handshake This defines the type of handshake that the connection must use. There are two types of connection handshakes:
  1. Initiator : The local VPN gateway acts as the initiator of the connection and thus initializes the VPN tunnel.
  2. Acceptor : The local VPN gateway accepts or rejects incoming connection requests from other gateways.
Note: In a VPN connection do not configure both the gateways (local gateway and remote gateway) in an endpoint as Initiators or as Acceptors. If you configure the local gateway as Initiator then configure the remote gateway as Acceptor in one endpoint and vice-versa in the (other) remote endpoint.
Remote Gateway For a specific VPN connection, set the remote gateway as Initiator or Acceptor when you configure the VPN connection on the Remote Gateway.
VPN Gateway Select the appropriate VPN Gateway as the remote gateway for the VPN connection.
VTI Prefix - Remote Gateway The VPN Tunnel Interface IP address with prefix for the local gateway. Provide a IPv4 Address with /<prefix>. Example: 10.25.25.2/30.

This is the VPN Tunnel Interface IP address with prefix for the local gateway. The subnet for this IP address must be a /30 subnet with two usable IP addresses. One of the IP addresses is used for Local Gateway. Use the other IP address for the Remote Gateway.

Advanced Settings Set the traffic route priority for the VPN connection. The route priority uses Dynamic route priority because the priority is dependent on the routing protocol configured in the VPN gateway.
Route Priority - Dynamic Route Priority Set the route priority as an integer number. The greater the number, higher is the priority.
  1. Click Save .

What to do next

The VPN connection you create is displayed in the VPN Connections page.

Updating VPN Connection

About this task

You can update a VPN Connection using the Update VPN Connection dialog box.

You can open the Update VPN Connection dialog box. The parameters in the Update VPN Connection dialog box are the same as those in the Create VPN Connection dialog box.

Procedure

  1. Select the VPN Connection you want to update on the VPN Connection .
  2. Click Update in the Actions menu.
  3. Update the required details in the Update VPN Connection dialog box.
  4. Click Save .

Deleting a VPN Connection

About this task

To delete a VPN connection, do the following on the VPN Connection page.

Procedure

  1. Do one of the following.
    • Select the check box next to the name of the VPN connection and, in the Actions drop-down list, click Delete .
    • Click the name of the VPN connection and, in the details page, click Delete .
  2. In the confirmation dialog box, do the following.
    • Click Delete to delete the entity.
    • Click Cancel to exit without deleting the entity.

VPN Connection within Same Prism Central

You can connect two VPCs within the same Prism Central availability zone using a VPN connection.

About this task

Assume that you have created two VPCs named vpc-a and vpc-b with overlay subnets named subnet-a and subnet-b .

To connect the two VPCs within the same Prism Central using a VPN connection, do the following.

Procedure

  1. Do the following for local gateways:
    1. Create a local VPN gateway with dynamically assigned address for vpc-a , for example, named local-vpn-a . Note or write down the assigned IP address.
    2. Create a local VPN gateway with dynamically assigned address for vpc-b , for example, named local-vpn-b . Note or write down the assigned IP address.

    See Creating a Network Gateway for more information about creating a VPN gateway.

  2. Do the following for remote gateways:
    1. Create a remote VPN gateway with the IP address noted in 1.a for vpc-a , for example, named remote-vpn-a .
    2. create a local VPN gateway with the IP address noted in 1.b for vpc-b , for example, named remote-vpn-b .

    See Creating a Network Gateway for more information about creating a VPN gateway.

  3. Create a VPN connection between vpc-a and vpc-b named, for example, vpn-conn-a-to-b .
    Ensure that the VTI IP addresses for the local and remote gateways is unique with /30 prefix.
    Note: The VPN Tunnel Interface IP address with prefix for the local gateway. The subnet for this IP address must be a /30 subnet with two usable IP addresses. One of the IP addresses is used for Local Gateway. Use the other IP address for the Remote Gateway.

    Ensure that you select local-vpn-a as the local gateway with Connection Handshake set as Acceptor .

    Ensure that you select remote-vpn-b as the remote gateway.

  4. Create a VPN connection between vpc-b and vpc-a named, for example, vpn-conn-b-to-a .
    Ensure that the VTI IP addresses with /30 prefix for local and remote gateways are the reverse (vice versa) of what you configured for the VPN connection in previous step. For example, if in previous step you configured the VTI IP addresses as 10.20.20.5/30 for local and 10.20.20.6/30 for remote then for VPN connection in this step, configure 10.20.20.6/30 for local gateway and 10.20.20.5/30 for remote gateway respectively. These IP addresses do not need to be reachable anywhere else in the network. However, ensure that these IP addresses do not overlap with any other IP addresses assigned in the network.

    Ensure that you select local-vpn-b as the local gateway with Connection Handshake set as Initiator .

    Ensure that you select remote-vpn-a as the remote gateway.

Layer 2 Virtual Network Extension

You can extend a subnet between on-prem local and remote clusters or sites (Availability Zones or AZs) to support seamless application migration between these clusters or sites.

Note: One or more on-prem cluster or sites managed by one Prism Central instance is defined as an Availability Zone or AZ. In this section, Availability Zone or AZ refers to and must be understood as one or more on-prem clusters or sites managed by one Prism Central. Local AZ refers to local on-prem clusters or sites managed by a Prism Central instance and remote AZ refers to another on-prem cluster or site managed by another Prism Central instance.

With Layer 2 subnet extension, you can migrate a set of applications to the remote AZ while retaining their network bindings such as IP address, MAC address, and default gateway. Since the subnet extension mechanism allows VMs to communicate over the same broadcast domain, it eliminates the need to re-architect the network topology, which could otherwise result in downtime.

Layer 2 extension assumes that there are underlying existing layer 3 connectivity already available between the Availability Zones. You can extend a subnet from a remote AZ to the primary (Local) AZ (and other remote AZs in case of VTEP-based subnet extensions)

  • You can extend a Layer 2 subnet across two Nutanix AZs over either VPN or Virtual tunnel End Point (VTEP). SeeLayer 2 Virtual Subnet Extension Over VPN.
  • You can extend a Layer 2 subnet between a Nutanix AZ and one or more non-Nutanix datacenters only over VTEP. See Layer 2 Virtual Subnet Extension Over VTEP.

You can extend subnets for the following configurations.

  • IPAM Type. Managed and unmanaged networks.
  • Subnet Type. On-prem VLAN subnets and VPC subnets.
  • Traffic Type. IPv4 unicast traffic and ARP.
  • On-prem Hypervisor. AHV and ESXi
    Note: If your cluster is ESXi, use vCenter Server to manually configure the port group attached to the subnet you want to extend. Set the security settings, Promiscuous mode and Forged transmits to Accept on the vSwitch as shown in the following image.
    Figure. ESXi Host Port Group Configuration Click to enlarge ESXi port group settings

Prerequisites for Setting Up Subnet Extension

Ensure the following before you configure Layer 2 subnet extension between your on-prem AZs.

  • Ensure that the Prism Central versions support Layer 2 virtual subnet extension as specified in the Release Notes. See AOS Family Release Notes and Release Notes | Prism Central as applicable.

    See the Prism Central Upgrade and Installation Guidelines and Requirements section of the Acropolis Upgrade Guide for instructions about how to upgrade a Prism Central instance through the Prism Central web console.

  • Ensure that you pair the Prism Central at the local AZ with the Prism Central at the remote AZ to use Create Subnet Extension wizard to extend a subnet across the AZs and facilitate bidirectional communication between these clusters or sites. Using paired availability zones it is possible to configure both VXLAN over VPN and VTEP based subnet extension. You can also extend subnets using the manual gateway and connection workflows instead of pairing the AZs.

    See the Pairing Availability Zones for instructions about how to pair the local and remote AZs.

  • Ensure that you set up a default static route with 0.0.0.0/0 prefix and the External Network next hop for the VPC you use for any subnet extension. This allows NTP and DNS access for the Network Gateway appliance.

Best Practices for Subnet Extension

Nutanix recommends the following configurations to allow IP address retention for VMs on extended subnets.

  • When using Nutanix IPAM ensure the address ranges in the paired subnets are unique to avoid conflict between VM IP addresses across extended subnets.
  • If the source and target sites use third-party IPAM, ensure that there are no conflicting IP address assignments across the two sites.
    Note: If the source and target sites use Nutanix IPAM, the Prism Central web console displays a message that indicates an IP address conflict if one exists.
  • If connectivity between sites already provides encryption, consider using VTEP only subnet extension to reduce encryption overhead.
  • Use the Subnet Extension to a Third Party Data-Center workflow in the following scenarios
    • To extend a subnet to more than one other AZ. This is also known as point to multi-point.
    • To extend subnets between clusters managed by the same Prism Central.

Subnet Extension Workflow

You can manage Layer 2 subnet extension on the Subnet Extensions tab of the Connectivity page. Open the Subnet Extensions by clicking the hamburger icon in the top-left corner of the Dashboard and then clicking Connectivity .

  • You can create point-to-point Layer 2 subnet extensions between two AZs over VPN or VTEP by opening the Create Subnet Extension Across Availability Zones dialog box. See Extending a Subnet Over VPN for VPN-based extensions. See Extending a Subnet Across Availability Zones Over VTEP for VTEP-based extensions.

  • You can create point-to-point or point-to-multipoint Layer 2 subnet extensions to third party datacenters over VTEP by opening the Create Subnet Extension To A Third Party Data-Center dialog box. See Extending a Subnet to Third Party Datacenters Over VTEP.

  • You can update a subnet extension that extends across AZs using the Update Subnet Extension Across Availability Zones dialog box. The Update Subnet Extension Across Availability Zones has the same parameters and fields as the Create Subnet Extension Across Availability Zones dialog box. You can open the Update Subnet Extension Across Availability Zones dialog box by:

    • Selecting the subnet extended across AZs in the Subnet Extensions and clicking the Update button.

    • Clicking the subnet extended across AZs in the Subnet Extensions and clicking the Update button on the Summary tab.

You can update a subnet extension that extends to multiple AZs or third party datacenters using the Update Subnet Extension To A Third Party Data-Center dialog box. Update Subnet Extension To A Third Party Data-Center dialog box has the same parameters and fields as the Create Subnet Extension To A Third Party Data-Center dialog box. You can open the Update Subnet Extension To A Third Party Data-Center dialog box by:

  • Selecting the subnet extended to third datacenters in the Subnet Extensions and clicking the Update button.

  • Clicking the subnet extended to third datacenters in the Subnet Extensions and clicking the Update button on the Summary tab.

See Updating an Extended Subnet.

Layer 2 Virtual Subnet Extension Over VPN

Subnet extension using VPN allows seamless, secure migration to a new datacenter or for disaster recovery. VPN based Layer 2 extension provides secure point to point connection to migrate workloads between Availability Zones. Consider VTEP-only Subnet Extension without VPN when encryption is not required.

Subnet extension using VPN is useful:

  • When the two Availability Zones (where the subnets to be extended belong) do not have any underlying secure connectivity. For example, when connecting over the Internet, VPN (IPSec) provides the necessary connectivity and encryption (security).
  • Sometimes when you need to move (lift-and-shift) workloads from a VLAN subnet to a VPC subnet retaining the same VM IP addresses . You need connectivity from other subnets to workloads that have already migrated to VPC. In such cases, VPN provides the Layer 3 connectivity and encryption between the VPC segment of extended subnet to other VLAN subnets.

Prerequisites for Setting Up Subnet Extension Over VPN

  • See Layer 2 Virtual Network Extension for general prerequisites to extend subnets.

  • Set up VPN gateway services and a VPN connection between local AZ and the remote AZ. The subnet extension feature supports only the Nutanix VPN solution (not a third-party VPN solution) at the both the local and remote AZs. See the Virtual Network Connections for instructions about how to upgrade the VPN gateway VM at the local and remote clusters or sites.
    Note: Ensure that the VPN gateway version is 5.0 or higher. See the Updating a Network Gateway section of the Nutanix Flow Networking Guide for instructions about how to upgrade the network gateway at the local and remote sites.
  • Configure subnets with the same IP CIDR prefix at the source and target sites. For example, if the IP prefix at one site is 30.0.0.0/24, the IP prefix at the other site must also be 30.0.0.0/24. The network and mask must match at both AZs.
  • Configure distinct DHCP pools for the source and target sites with no IP address overlap. Separate DHCP pools ensure no IP address conflicts occur for dynamically assigned IP addresses between the two AZs.
  • Procure two free IP addresses, one from each subnet, for the Network Gateway in the subnets to be extended. These IP addresses are configured as local IP address and remote IP address for the subnet extension in the Subnet Extension wizard. These two free IP addresses are the externally accessible IP addresses for the local gateway, and the remote gateway. Those two usable IP addresses are already contained inside the VPN connection and must not conflict with the following:
    • DHCP pools on any of the Availability Zones.
    • Gateway IP address on any of the Availability Zones.
    • IP addresses allocated to existing user VMs on any of the Availability Zones.
    • IP addresses used by Network Gateway Management NIC subnet (IP pool 100.64.1.0/24)

Limitation

To use subnet extension over a VPN, both sites must use the VPN service of the Nutanix Network Gateway. Consider VTEP-only subnet extension to connect to non-Nutanix third party sites.

Pairing AZs (Nutanix Disaster Recovery)

To replicate entities (protection policies, recovery plans, and recovery points) to different on-prem AZs (AZs) bidirectionally, pair the AZs with each other. To replicate entities to different Nutanix clusters at the same AZ bidirectionally, you need not pair the AZs because the primary and the recovery Nutanix clusters are registered to the same AZ (Prism Central). Without pairing the AZs, you cannot perform DR to a different AZ.

About this task

To pair an on-prem AZ with another on-prem AZ, perform the following procedure at both the AZs.

Procedure

  1. Log on to the Prism Central web console.
  2. Click the hamburger icon at the top-left corner of the window. Go to Administration > AZs in the left pane.
    Figure. Pairing AZ
    Click to enlarge Pairing AZ

  3. Click Connect to AZ .
    Specify the following information in the Connect to Availability Zone window.
    Figure. Connect to AZ
    Click to enlarge Connect to AZ

    1. AZ Type : Select Physical Location from the drop-down list.
      A physical location is an on-prem AZ (AZ). To pair the on-prem AZ with Xi Cloud Services, select XI from the drop-down list, and enter the credentials of your Xi Cloud Services account in step c and set d.
    2. IP Address for Remote PC : Enter the IP address of the recovery AZ Prism Central.
    3. Username : Enter the username of your recovery AZ Prism Central.
    4. Password : Enter the password of your recovery AZ Prism Central.
  4. Click Connect .

Extending a Subnet Over VPN

The subnet extension allows VMs to communicate over the same broadcast domain to a remote site or Availability Zone (AZ).

Before you begin

See Layer 2 Virtual Network Extension and Layer 2 Virtual Subnet Extension Over VPN for information on prerequisites and best practices for extending a subnet.

About this task

Perform the following procedure to extend a subnet from the on-prem site.

Procedure

  1. Click the hamburger icon in the top-left corner of the Dashboard > Networking & Security > Connectivity > Subnet Extension .
  2. On the Subnet Extension page, select Create Subnet Extension > Across Availability Zones .
  3. In the Create Subnet Extension Across Availability Zones dialog box, enter the necessary details as described in the table.
    Figure. Create Subnet Extension Across Availability Zones Click to enlarge Create Subnet Extension Across Availability Zones using VPN service

Fields Description Values
Extend Subnet over a Select the gateway service you want to use for the subnet extension. (VPN or VTEP)
Note: Configure the following fields for the Local and the Remote sides of the dialog box.
Availability Zone (For Local) Local AZ is pre-selected default.

(For Remote) Select the appropriate AZ from the drop-down list of AZs.

(Local: Local AZ)

(Remote: Dropdown list of AZs.)

Subnet Type Select the type of subnet that you want to extend. (VLAN or Overlay)
Cluster Displayed if your selected VLAN subnet. Select the cluster from the dropdown list of clusters. (Name of cluster selected from dropdown list)
VPC Displayed if your selected Overlay subnet. Select the appropriate VPC from the dropdown list of VPCs. (Name of VPC selected from dropdown list)
Subnet Select the subnet that needs to be extended. (Name of subnet selected from dropdown list)
(Network Information frame) Displays the details of the VLAN or Overlay network that you selected in the preceding fields. (Network information)
Gateway IP Address/Prefix Displays the gateway IP address for the subnet. This field is already populated based on the subnet selected. (IP Address)
(Local or Remote) IP Address Enter a unique and available IP address that are externally accessible IP addresses in Local IP Address and Remote IP Address . (IP Address)
VPN Connection Select the appropriate VPN Connection from the dropdown list that Flow networking must use for the subnet extension. See Creating a VPN Connection for instructions to create VPN connection. (Name of VPN connection selected from the dropdown list)
  1. Click Save .

    A successful subnet extension is listed on the Subnet Extension dashboard. See .

Layer 2 Virtual Subnet Extension Over VTEP

Subnet extension using Virtual tunnel End Point (VTEP) allows seamless migration to new datacenters or for disaster recovery. VTEP based Layer 2 extension provides point-to-multipoint connections to migrate workloads from one Availability Zone to multiple Availability Zones without encryption. If you need security and encryption, consider using Subnet Extension over VPN.

Subnet extension using VTEP is useful:

  • When both subnets that need to be stretched are Nutanix subnets (managed or unmanaged). VTEP provides an optimized workflow to stretch the two subnets.
  • When both subnets are connected over an existing private and secure link that does not need additional encryption.
  • When one Nutanix subnet needs to be stretched across one or more non-Nutanix networks, sites, or datacenters. Subnet Extension with third-party VTEPs provides point-to-multipoint connectivity to third party datacenters assuming that there is underlying layer 3 connectivity between these VTEPs.

VTEP-based Layer 2 Subnet Extension provides the following advantages:

  • Layer 2 subnet extension from one AZ to multiple AZs.
  • Layer 2 subnet extension between Nutanix AZs and non-Nutanix third party VTEP-based AZs.
  • The Remote VTEP Gateway is a set of endpoint IP addresses. You can add endpoint IP addresses to an existing operational Remote VTEP Gateway without stopping the subnet extension services. This on-the-fly addition enables you to extend the subnets to more AZs than originally planned, or perform maintenance, without disrupting the running services or configuring new remote VTEP gateways.

Prerequisite for Setting Up Subnet Extension Over VTEP

  • See Layer 2 Virtual Network Extension for general prerequisites to extend subnets.

  • Set up VTEP local and remote gateway services on local and remote AZs. In case of point-to-multipoint extension, ensure that you create local and remote VTEP gateways on all the remote AZs that the subnet needs to be extended to.

  • For each extended subnet within the same Network Gateway appliance ensure that you have unique VxLAN Network Identifiers (VNIs) that you can use for the VTEP subnet extensions. VNI may be any number between 0 and 16777215.

Extending a Subnet Across Availability Zones Over VTEP

The subnet extension over VTEP allows VMs to communicate two Availability Zones (AZ) without a VPN connection.

Before you begin

See Layer 2 Virtual Network Extension and Layer 2 Virtual Subnet Extension Over VTEP for information on prerequisites and best practices for extending a subnet.

About this task

To extend a subnet over VTEP across two availability zones (AZs), do the following.

Procedure

  1. Open the Create Subnet Extension Across Availability Zones in one of the following ways:
    • On the Subnet Extensions tab, click > Create Subnet Extension > Across Availability Zones > .

    • In the Subnets dashboard, select the subnet you want to extend and click Actions > Extend > Across Availability Zones

    • In the Subnets dashboard, click the subnet you want to extend. On the subnet details page, click Extend > Across Availability Zones .

    Figure. Example of Create VTEP Extension Across AZs with VLAN Subnet Click to enlarge Displaying example of Create Subnet Extension Across Availability Zones for VLAN Subnet over VTEP

  2. For Extend Subnet over a , select VTEP .
  3. Enter or select the necessary values for the parameters in the Local and Remote (AZ) sections as described in the table.
Parameters Description and Value
Availability Zone Displays the name of the paired availability zone at the local AZ.
Subnet Type Select the type of the subnet - VLAN or Overlay that you are extending.
Cluster Select the name of the cluster in the local AZ that the subnet is configured for.
Subnet Select the name of the subnet at the local AZ for network. The VLAN ID and the IPAM - managed or unmanaged are displayed in the box below the Subnet field.
Gateway IP Address. Enter the gateway IP address of the subnet you want to extend. Ensure that you provide the IP address in <IP-address/network-prefix> format. for example the gateway IP is 10.20.20.1 in a /24 subnet then provide the gateway IP address as 10.20.20.1/24 .
Note: For an unmanaged network, enter the gateway IP address of the created subnet.
Local IP Address Enter a unique and available (unused) IP address from the subnet provided in Subnet for the Network Gateway appliance.
Remote IP Address Enter a unique and available (unused) IP address from the subnet provided in Subnet for the remote Network Gateway appliance.
Local VTEP Gateway