Category Archives: Networking

Blogs about networking, mainly using cisco technologies.

Cisco ASR 9000 series Router

What is the Cisco ASR 9000 ?

  • It is a carrier -class Ethernet Access and Aggregation platform with Layer 3 intelligence. 
  • Aggregates 1 gig access to 10 Gig edge
  • Combines full Ethernet service flexibility with MPLS scalability
  • Multicast performance to support video networking

asr 9000


2 main types of cards are used here.. Route switched processor (RSP) cards , and Line cards.

RSP cards provides switch fabric redundancy and chassis control.

Line cards provide Gigabit Ethernet and 10 Gibabit Ethernet Ports.We can have 3 types line cards, 4 port 10G, 8 port 10G and 40 port GE. This can change as cisco adding new cards all the time.

To mention, each line card can operate on 40 Gig . So 8 port 10 gig will be over-subscribed.

In old days, CEs were interconnected by way of a Frame Relay or Asynchronous Transfer Mode(ATM) or leased-line circuits. This used to be build once and sold once. Each circuit was dedicated to each customer.

The new way is IP/MPLS VPN Cloud.

MPLS combines the privacy and QoS of FR or ATM networks with the flexibility and scalability of IP.These are Any-to-Any connectivity, which allows us to build once , sell many times.

Cisco ASR 9000 are used as a PE(provider edge) devices.



How to configure OTV – Overlay Transport Virtualization – Nexus 7K

OTV  or Overlay Transport Virtualization extends layer 2 VLANs between data centers. This allows us to maintain our layer 3 hierarchy and enable us to move the virtual machines cross data centers.

Think about it, if you are able to move your VMs from one location to another and not having to change any IP addresses or routing, how cool will it be ?

OTV to function ,we can either use Multicast or Unicast . I will talk about enabling OTV using Multicast here..

Here’s the steps to configure OTV on a Multicast Core : 

  1. Enable PIM sparse-mode and IGMP version 3 on all core devices.
  2. Configure the OTV external interface for IGMP version 3 on the edge device.
  3. Enable the OTV feature and designate a site VLAN
  4. Configure the OTV overlay interface on edge device.

1. Enabling PIM Sparse Mode on Nexus 7K

Enabling PIM also has 4 steps..

  • Enable the protocol independent multicast (PIM)
  • Configure the rendezvous-point (RP)
  • Enable PIM sparse mode on all core interfaces
  • Enable IGMP version 3 on all core interfaces.

feature pim


ip pim rp-address  a.b.c.d


interface eth1/31

ip pim sparse-mode

ip igmp version 3

2. Configure the external Interface

interface eth1/41

ip igmp version 3

//  note we ware not enabling PIM sparse mode on the external interface

3. Designate a Site VLAN

OTV uses the site VLAN to send hello messages to determine the authoritative edge devices . Ofcourse we need to enable OTV feature first in NX-OS

feature otv

otv site-vlan 13

Do not extend the site VLAN accross the overlay network, because it is only used locally.

4. Configuring the OTV Overlay Interface

The overlay interface is used to encapsulate Layer 2 frames into ip packets. To configure the overlay interface specify the following:

  • Control group : Multicast group used for the OTV control plane
  • Data  group : range of multicast SSM groups to use for multicast traffic
  • Join interface: used to advertise MAC address reachability info across overlay
  • Entend VLAN : VLANs to extend across the overlay network

interface overlay 1
otv control-group
otv data-group
otv join-interface ethernet1/41
otv extend-vlan 2000 ====> this is the vlan we want to extend
no shutdown


Verifying the Overlay Interface:

use the command  show otv overlay <interface>

VPN state should be UP.

Do a show otv adjacency to verify one or more adjacency are present.

Do a show otv site to verify one or more site local adjacencies are present.

Do a show otv vlan to see the extended vlans .

Do a show mac address-table <vlan> will show the port and mac learned via OTV interface.



Fabric Extender

What is Nexus 2000 Fabric Extender and how to configure them

Hello everyone ! Did not get a chance to write anything for few weeks .

I would like to talk about Nexus 2000 Fabric Extender and it’s relationship with Nexus 5000 and how to configure them.

Nexus 2000 connects directly to Nexus 5010 or 5020 using upto 4 10G ethernet uplink ports. All configuration are done on nexus 5K.  2000 series switch acts as a remote I/O module or virtual line card of nexus 5K.

12 to 20 Cisco Nexus 2000s can be attached to each Cisco Nexus 5000 .

Cisco Nexus 2000 – Deployment Benefit:

  • Physically resides on the top of each rack but logically acts like an end-of-row access device. 
  • Having this on top of the rack, it reduces the cable runs to the servers.
  • Nexus 2000 are managed from Nexus 5000 switch, so it reduces the management points.
  • Ensures features consistency across hundreds of servers.
  • Nexus 2000 basically makes Nexus 5000 to become a high density 1G access layer switch.
  • Investment protection
  • Virtual network link capabilities.


Cisco Nexus 2148T is the First product of Nexus 2000 series.

  • 1 rack-unit fixed-config form factor- 48 GE server-facing (downlink) ports, and 4×10 GE network-facing (uplink) ports.
  • No local switching, no spanning-tree.
  • Server-to-server traffic goes to Cisco Nexus 5000 and back.
  • Nexus 2000 switch has Front-to-back airflow(same as servers) – This helps Hot-aisle,cold-aisle Datacenter Design.
  • Redundant hot-swappable power supplies
  • Removable fan tray with redundant fans
  • Over subscription : 1.2 : 1 to 4.8:1




Cisco Nexus 2000 Deployment Model..

2k deployment model

This kind of model provides a straight forward switching and cabling strategy that simplifies network operations, and prepares for future networking technologies.

Configuring the Cisco Nexus 2000 Fabric Extender: 

To configure the Cisco Nexus 2148,  the cisco Nexus 5000 requires Cisco NX-OS Software Release 4.0(1a)N2(1).

All configuration are done on the Nexus 5K.

First Step is to create the Fabric Extender instance..


Tulip-Tech-N5K-Access1(config)# fex 110
Tulip-Tech-N5K-Access1(config-fex)# description FEX0110
Tulip-Tech-N5K-Access1(config-fex)# pinning max-links 4
Chage in Max-links will cause traffic disruption.

Tulip-Tech-N5K-Access1(config-fex)# exit

Pinning max-links command   binds 48 downlinks to  uplinks, here all 48 ports can use all 4 uplinks. 

Second step is to Associate the Fabric Extender to a Nexus 5000 ethernet interface(static pinning)

Tulip-Tech-N5K-Access1(config)# interface ethernet 1/7
Tulip-Tech-N5K-Access1(config-if)# switchport mode fex-fabric
Tulip-Tech-N5K-Access1(config-if)#no shut
Tulip-Tech-N5K-Access1(config-if)#fex associate 110


We can also associate the Fabric Extender to a Cisco Nexus 5020 port channel (load balancing). Most of the time we will be doing this kind of configs.. here’s how a full Fex Config looks like on a Port-channel..

Tulip-Tech-N5K-Access1(config)# interface ethernet 1/29
Tulip-Tech-N5K-Access1(config-if)# switchport mode fex-fabric
Tulip-Tech-N5K-Access1(config-if)#no shut
Tulip-Tech-N5K-Access1(config-if)#fex associate 110
Tulip-Tech-N5K-Access1(config-if)#channel-group 110
Tulip-Tech-N5K-Access1(config)# interface ethernet 1/30
Tulip-Tech-N5K-Access1(config-if)# switchport mode fex-fabric
Tulip-Tech-N5K-Access1(config-if)#no shut
Tulip-Tech-N5K-Access1(config-if)#fex associate 110
Tulip-Tech-N5K-Access1(config-if)#channel-group 110
Tulip-Tech-N5K-Access1(config)# interface port-channel110
Tulip-Tech-N5K-Access1(config-if)# switchport mode fex-fabric
Tulip-Tech-N5K-Access1(config-if)#no shut
Tulip-Tech-N5K-Access1(config-if)#fex associate 110

How to verify Fabric Extenders or FEX…

Tulip-Tech-N5K-Access1# sh int port-channel 110 fex-intf  

Fabric FEX
Interface Interfaces
Po110 Eth110/1/48 Eth110/1/47 Eth110/1/46 Eth110/1/45
Eth110/1/44 Eth110/1/43 Eth110/1/42 Eth110/1/41
Eth110/1/40 Eth110/1/39 Eth110/1/38 Eth110/1/37
Eth110/1/36 Eth110/1/35 Eth110/1/34 Eth110/1/33
Eth110/1/32 Eth110/1/31 Eth110/1/30 Eth110/1/29
Eth110/1/28 Eth110/1/27 Eth110/1/26 Eth110/1/25
Eth110/1/24 Eth110/1/23 Eth110/1/22 Eth110/1/21
Eth110/1/20 Eth110/1/19 Eth110/1/17 Eth110/1/16
Eth110/1/13 Eth110/1/9 Eth110/1/14 Eth110/1/15
Eth110/1/10 Eth110/1/12 Eth110/1/11 Eth110/1/18
Eth110/1/7 Eth110/1/8 Eth110/1/6 Eth110/1/5
Eth110/1/3 Eth110/1/4 Eth110/1/1 Eth110/1/2


Tulip-Tech-N5K-Access1# sh fex detail

FEX: 110 Description: Tulip-Tech-N5K-Access1-fex110 state: Online

FEX version: 5.2(1)N1(1) [Switch version: 5.2(1)N1(1)]
FEX Interim version: 5.2(1)N1(1)
Switch Interim version: 5.2(1)N1(1)
Extender Serial: SSI16XXXXX
Extender Model: N2K-C2248TP-1GE, Part No: 73-13232-01
Card Id: 99, Mac Addr: 64:d8:14:ZZ:XX:YY, Num Macs: 64
Module Sw Gen: 12594 [Switch Sw Gen: 21]
post level: complete
pinning-mode: static Max-links: 1
Fabric port for control traffic: Eth1/29
FCoE Admin: false
FCoE Oper: true
FCoE FEX AA Configured: false
Fabric interface state:
Po110 – Interface Up. State: Active
Eth1/29 – Interface Up. State: Active
Eth1/30 – Interface Up. State: Active
Fex Port State Fabric Port
Eth110/1/1 Up Po110
Eth110/1/2 Up Po110
Eth110/1/3 Up Po110
Eth110/1/4 Down Po110
Eth110/1/5 Down Po110
Eth110/1/6 Down Po110
Eth110/1/7 Down Po110
Eth110/1/8 Up Po110
Eth110/1/9 Up Po110
Eth110/1/10 Up Po110
Eth110/1/11 Down Po110
Eth110/1/12 Down Po110
Eth110/1/13 Down Po110
Eth110/1/14 Down Po110
Eth110/1/15 Up Po110
Eth110/1/16 Up Po110
Eth110/1/17 Up Po110
Eth110/1/18 Down Po110
Eth110/1/19 Down Po110
Eth110/1/20 Down Po110
Eth110/1/21 Down Po110
Eth110/1/22 Up Po110
Eth110/1/23 Up Po110
Eth110/1/24 Up Po110
Eth110/1/25 Down Po110
Eth110/1/26 Down Po110
Eth110/1/27 Down Po110
Eth110/1/28 Down Po110
Eth110/1/29 Down Po110
Eth110/1/30 Down Po110
Eth110/1/31 Down Po110
Eth110/1/32 Down Po110
Eth110/1/33 Down Po110
Eth110/1/34 Down Po110
Eth110/1/35 Down Po110
Eth110/1/36 Down Po110
Eth110/1/37 Down Po110
Eth110/1/38 Down Po110
Eth110/1/39 Down Po110
Eth110/1/40 Down Po110
Eth110/1/41 Down Po110
Eth110/1/42 Down Po110
Eth110/1/43 Down Po110
Eth110/1/44 Down Po110
Eth110/1/45 Down Po110
Eth110/1/46 Down Po110
Eth110/1/47 Down Po110
Eth110/1/48 Up Po110
05/26/201304:24:06.19842: Module register received
05/26/201304:24:06.21062: Registration response sent
05/26/201304:24:06.199827: Module Online Sequence
05/26/201304:24:14.829097: Module Online

Tulip-Tech-N5K-Access1# sh int fex-fabric

Fabric Fabric Fex FEX
Fex Port Port State Uplink Model Serial
110 Eth1/29 Active 1 N2K-C2248TP-1GE SSI111111JX
110 Eth1/30 Active 2 N2K-C2248TP-1GE SSI222222JX

I hope this helps..

I have received few emails asking which book covers the most of these topics.

I have enjoyed reading and certainly can recommend
NX-OS and Cisco Nexus Switching: Next-Generation Data Center Architectures (2nd Edition) (Networking Technology) By Ron Fuller.

What is vPC-Virtual Port-Channel and how to configure them – Nexus 5K,7K

We all know what a Port-Channel is right ? A port-channel is nothing but bundling a bunch of ports together, so that is seems like one big pipe. Why do we do that ? For redundancy,for higher throughput.

A server can have 4 NICs connecting to a switch. If one of them goes down, in a port-channel the other one takes over without any delay. This can be true for connecting two switches. That’s why we see lots and lots of port channels running between core switches and core and dist or access switches.

This also allows us to have a loop free layer 2 network. Well.. to get a loop free layer 2 network, we also have to run Spanning Tree. Spanning-Tree protocol will block the redundant link , and will only open it if the first port goes down. It’s fine for a 100 meg port or 1 gig port.

What if you have a network, specially in a datacenter where you have 10 Gig ports ? And two 10gig ports forming  a Port-Channel ? One of them will be blocked !! Cisco recognizes this as a waste of Bandwidth.

So Cisco came up with vPC, it’s a port-channel, but it’s virtual. On a traditional port-channel, all the ports needed to be connected to the same device to form a port-channel, with vPC, you don’t ! You can connect one port to one core, and second one to second core. And use vPC, your switch will see both of those connection as One Uplink. Since there are no multiple paths, there are no Loops ,  and Spanning tree will not block anything !

imporving layer2 with vpc


vPC is supported on both Nexus 5000 and 7000 models.

Don’t think vPC can only be between switches. We can also deploy vPC between Servers and Switches too. If a server had 4 NICs and running ESXi, we can configure 2 NICs to go to one switch and other two to go to other switch. Incase one of the switch fails, we will be still up and running.

Now let’s take a look at what’s in a vPC ? Or what do we need to have a vPC.

  1. vPC Peers – a pair of vPC enabled swtiches.Two nexus 7K or 5Ks. 
  2. vPC peer Link – this link will carry the vPC control traffic between peers.
  3. vPC Peer Keepalive link- We need a layer 3 , routed interface for both peer to talk.
  4. CFS – Cisco Fabric Service protocol , Nexus 5k and 7K support this.
  5. vPC – the actual vPC port, which will be usually on the Nexus end.
  6. vPC Member Port – this is one of the ports in one of the switches which will be part of the vPC
  7. vPC Domain – it’s a number, all switches has to be part of same domain

vpc arch


Configuring vPC: 

  1. Configure the vPC domain
  2. Configure/Establish the vPC Peer keepalive link
  3. Configure the vPC peer-link
  4. Configure the vPCs
  5. Optimize vPC

1.Configure the vPC domain

Tulip-Tech-N7K-Core1# conf t
Tulip-Tech-N7K-Core1(config)#feature vpc
Tulip-Tech-N7K-Core1(config)# vpc domain 2

Tulip-Tech-N7K-Core1# sh vpc role

vPC Role status
vPC role : primary
Dual Active Detection Status : 0
vPC system-mac : 00:23:04:ee:be:02

See here .. the last part of the mac address of the vPC system-mac co-relates to domain number we used. If we used vpc domain 10 , last portion will be 0a.


 2.Configure/Establish the vPC Peer keepalive link

Tulip-Tech-N7K-Core1(config-vpc-domain)# peer-keepalive destination source

Tulip-Tech-N7K-Core2(config-vpc-domain)# peer-keepalive destination source

Note, I did not mention a vrf here, in this case management vrf will be used. We can also specify a specific vrf for this keepalive.

Tulip-Tech-N7K-Core1(config-vpc-domain)# peer-keepalive destination source vrf VPC-KEEPALIVE

Rule of thumb: Do not use a cross over cable to connect the management port. Use a dedicated management switch or OOB switch .

3.Configure the vPC peer-link

The peer-link is the link connecting two Nexus switches. Refer to my previous diagram. Here are the rules..

  • peer-link should be a port-channel
  • It should be configured as trunk
  • Should have atleast 2 10gig ethernet ports in the port-channel.

Tulip-Tech-N7K-Core1(config)#interface port-channel1
Tulip-Tech-N7K-Core1(config-if)# description VPC Peer Link
Tulip-Tech-N7K-Core1(config-if)# switchport
Tulip-Tech-N7K-Core1(config-if)# switchport mode trunk
Tulip-Tech-N7K-Core1(config-if)# spanning-tree port type network
Tulip-Tech-N7K-Core1(config-if)# vpc peer-link

And here’s the interfaces I configured to be part of Port-Channel 1 . See I have used one port on module 8 and another port on module 18. So in case we have one of the module fail, we will still have 10G connectivity.
Tulip-Tech-N7K-Core1#sh run interface Ethernet8/32
description Port Channel to Tulip-Tech-N7K-Core2
switchport mode trunk
channel-group 1 mode active
no shutdown

Tulip-Tech-N7K-Core1# sh run int e18/32
interface Ethernet18/32
description Port Channel to Tulip-Tech-N7K-Core2
switchport mode trunk
channel-group 1 mode active
no shutdown


4.Configure the vPCs:

I used to get confused between vPCs and vpc-peer links. Remember, vpc peer-link is between your main nexus switches,  and vpcs will be connecting whatever device (another nexus 5k, or non nexus switch, or even servers) . And to have vPCs we need Ether Channels.

In our example Etherchannel 1 was configured as our vpc peer-link. And now we need to configure a VPC to connect say another swtich. We will use VPC 101 . We will also need to have an Etherchannel to have this VPC 101 up . So we will create a Port-Channel 101.

One of more physical interfaces has to be part of that port channel. I am going to use only 1 interface for Port-channel 101. This will be configured on both Tulip-Tech-N7K-Core1 and Tulip-Tech-N7K-Core2.
Tulip-Tech-N7K-Core1# sh run int e1/1
description Tulip-Tech-N5K1
switchport mode trunk
channel-group 101 mode active
no shutdown

Tulip-Tech-N7K-Core2# sh run int e1/1
description Tulip-Tech-N5K1
switchport mode trunk
channel-group 101 mode active
no shutdown

And now configure vpc on the Port-channel 101 on both both core switches..

Tulip-Tech-N7K-Core1(config-if)# interface port-channel 101
Tulip-Tech-N7K-Core1(config-if)# description description VPC to Tulip-Tech-N5K1
Tulip-Tech-N7K-Core1(config-if)# switchport mode trunk
Tulip-Tech-N7K-Core1(config-if)# vpc 101

Tulip-Tech-N7K-Core2(config-if)# interface port-channel 101
Tulip-Tech-N7K-Core2(config-if)# description description VPC to Tulip-Tech-N5K1
Tulip-Tech-N7K-Core2(config-if)# switchport mode trunk
Tulip-Tech-N7K-Core2(config-if)# vpc 101


5. Optimize vPC

We can use peer-gateway to optimize the traffic flow via vPCs.
Tulip-Tech-N7K-Core1(config)# vpc domain 2


The peer-gateway feature allows a vPC switch to act as the active gateway for packets addressed to the peer router MAC. This keeps forwarding of traffic local to the vPC node and avoids using the peer-link.

peer-switch allows a pair of vPC devices to appear as one root bridge in the STP by using the same bridge-ID , this make the STP topology simpler, and make the convergence time faster in times of failure.

Couple of commands to verify vPC will be very handy..

show running config vpc

show vpc

show vpc consistency-parameters global

show vpc peer-keepalive

Here’s a vPC troubleshooting command chart which might come handy..

vpc troubleshoot



That’s all folks.. I know this is a very long post and sometime hard to get our head around. But knowing this will really make a difference and a must for the next generation switching technology.

Adios !

I have received few emails asking which book covers the most of these topics.

I have enjoyed reading and certainly can recommend
NX-OS and Cisco Nexus Switching: Next-Generation Data Center Architectures (2nd Edition) (Networking Technology) By Ron Fuller.


How to do initial device setup – Cisco Nexus 7000

Cisco NX-OS made of two main pieces of software image.

  • Kickstart Image – low level kernel shell with CLI available for managing the device before the system image is initialized
  • System Image- the operating system software that runs the system.

Initializing a Cisco Nexus device first time: 

Upon first time bootup the system will autometically run a system setup script that prompts for:

  • Secure password enforcement (y/n) – Required
  • Admin Password – Required
  • Additional login accounts (y/n)
  • SNMP read and write community strings
  • Switch name
  • Enable license grace period(y/n)
  • Out-of-band (mgmt0) configuration
  • Enable telnet(y/n) or SSH(y/n)
  • Configure NTP
  • ….etc

We can do first two steps then move to command line to finish the rest of the configs.

Once in the CLI mode ,  we should perform few initial startup checks..

Tulip-Tech-N7K-Core1# sh version

====== output omitted ====
BIOS: version 3.22.0
kickstart: version 5.2(4)
system: version 5.2(4)
BIOS compile time: 02/20/10
kickstart image file is: bootflash:///n7000-s1-kickstart.5.2.4.bin ===> Software version
kickstart compile time: 12/25/2020 12:00:00 [03/08/2012 03:58:13]
system image file is: bootflash:///n7000-s1-dk9.5.2.4.bin =============> Software version
system compile time: 1/28/2012 23:00:00 [03/08/2012 04:59:07]
cisco Nexus7000 C7018 (18 Slot) Chassis (“Supervisor module-1X”)
Intel(R) Xeon(R) CPU with 8260944 kB of memory.
Processor Board ID JXXXXXXX

Device name: Tulip-Tech-N7K-Core1
bootflash: 2048256 kB
slot0: 0 kB (expansion flash)

Kernel uptime is 0 day(s), 16 hour(s), 20 minute(s), 1 second(s) ====> Uptime

====== output omitted ====
Tulip-Tech-N7K-Core1# sh running-config

!Command: show running-config
!Time: Wed Apr 17 17:10:47 2013
version 5.2(4)
username admin password 5 $1$ObDfvvZUpZz54B6rawM6CbG. role network-admin
Tulip-Tech-N7K-Core1# sh startup-config

!Command: show startup-config
!Time: Wed Apr 17 17:14:26 2013
!Startup config saved at: Fri Apr 12 18:39:47 2013
version 5.2(4)

Display the currently installed licenses…
Tulip-Tech-N7K-Core1# show license usage
Feature  Ins  Lic  Status  Expiry  Date  Comments
MPLS_PKG Yes – In use Never -
STORAGE-ENT No – Unused -
FCOE-N7K-F132XP No 0 Unused -

Tulip-Tech-N7K-Core1# install licenses bootflash:license_file.lic

You can go to config mode and change the hostname also can change username and password

Configure remote management via Telnet or SSH on Nexus 7K

Nexus 7000 telnet ssh setup


Minimum configuration Best Practices :

Nexus 7000 minimum best practice


The last line will make all the make all the interfaces shut down.

Displaying Important System Files:

The most important file system on a Nexus switch is  bootflash: 

Bootflash: stores all the images , to see the content of bootflash we need to do   dir bootflash: command.
Tulip-Tech-N7K-Core1# dir bootflash:
161980383 Nov 04 06:39:27 2011 n7000-s1-dk9.5.2.1.bin
162744737 Mar 22 16:14:02 2012 n7000-s1-dk9.5.2.4.bin
29471232 Nov 04 06:38:16 2011 n7000-s1-kickstart.5.2.1.bin
29312000 Mar 22 16:05:24 2012 n7000-s1-kickstart.5.2.4.bin
4096 Nov 04 07:20:32 2011 vdc_2/
4096 Nov 04 07:20:32 2011 vdc_3/
4096 Nov 04 07:20:32 2011 vdc_4/

Usage for bootflash://sup-local
511672320 bytes used
1341444096 bytes free
1853116416 bytes total


To delete any file from bootflash: we need to do the following

Tulip-Tech-N7K-Core1# delete bootflash: [file name ]

Restarting the System, Nexus 7K

Nexus 7000 reload



Remember, while you reload any module, it will not pass any traffic ;)  .

Displaying installed System Components: 

We can use show module in nexus 7Ks to see what modules are installed..

Nexus 7000 show module

To monitor system environment, do a show environment command.


Nexus 7000 show module Nexus 7000 show environ


Displaying the System Locator LEDs: 

Locator LEDs can be turned on and off for various components from command line. This helps to locate the right module if you are due to replace them . specially if the filed tech is doing the replacement.

Nexus 7000 locator led


Monitoring Interfaces: 

Show interface brief  , shows all the interfaces , type, mode, status, speed , etc..

Show ip interface brief , shows the layer 3 interfaces, or interfaces with IP addresses.


To verify that traffic is passing on an interface, use the show interface command.


Obtaining Tech-Support details: 

If you need to get a tech-support from nexus 7k , Good Luck !! . The file is huge ! The best option is the save the tech-support to the local flash, and the use FTP or TFTP to get it out.

Tulip-Tech-N7K-Core11# sh tech-support  >> bootflahs: Tech-support.txt 

This file can be around 144 Meg. Nexus has Gzip installed. So we can use it to Zip the file.

Tulip-Tech-N7K-Core11# gzip bootflash:Tech-support.txt 

Even better, save the file in flash as a zip.. the following command will save the show-tech as a zip file on to the flash. This will save you some valuable time.

Tulip-Tech-N7K-Core11# tac-pac bootflash://show-tech


Hope this helps.. very basic steps to bring up and running a Nexus 7000 switch.


Troubleshooting Process Nexus 7000

Troubleshooting Nexus 7000 series is not that different then troubleshooting any other datacenter switches. However following a logical step will help us navigate and isolate the issue. Here I will talk about the steps , not the detailed troubleshooting of specific issues.

We can actually break down the issues Nexus switches can have in 4 areas..

  1. Physical Port issues
  2. Physical switch issues
  3. Layer 2 issues
  4. Layer 3 issues

Once we identify the key area of the problem we can then drill down in depth to find where exactly the issue might be coming from .

Have a look at the below chart




Troubleshooting Upgrades and Reboots: 

Time to time we might have to do upgrades or reboots for network maintenance. We have to take extra care while doing so on a Nexus 7000 series switch. We should try to minimize the network disruption while performing the upgrades and reboots specially in production environment. We must know how to recover quickly if something goes wrong.

Cisco NX-OS consist of two images. Kickstart image and the system image. These two images should be same version to bring up the system.

We should follow the following check-lists while preparing for upgrades..

troubleshooting-chart 2

Troubleshooting Licensing:

Let’s look at the guidelines for cisco NX-OS licensing..

  1. Do not ignore the grace period.  Allow 60 days before the expiring date to order, ship and install the new licenses. 
  2. Carefully determine what features we should use and what licenses do we need for those features.
  3. Order your license accurately , serial number of the chassis is very important. It must be the same on the  PAK  key. Use show  license host-id  to obtain the chassis serial number.
  4. Always archive /back up the license .
  5. Don’t attempt to use it on another device.

Troubleshooting VDCs on Nexus switches:

  • Verify that you are logged into the device as admin if you are creating or modifying VDCs
  • Verify that you are in correct VDC. You must be in the default VDC to configure VDCs
  • Verify that you have installed the Advanced Services License to configure VDCs.
  • Verify that you are not attempting to create more than three non-default VDCs.


Troubleshooting Ports:

  • Check the physical media to ensure there are no damaged parts.
  • Verify the SFP
  • Verify that the port is no shutdown.
  • Do show interface command from CLI to see the state of the port
  • Verify that  you have configured port as a dedicated port, and not connected to any other 3 ports in the port group.

To troubleshoot port channels and trunking issue , we should check the following issues first..

  • show port-channel compatibility-parameters  command to to determine port-channel requirements. This output shows what ports in what condition can be part of a port-channel.
  • Ensure all interface on the port channel have same destination device.
  • Vefify that either side of the port channel is connected to the same number of interfaces
  • Verify that each interface is connected to same type of interface on the other side
  • Verify that all required VLANs on a trunk port are in the allowed VLAN list.

Troubleshooting VLANs: 

  • Verify the physical connectivity for any problem ports or VLANs
  • Verify that you have both end devices in the same VLAN
  • Verify that any private VLAN configuration and associations are correct.

Troubleshooting Spanning Tree Protocol on Nexus 7000: 

Spanning tree protocol works at layer 2 level, to provide a loop free network. Layer 2 LAN ports send and receive STP frames at regular intervals. Network do not forward these frames , but use the frames to construct a loop free path.

Here’s the checklist to troubleshoot STP issues

  • Verify the type of spanning tree configured on all ports in your LAN
  • Verify the network topology including all interconnected ports and switches.
  • Verify the primary and secondary root bridge and any configured Cisco STP extensions.

Troubleshooting Routing : 

Begin troubleshooting routing issues by checking the following issues first..

  • Verify that the routing protocol is enabled
  • Verify that the address family is configured if necessary
  • Verify that you have configured the correct VRF for your routing protocol.


I know this is not the whole picture of a troubleshooting, but some logical steps. In each steps we can have so many different scenarios. Hopefully this guideline will help us towards our troubleshooting.


What is VDC-Virtual Device Context , and how to Configure them ? Cisco Nexus 7K

Nexus 7000 series introduced Virtual Device Context or VDC. VDC enables users to divide one Cisco Nexus 7000 switch into 4 different switches, where each switch operates as a stand alone switch. Each unit has dedicated processor, ports, and protocols.

This allows consolidating multiple physical devices into one , still having 4 different switches.

Usually on standard data center environment we will have multiple VLANs and multiple VRFs segregating different types of data. However problem on the switches control plane can cause all those to fail simultaneously.

So following same technology of Hypervisor multiple logical switches can be configured on a single physical switch. Ports can be reallocated between VDCs in a flexible manner.

Using VDCs, we can configure 2 nexus switches to participate in both Core and Aggregation layer instead of having to buy more hardwares.

VDC1 is the default VDC, and has a special role. VDC1 can create other VDCs, allocate resources, all non default VDCs are strictly separated.

VDC Rules of Engagement:

  1. The Cisco NX-OS advanced Services License is required to create, delete or modify VDCs. A grace period exists(120 days), but once it’s over any VDC configuration will be deleted. 
  2. VDCs are created from the Default VDC global config mode. Network-admin role is needed to create, delete or modify VDCs.
  3. Physical and logical resources (ports, memory) are allocated to VDCs from Default VDC. Once done those ports can be configured only from that VDC.

 Configuring VDC on Nexus 7000 switches:

Creating VDC: enter global configuration mode, and just type VDC Name, here we are creating a VDC named RED.

creating and deleting vdc


Now that the VDC is created, lets allocate some Interfaces to it. Remember VDC Red is now a stand alone switch. and switches need interfaces right ?

Allocating interfaces:

allocating interface


To Verify Interface allocations in VDC do the following command..

show vdc membership

It will show all the VDCs and ports allocated to them. Almost like show vlan command. This has to be done from Default VDC or VDC1. Otherwise it will show only that VDC information, not all.

we can issue  show vdc details to see more details on each vdc.

When a VDC is created, default template is allocated. Which can be shown with following command..

show vdc [vdc Name] resource

Configuring Resource assignment:

resource allocation


We can also create resource templates to streamline resource allocations.

Navigating to VDCs:

switchto VDC [vdc name] will take us to the vdc we want to be in.

switchback command , will bring us back to default .

To save the running config of all vdc we should do

copy run start vdc-all

Configuring high availability:

high availability


VDC Configuration best practices:

best practice


I hope this is enough to understand and configure the VDCs on a nexus 7000 switch. I will talk about troubleshooting VDC issues if when I get a chance.

Have a good day!

Cisco Nexus Product Overview

This blog is to help to identify new Cisco Nexus Product family. Specifically the Cisco Nexus 7000 switch chassis and components. I will also talk about Cisco Nexus 5000 switch, and the Cisco Nexus 2000 Fabric Extender and Nexus 1000V .

Lets start at the bottom of the tree.

Nexus 1000V:  In this new world of virtualization  we need to be able to connect to many virtual machines on the network without compromising the throughput and performance on the network layer. Nexus 1000V is a Virtual Switch connects these virtual machines. It’s easily integrates with Vmware environment. Nexus 1000V runs same NX-OS operating system same as other Nexus switches. Though it’s virtual switch, it still provides the rich feature sets in a virtual environment.

Once it’s installed and operational, the control of the switch is back in the hand of the network administrator, instead of relying upon server admin . Network administrator can configure and manage this switch same way as he/She would do a normal switch.

Nexus 1000V


So, how does this forward traffic ?

Once installed, it replaces the VMware’s virtual switch component. The data plane is called VEM or Virtual Ethernet Module. VEM installs directly to ESXi Hypervisor. For control plane and management plane functionality it uses something called VSM or Virtual Supervisor Module. This VSM can be hosted on ESXi as a Virtual Machine or on an appliance called Nexus 1010.

VSM does not get involved in data transfer or connectivity, that is VEM’s job.

Getting complicated ? Just think about a Modular switch 6500. When we do show module , we see the supervisor module and all other modules installed on them. Similarly here when we do show module on VSM, we see Virtual Supervisor Modules and Virtual Ethernet Module. We can have multiple Virtual Supervisor Modules to be as Hot Standby for failover scenario. Failing of one Supervisor does not affect the VEM or Data Transfer through VEM.

Think about this, Cisco 1000 V is an access switch connecting the servers..(just on a virtual environment )

Cisco Nexus 5000 Series : 

Nexus 2000

This is cisco’s first offering of high density, low latency 10 Gig/sec Ethernet switch. This are Layer 2 switches.

It has Redundant Hot swapable Power supplies, Redundant hot swapable fans. Cooling is front to back . This is good to deploy in a hot isle/cold isle data center. N5K supports both FCoE and Native Fiber channel. Which is good if you have older fiber channel environment  but would also like to have FCoE functionality in today’s environment.

There are two Models in N5K family.. Nexus 5010 and Nexus 5020 .

Nexus 5020 can support upto  56 Ports. Out of these 40 ports are fixed. And it has two expansion modules. You can install modules according to your need.

Nexus 5010 has exactly half the ports. It has 20 fixed ports with 1 expansion bay.

We can also use Nexus 2000 series Fabric extender with N5K switches. Fabric extender works as a extended Module of the Nexus 5K or Nexus 7K switches.

There are two new models introduced recently in Nexus 5000 series. They are Nexus 5548(with 36 10G ports and 1 expansion module) and Nexus 5596(with 48 10G ports and 3 expansion Module)

So based on the requirement and port density we can choose either of these switches.

Nexus 5500 series adds exciting features which earlier Nexus 5010 and 5020 did not have. 5548 and 5596 series have Layer 3 routing capabilities where 5010 and 5020 were Layer 2 only switches. This also support Cisco Fabric Path technology which was only available on Nexus 7K before.

Here’s the chart showing the improvements of Nexus 5500 over nexus 5000 series switches. Take note of the Throughput, VLAN config  and Port to Port Latency between the models..

Nexus 5000 chart


Cisco Nexus 7000 Series:  

You can call it the next big thing, the Evolution . This was designed ground up to support modern day networks, with high throughput, scalable, modular switching technology. Without a doubt it was built for datacenters.

Nexus 7000 has 3 models..

Nexus 7009, 7010 and 7018 .

Have a look at the below Chart to see the difference between them..

Nexus 7000 chart-1


In depth look at 7009 Chassis..

The modules are placed horizontally, so the airflow must be side to side. Cable management are built in both sides to offer cable run from both sides. The front door can be locked to prevent any accidental cable movement. 7009 isn’t as deep as 7010 or 7018.

Nexus 7009 chassis


Next have a look at Nexus 7010 Chassis..

This was built for power and cooling optimization and resiliency. Every component in Nexus 7010 is redundant. I/O modules are mounted vertically, this allows  Front to Back air flow. It has cable management option at the top of the rack with is really cool. It also has the option to lock the doors at the front to prevent accidental cable disruption.

Nexus 8010 chassis


Finally.. the big boy.. Nexus 7018 Chassis..

It’s an 18 slot chassis, including 2 supervisor module. It leaves us with 16 slots for I/O modules. It also has Side to Side Airflow and integrated cable management. Again , everything are redundant. From Fabric modules to Fans to Power supply.

The Fans rotations increases or decreases according to need, which saves electricity.  Here’s  how it looks ..

Nexus 7018 chassis



Woooh.. thats all about the chassis.. however we aren’t done yet !!. We need to know about the Line modules goes into these chassis right ? You can’t just buy the chassis not have any line modules ! That’d be like buying a box of chocolate without the chocolates in it !.. not a good example but you get the point..

Nexus 7000 Line modules..

First up .. Supervisor engine..

Nexus 7000 Supervisor engine(Sup 1)..

This is a true supervisor engine which controls management plane and control plane only. No switching is performed on the Supervisor engine. This is nothing but a management engine which is always on ,  N7K to be fully redundant , we need to have 2 supervisor engines.

So..what’s on this supervisor engine or how does it look.. here it is..

N7K supervisor engine


Next up is the Crossbar Fabric Modules..

Fabric modules are the component which provided high throughput to the I/O slots, ie slots that connects the cables. Those IO slots connects to the Fabric Modules on the backplane , which ensures the High Throughput.

Remember those 7010 and 7018 switches ?  at the back it had 5 slots for Fabric modules. These are the modules I am talking about here.

cross bar fabric module



So far we have seen both sup module and fabric module, now lets look at the I/O modules, which I call Port modules. Just so that I remember, these are the modules with the Ports on them and takes the data.

First up is 8 port 10Gig module. It’s called M1 module. It gives us 80 gig full duplex fiber connectivity. If we have other switches to connect or up-links, you can use this type of modules.

m1 IO module


Second is 32-port 10Gig Module. It is also M1 module. It gives us higher port density for 10 Gig connectivity with Over subscription. Here’s how it looks..

m1 IO module 32 port


Next up is Nexus 7000 48 port 1Gig I/O module..

It has 48 1 gig ports with either SPF or RJ45 . It looks like same as other Modules we have seen on 6500 switches.


Last but not Lease.. Nexus 7000 F1 series I/O module.. lol F1. I call it  Formula One module. This module is used for High performance low latency throughput. A mixture of SFP and RJ45 ports is used. It also can be dual speed, ie 1Gig or 10 Gig .

I call this Formula One module

I call this Formula One module

F1 modules do not support Layer 3 functionality !! We will need to use M1 cards for Layer 3 !

That’s all about the Modules ..incase you are forgetting.. we have Nexus 2000 models , which are used as Fabric extenders. Now think about it, you might have 1000s of servers in a datacenter which needs to be connected to the network. We aren’t going to install N7Ks everywhere, we will install 1 at the end of the Rac, and use Nexus 2K on top of each rack to connect the servers. This 2Ks are nothing but an external Modules to the 7Ks. That’s the beauty !..

And this is how it will look..

Nexus 2000 in a diagram


That’s all for today !!! Hopefully in future I will talk about the configuration and troubleshooting of Cisco Nexus devices.

Adios !