Main Nav

We are about to crank up the first Cisco Nexus in our environment and thought it’d be wise to see if others out there that are already running Nexus gear had any insight, gotchas, etc to share.  Seems like there are always those things you run into and think “Dang, I wish I would have known about XYZ ahead of time” so I figured this would be a good place to ask the question.  This box is a Nexus 7k and we’ll have several 2k’s hanging off of it for connecting servers and SANs into.

 

Thanks,

Robbie

 

Robbie Woodley
Sr. Network Analyst
The University of Texas at Tyler
(903) 566-7125

 

 

********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/.

Comments

We’ve had Nexus 7000’s in production for a while now, but don’t use the 2K’s.  The issues we’ve had are mostly related to certain line cards.

 

The 32-port M1 module has had some reliability issues.  We’ve had a few hardware failures.  They also have performance limitations where 4 ports are serviced by a single ASIC.  We had a situation where high volume research traffic on one interface exceeded the capacity of the ASIC affecting production traffic on an adjacent interface.

 

At one point we considered the 48-port 1/10 Gbps F2 module, but it has a MAC table limitation of 16K entries.  We felt that with a large flat network, we might exceed the capacity in the future, so we don’t use them.

 

ISSU is a great feature, but watch out for spanning-tree instabilities.  We’ve had upgrades abort because of a spanning-tree topology changes.

 

Other than that, the 7000 has been a good product.

 

Ron

 

Message from duvalld@lanecc.edu

We were early adopters and have been running Nexus for 3+ years. When we jumped in, the 2Ks were 1G (access ports) only and it forced us to hang 10/100 switches in the racks so we could connect ILOMs and management ports that are still 100Mb. Cisco came out with 10/100/1000 2Ks just after we bought. Make sure your 2Ks do 100Mb and save the management and clutter of extraneous switches. Dale Dale DuVall Server Admin Information Technology Lane Community College 4000 E 30th Ave Eugene, OR 97405 541-463-3341 duvalld@lanecc.edu cell 541-359-4374 >>> Ronald Martin Bunal 2/3/2014 7:39 AM >>> We've had Nexus 7000's in production for a while now, but don't use the 2K's. The issues we've had are mostly related to certain line cards. The 32-port M1 module has had some reliability issues. We've had a few hardware failures. They also have performance limitations where 4 ports are serviced by a single ASIC. We had a situation where high volume research traffic on one interface exceeded the capacity of the ASIC affecting production traffic on an adjacent interface. At one point we considered the 48-port 1/10 Gbps F2 module, but it has a MAC table limitation of 16K entries. We felt that with a large flat network, we might exceed the capacity in the future, so we don't use them. ISSU is a great feature, but watch out for spanning-tree instabilities. We've had upgrades abort because of a spanning-tree topology changes. Other than that, the 7000 has been a good product. Ron
We implemented the nexus with the earlier 2K cards.  It had slipped by us that the 2Ks will not support a switch being up linked to it.  IIRC, as soon as a 2K port saw a bpdu, it shut down the port.  Not sure if this behavior has changed over the years.  You probably are aware of this but the 2Ks do not switch, they punt all packets up to the 7K in this case again if I recall correctly with a proprietary header. 

There is something has tripped up a few of us (I know because I have mentioned it at conferences and seen the affirming nods) is the default Control-Plane policing.  The nexus ships with a very strong copp access-list.  This is fine, but can cause very weird behaviour when there is a problem: the one that happened to us was intermittent connectivity.  Things here and there would work for while and then not and then they would, etc.  Connectivity up and down, up and down.  Even pinging from the Vlan Interface on one Nexus to the corresponding  Vlan Interface on it's hsrp partner was intermittent.  In our particular case, it was a switch that had been near a  lightening strike and was kinda fried and was arping a LOT, ddossing us.  The copp policy caused most arps (both from the corrupted switch and legitimate arps) to be dropped, except once in a while a legitimate one would not get dropped and things off that segment would work till the arp cache timed out.  So, if things are simply not making sense, that COPP is one place to look.

Good Luck!
dennis



Dennis Bohn
Manager of Network and Systems
Adelphi University
bohn@adelphi.edu
5168773327


Message from mark.duling@biola.edu

Same here. At the time we purchased, the 2k were model 2148 that didn't do 100M. We know that but we didn't know that it also didn't do etherchannel. They replaced those with the 2248 that added 100 MB operation (and maybe 10 also) and etherchannel. But for those buying new gear that wouldn't be an issue now.
We have been using the 2K for a number of years now-- first with the 5Ks and now with the 7Ks. There are a number of features I would like Cisco to add to them. Some will be added soon. Some are waiting for more people to request them. 1. CDP when connected to 7K 2. vpc+ of FEX when connected to 7K 3. Full PVLAN functionalility (This may have been addressed. I have not tried to do PVLAN on them in a few years.) 4. Switchport monitor destination To be honest, while I like the concept of the 2K, I do not think Cisco is putting enough resources into the platform. If they are line card extensions, then FEX should have the same functionalities as a line card (or at least a cheap desktop switch). Regards, -- Christina Christina Klam Network Engineer Institute for Advanced Study Email: cklam@ias.edu Einstein Drive Telephone: 609-734-8154 Princeton, NJ 08540 Fax: 609-951-4418 ********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/.