Cisco 6880X & 6800ia Part 3

We continue to have problems with these boxes, here is the latest 'Bus Error":

This was in production so we took about a 15 minute downtime, not pretty. Still working with Cisco TAC to find the root cause.

For Palo Alto Networks Cult members :)

https://www.nsslabs.com/blog/seriously

Floor is open for comments :)

Cisco 6880X & 6800ia Part 2

Our deployment of 6880’s and 6800ia’s is in a large healthcare system, which in hindsight was not the best move. We received the 6880’s with 15.1(2)SY1 code which had some serious issues, the biggest being SDP error messages which was causing 6800 extenders to flap constantly. Upon contacting Cisco TAC, I was told to upgrade the code to 15.1(2)SY2 (TAC engineer knew about it beforehand but there was no official documentation available….hmmm :)

Upgrading to SY2 did fix the flapping issue but caused a few other problems. Most important was that the extenders were getting stuck during the code upgrade (while pulling the new code from the parent switch). After a month long correspondence with TAC and BU engineers, we were told that “most” of these issues have been addressed in SY3 release of the code so we should upgrade to SY3. Keep in mind that every time you upgrade the 6880, the attached fex’s have to pull down the new code all over again and in the case of VSS, the fex’s reboot twice before during the process (when using eFSU). This may not be a big deal for a small setup but it is a huge problem in a 24x7 hospital environment. This problem is compounded by the fact that fex’s take 6-9 minutes to be fully operational after the reload. To make a long story short, even after the upgrade to SY3 we are still having major issues (including VSL link failure).

Here is a list of outstanding issues for 6880x (August 31, 2014):

• Random VSL Link Failure.
•  “ENTROPY_FAILURE: Unable to collect sufficient entropy”
•  SSH stops working if the active switch goes into recovery mode. The only “fix” we have found so far is to reload both shelves.
• 7-9 minutes boot time for the fex’s.
•  In case of a single homed fex, uplink interface shutdown/failure or a simple twinax/fiber failure will reboot the fex (per cisco, it’s a “security” feature but it’s an issue for me).
• ISSU/eFSU doesn’t provide much visibility into the upgrade process, leaving you wondering if it is stuck in the process (per cisco SY3 will show more “messages” during the upgrade process, I haven’t seen anything new so far).

We have received 12 of these boxes and have had 3 DOA linecards (C-6880-X-16P10G) so far. Keep in mind that you can’t interchange C-6880-X-LE-16P10G & C-6880-X-16P10G linecards as the LE is for Lite Edition (smaller hardware table) and will absolutely NOT work on the X (bigger hardware table) chassis. Installing/inserting these linecards into the chassis is tricky as well. If you don’t have it aligned exactly at the proper angle, it will get stuck and you will not be able to yank it out without messing up something else…..poorly designed linecard to say the least.

If you are not bothered by any of this stuff then you are a good candidate for deploying 6880’s J

Cisco 6880X & 6800ia Part 1

If you are thinking about deploying 6880X with some 6800ia’s then you may want to read this post in its entirety before making the final decision. I will be updating either this post or adding new posts as I come across new/relevant information.

I’ve long been a Cisco fan and one could have sold me a brick with a cisco logo on it (up until a couple of years ago) and I would have been very happy with that purchase. I would still have the same love for Cisco had I not touched the Cisco ACS (4.x & 5.x), Cisco Prime LMS 4.1 in the last few years, and the new 6880X with 6800 instant access switches. For now, let’s forget about the ACS/LMS and discuss 6880/6800ia. Being that routing and switching is one of Cisco’s core competencies; one would expect a very stable, reliable, and a feature rich (new features) product (like the 6500, Nexus 7K, ASR etc.). Unfortunately 6880X doesn’t enjoy any of those traits for now.

There is no doubt that the idea is good, you take the same model as nexus 7K/5K with fex’s and make it available outside of the datacenter environment but the execution of this plan has been subpar. With that being said, let us start with some of the “good” stuff about these boxes:

    1.      Extremely competitive price (compared to the 6500/6807 with Sup 2t).
    2.      PoE availability on the 6800 instant access switches.
    3.      Great 10g port density for the price.
    4.      Feature rich (L2/L3, full MPLS, GRE in hardware).

Please note that my assumption of “good” is heavily based on the pricing.

6880X datasheet:

http://www.cisco.com/c/en/us/products/collateral/switches/catalyst-6880-x-switch/data_sheet_c78-728228.html

And now some of the “not so good” list:

1.  Max instant access switch/fex ports restricted to 1008. This means that you can only deploy 21 6800ia’s switches/fex’s per VSS pair. Per Cisco, this number will be increased to perhaps 2000 ports or more by the end of the year.
2. You can only stack up to (3) 6800ia switches.
3. You can only use FEX id’s 1-12 for now. So if you have a deployment where you need 15 single 6800ia switches….well you can only deploy 12 for now J 

Here is a complete list of restrictions and the things you can’t do on the 6800ia’s:

http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst6500/ios/15-1SY/config_guide/sup2T/15_1_sy_swcg_2T/instant_access.pdf

In the next post, I will share our 6880/6800ia deployment and the ongoing struggles with these boxes.