Friday, September 20, 2019

Cisco 9504 %SYSMGR-2-CFGWRITE_ABORTED: Configuration copy aborted.


%SYSMGR-2-CFGWRITE_ABORTED: Configuration copy aborted.
%SYSMGR-3-CFGWRITE_FAILED: Configuration copy failed (error-id 0x401E0000).
%SYSMGR-2-CFGWRITE_ABORTED_CONFELEMENT_RETRIES: Copy R S failed as config-failure retries are ongoing. Type "show nxapi retries" for checking the ongoing retries.
%SYSMGR-3-CFGWRITE_SRVFAILED: Service "confelem" failed to store its configuration (error-id 0x00000079).

Ran into this issue a couple of weeks ago while working on replacing some old 7K's with the newer 9500's running NXOS 7.0.3.I7.6. We had a few hundred lines of config and decided to copy/paste, everything went fine -- or so we thought until we tried saving the config.

Issue was related to missing/incomplete config. Neither I nor my coworker noticed any error messages when pasting the config but when you try to save the config, you are greeted with the error messages. Issue was resolved by the tac engineer after he noticed that we were missing the hsrp config under the svi's (only had the command "hsrp version 2" but missing the hsrp group and the ip). It would have been nice if the error message was a little more descriptive or at least offered a hint towards the actual issue. 
"show nxapi retries" didn't return any info either.

Tuesday, September 17, 2019

Cisco Firepower - FireSight- FTD/FDM/FMC

I've been meaning to write a few things about Cisco FireSight/Firepower/FMC/FDM/FTD (please feel free to share acronyms for this product that I might have missed) for a while now but decided against it- until now. And if you are a Cisco employee working on the firepower product or just a hardcore Cisco security lover, this post will probably not sit well with you. With that being said, my intention is not to bash the Firepower team but rather provide constructive criticism -- or something of that nature :) I'm not going to go into any technical details as that would take a long time since I like to have all the evidence when incriminating someone or something - And since this is not a paid post, I will keep it simple.  

My first experience with Firepower was on an ASA using the CX modules around 2015 (FireSight/FMC 5.4.x) and after a few hours of use, I had a list of things that I thought required immediate attention (I've done a few beta tests for this product over the past few years as well):

1. Antiquated interface -- reminds me of the 1990's web interfaces.
2. Dashboard widgets took forever to load.
3. Excruciatingly slow when applying "deploy" changes. Even a minor change took several minutes to deploy. The initial logic for the deployment never made sense to me, it required snort engine services to be stopped, traffic was dropped, remove & reapply config -- just bad design I guess. It has gotten better over the years but not the time it takes to deploy. The screenshot below is after the 6.4 upgrade (before someone from the Cisco team asks, yes, we are using FS 4000, not vFMC, 4100's are still running 6.3 but 2130's are on 6.4)


4. No real live logs in FMC. Noticeable lag when looking at Connection Events.
5. Extremely limited/Non existent VPN related troubleshooting via FMC. Have to rely on the CLI.
6. Throughput limitations when enabling all the required features -- IPS/URL Filtering/Malware etc.
7. No support for SSL decryption in hardware (until 2017/2018 with Cavium Nitrox crypto chips). Even with the hardware support the ssl decrypt sucks -- try it on your "Prod" 4110 if you have one and let me know how much more love you gained for Cisco afterwards :) 

I dare not get into the details of the pain of using FMC on daily basis, the pain of upgrading 4100's (FXOS/FTD, FMC HA), or the countless hours of headaches just to find out that there is no real feature parity with the ASA's or an alternative to it. So fast forward to September 2019, we are now running 6.4.0.4 and guess what? If you said not much has changed then you are a winner. It baffles me that an industry giant like Cisco is failing yet again to deliver a solid product (remember LMS 4.0, now called Cisco Prime, how about Cisco Cius, or my favorite 6880's with IA's). They spent 2.7 billion on Sourcefire acquisition and millions on redoing the code and what do we have? A half baked product that still can't stand up to the competition. Oh wait, here is something that I've been complaining about for years that has NOT been fixed (we are in the year 2019 -- do I need to say more):


Now you might be thinking that I'm just venting -- well guess what, I am, thanks to the Cisco Firepower team :) But don't just take my word for it, look at what Todd has to say...yes, Todd Lammle, not my friend Todd who thought I studied for "Sysco" exams and drove the "Sysco Foods" big rig for living until I educated him on Cisco:


I'm not a big fan of Palo Alto Networks (mainly due to their poor support) but PAN and Fortinet have a much better firewall product than Cisco. In the coming weeks, Cisco is going to release 6.5 for the general public and those drinking Cisco Kool-aid are already raving about how it's going to turn this flawed product into one of the best -- I on the other hand am not holding my breath. I tried to do my part by providing feedback, showed up to the meetings (including the security forum, beta testing, multiple Cisco Lives), emails and phone calls to the AM's, SE's, BU and to anyone who was willing to listen. I had a detailed/granular list  of all the issues, resolutions, feedback that I kept until late last year but no more -- I realized it is just a waste of my time, that Cisco firepower team literally has no clue what to do with this flawed product/software (no offense Cisco team, it is what it is). I personally think that Cisco Firepower team is doing nothing more than putting the lipstick on a pig with this release of FTD 6.5.

Are you an FTD/FMC users? Let me know your thoughts/comments.