I’ve done more Cisco configuration in the past month then I have done in my entire 10 year stint at Lambesis. I’ve found the Cisco IOS can be very daunting at times (all the time) but once you’ve got a handle on it, it’s incredibly powerful and robust. One project that I’ve had on my plate for quite some time now has been a firewall upgrade for both the main office (HQ) and our datacenter. This was not a simple project as I couldn’t screw up any of the VPN tunnels currently in place between the locations. Our existing VPN setup used the Cisco Concentrator 3000 Series. This is a great VPN device but it’s just that and no firewall. I was looking at the Cisco ASA 5510 or better however the price tag is insanely high. It is a great device but not worth the money over what you can get with a used PIX. Just like all technology there is a premium for the latest and greatest.

Cisco PIX 515E FirewallI turned to eBay to purchase all of the gear. I went with a PIX 515E Unrestricted Failover Pair w/ VAC+, 512MB ram at the datacenter, a single PIX 515E w/ VAC+, 512MB RAM at our headquarters, and the PIX 506e for all remote locations. The PIX 515E VAC+ can handle 190Mbps cleartext throughput and 135Mbps IPSec VPN throughput. Our datacenter is on a 100Mbps pipe and our HQ is currently on a 6Mbps pipe with plans to push it over 40Mbps within the next 6 months. I’ll eventually add a Failover PIX at the headquarters office to be on the safe side. Just for comparison’s sake, a comparable new ASA 5510 runs around $3,500/each while the above PIX configuration ran around $600/each. I was able to purchase three PIX 515E units and three 506e units for around $1000 less than the price of a single new ASA 5510 device. Note: there is no warranty or Cisco support on these devices.

First off, I am not a Cisco certified engineer or anything close to it. I learn by trial and error and luckily I have the opportunity most of the time to do just that. Our existing system was not cutting it and I’m too embarrassed to even tell you what it was. I will tell you that our Cisco 3620 router was doing NAT for us and had some aggressive ACLs. I was asking the router to do way too much and anytime I would fire up an rsync session over the VPN, the router would just crap out.

With a little help from a good friend at Nextlevel Internet, I was able to get the PIX 515e setup at our headquarters and start testing the setup. Keep in mind I have remote locations and users connected over the VPN back to our headquarters. If I changed the gateway to the newly installed PIX, it would cripple those remote locations as they would not be able to see the DNS servers for starters.

Here is what I did:

  1. Configure PIX 515e at headquarters and setup an unused private IP for testing
  2. Setup a couple test machines on our local network and use the PIX 515e for the gateway
  3. Once happy with results, I had to work on the datacenter setup because of the VPN tunnel between the two locations
  4. Replaced the 506e firewall at the datacenter with the PIX 515e FO pair
  5. Establish a PTP tunnel back to the PIX 515e at the HQ
  6. Configure remote VPN capabilities on PIX 515e at HQ for software VPN clients
  7. Add a Guest network at HQ with access only to the public interface
  8. Remove the private LAN, NAT and the ACL from the Cisco 3620 router at HQ
  9. Change the IP of the PIX 515e at HQ to be the gateway IP
  10. Go home (ok, I was already there actually)

There are definitely some differences between the PIX IOS and the Cisco Concentrator 3000 configuration which gave me some trouble. On the Concentrator my VPN client IP pool was the same subnet as my private LAN which worked fine for many years. On the PIX, I had to create a new subnet for my VPN clients and ensure my ACLs are setup correctly for proper routing and access. In addition I discovered I need to have separate ACLs for each PTP or remote VPN configuration which was different then how the Concentrator 3000 was configured.

PIX ADSM ViewOne thing that bite me the hardest was arp. After I moved a server from the 506E to the 515E at the datacenter it no longer worked. Turns out that I needed to have my ISP for the datacenter run a “clear arp” on the uplink router. Once I figured that out I was able to coordinate all my upgrades with them for zero downtime.

After a lot of trial and error, I am happy to report a very successful stable firewall/VPN upgrade. My Cisco 3620 router is happy once again just routing.

What I still have to do:

  1. Enable hairpin routing to work so all VPN locations and traverse one another. This will allow an authorized user to VPN into HQ and get into the Datacenter, for example.
  2. Setup AAA against our Mac OS X 10.5 Leopard server (Open Directory) for the remote VPN clients (software)
  3. Setup proper QoS for voice traffic across all devices
  4. Add a PIX 515 FO unit to HQ and configure accordingly.