Main Nav

Message from

Anyone out there load balancing between 2 separate ISPs?


We’ve been tasked with setting this up by midyear 2012.


Any info would be appreciated.


Also, anyone that has gone through the trauma of changing ISPs and having to change all public IP addresses (current ISP owns our public addresses). Hints and gothchas would be appreciated.


Mearl Danner

Systems Programmer

Samford University Technology Services


********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at




If you don’t have anyone experienced with BGP there are appliances out there that do this sort of thing.  We have used a FatPipe WARP for the past year and a half and it works quite well for us.  We have a 100 Meg wireless connection from Towerstream and a DS3 through Centurylink balanced through ours.  For FatPipe, the solution can get expensive as your bandwidth grows, and if you start going over 500 Meg of throughput, it would be time to look at a more “standard” BGP type scenario.


A solution like this *might* help you with your renumbering problem by allowing you to renumber gracefully over time by NAT’ing the IP block from your (soon to be) old ISP.  NAT’ing has its downsides, but depending on your situation, it might help you manage the process at a better pace once you cut over to the new ISP.


Other products to look at would be:


FatPipe WARP

Ecessa’s Powerlink

F5’s Big IP Link Controller

Barracuda’s Link Balancer

Peplink Balance

Elfiq Link Balancer

Mushroom Networks Truffle

Radware’s Link Load Balancer

XRoads Networks Link aXcel

Xtera’s ascenlink


Some of these talk directly to datacenter control, but I would think most, if not all, could be tuned to handle the entire LAN.


Hope this helps from the appliance side.


Paul Walker

Division Manager, Computer & Network Support | Information Systems



Message from


As a systems programmer, you might appreciate pfSense or just doing it in Linux with iproutes. 


Jesse Safran
Sr. Desktop Supervisor/Assist. Network Admin
Green Mountain College

Glad you asked the question.  We plan on replace one of our ISPs this spring/summer too.   

About the load balancing, I am looking into using PfR (Performance Routing) to balance our routes based upon available bandwidth.  Is anyone using it in production?  

Thank you,

Christina Klam
Network Administrator
Institute for Advanced Study

Einstein Drive          Telephone: 609-734-8154
Princeton, NJ 08540     Fax:  609-951-4418

We are primarily concerned with outbound redundancy, as opposed to load balancing incoming traffic, so we decided to forego the BGP route, and have in place 2 devices, a Fatpipe, and a barracuda link balancer – one for the administrative network and one for our ResNet.  We’ve used the fatpipe box for many years and are enamored with it.  The barracuda is just turning a year old, but they’ve already proved their worth a few times, including 2 days ago, when one of our ISPs was having trouble (they claimed it was Verizon) and our link through them was down. The only change I had to make to our entire infrastructure was killing the connection to the smtp smart host who’s public IP is registered on that network, so it didn’t try to send mail during the outage.  I believe fatpipe does do inbound load balancing via DNS if that’s what you want, but we’ve never explored that feature.


Jeremy Pinquist

Network Manager

The Juilliard School


We went through a similar migration of IP addresses and switched to a multiple WAN environment a little less than a year ago. I can't say what would be best for Samford or some of the other institutions that have expressed interest but our solution has been very robust and addressed our needs. About two years ago we purchased an IPv4 address block and AS number from ARIN and we now announce that address space from our campus through two ISPs. While the BGP advertising from multiple ISP paths improves our redundancy for inbound connections we also use the separate ISP gateways to load-balance for traffic originating from campus. I was surprised to see a comment in this thread about pfSense because we have used pfSense continuously since 2005 when it was still in beta. Due to that history and fluency with the product, we decided to continue with it for this multi-wan project. pfSense gives us some surprisingly high end features and its reliability is incredible. One feature that we're very happy with is CARP, a variation on VRRP, which we're using to provide redundancy of the router itself. This has let us split our routing between two routers housed in different datacenters within the campus. We've also found that the solution scales well. While we originally used pfSense to handle 7Mb/s DSL connections we now balance two 100Mb/s connections. Also, choosing an open source solution didn't leave us in the cold for support. One of the benefits of pfSense is that commercial support is available and the provider we chose is staffed by the major code contributors of the project. They also authored pfSense: The Definitive Guide (ISBN:978-0-9790342-8-2) which we used extensively to develop our configuration. During the time since we've installed this system we've found issues with load-balancing behind a NAT. In most cases the problems have been simple to mitigate. In particular, we've found that many secure websites do not appreciate the IP address of a client changing during a session. This kind of load balancing puts you at the mercy of the balancer software's algorithms and we've found it useful to turn off balacing for certain protocols. Connection affinity, or sticky connections in pfSense parlance, has undergone some revision between the version that we're running and the current "shipping" edition and we've yet to test. Since load-balancing is the bread and butter of other products like FatPipe you may find that they handle these issues more gracefully. In the list of gotchas, I think you've already seen one of them. If the ISP owns your IP addresses you're going to have trouble changing to a new vendor. The initial pain of switching to our own addresses has been well worth it. Now, should we need a new ISP, they just need to provide us with a BGP peer and our addresses become available over the new path. Your mileage may vary but we found that our ISPs were very helpful in the initial setup. As we migrated our servers away from our old addresses our former ISP was willing to create a GRE tunnel from their network through our new routers so that we could continue using a few of the /24 blocks. One of the decisions we made early on was to purchase new Cisco routers at the ISP edge rather than attempt to configure BGP within pfSense which may have complicated matters during the setup. While we chose to announce our own addresses we had considered asking our ISPs to announce half of our address range each and statically route them to us. While this solution is a lot less flexible it wouldn't have required nearly as much configuration or equipment. In this model we planned on redundancy of our services rather than our specific IPs and we carried along this idea into the final network design. Our BGP announcements are set so that half of our IP block has an affiinity for one ISP's path and the other half are routed through the other ISP. We then set an MX and a DNS server in each network range. This way, even if BGP fails to converge as we expect, or as quickly as we expect, inbound SMTP and DNS resolution of the domain will continue to operate. We were able to bring these networks into campus through different datacenters too so it also offers us some datacenter redundancy. You could also bind addresses in both networks to servers needing high availability (e.g., web server cluster). ________________________________ Ian Bergeron - M.Ed, ACTC, ACMT, Net+ Administrator of Networked Systems MCLA Computer Support Services Office:(413)662-5394 - Cell:(413)663-0957 The EDUCAUSE Network Management Constituent Group Listserv writes: >Anyone out there load balancing between 2 separate ISPs? > > > >We've been tasked with setting this up by midyear 2012. > > > >Any info would be appreciated. > > > >Also, anyone that has gone through the trauma of changing ISPs and having >to change all public IP addresses (current ISP owns our public >addresses). Hints and gothchas would be appreciated. > > > >Mearl Danner > >Systems Programmer > >Samford University Technology Services > >[ ] > > > > >********** Participation and subscription information for this EDUCAUSE >Constituent Group discussion list can be found at > > > > ********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at

As a follow-up to my post, I noticed that the 12/4/11(Show 79) Packet Pushers podcast discussed link load balancers.  It was a sponsored podcast focusing on the Elfiq product line, but may be worth a listen nonetheless. 


Also, someone from the list pointed out another product to add to my list below, so I am sharing it with you all.



Paul Walker

Division Manager, Computer & Network Support | Information Systems