Main Nav

Message from rrichman@nd.edu

We are beginning a project to upgrade our core and distribution layers on campus. So items that are looking at is the ability to dual home buildings to the distribution layer.

Our current environment is cisco cat6509s for the core layer 3 and cat6509s for the distribution layer that homes the building.

One item we currently use is each distribution router/switch has a cisco firewall service module that we use in transparent mode to create 'zones' for staff, student, services, infrastructure devices, etc.  Endpoints are placed on the network in the appropriate zone using Clean Access server.

The challenge we are looking at is how we would utilize something like Cisco Nexus in the distribution layer that would allow us to dual home buildings with the vPC feature, but this leaves us with the FW layer that does not fit cleanly in this topology.

So what I am looking for input from the group is:

1.       Do you dual-home buildings, and if so, to the same distribution layer device or  multiple? Does history of uptime and time to repair the single linked buildings come into consideration?

2.       If you dual home, do you have layer 3 at the building, or do you ether-channel or spanning tree?

3.       Do you provide segmented vlans for users, and if so, how do you apply the policy between them?

Any other lessons learned would also be interesting to me too!

 

Thanks for your time,

 

Bob Richman

University of Notre Dame

********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/.

Comments

Bob,

We just finished replacing all of our core and distribution EOL 6509's with Nexus7009's this past August. VDC's and fabricpath were two major features we chose to use in the new design.
Within the same project, we also replaced our core active/active FWSM's with active/standby ASA5585X's.
We would be happy to discuss some of the project details with you. Please email me directly at your convenience.

For the list post -

1. We currently dual-home buildings to multiple distribution devices. There are many advantages to dual-homing - longer uptime, increased throughput capacity, potential for load-balancing, increased maintenance window approval.
2. Depending on firewalling needs, we currently have layer 3 either on the distribution VDC of the Nexus7k or as a sub-int of our multi-context ASA5585X. In most cases we use STP - when something breaks, a sub-minute STP outage is generally accepted by our clients.
We are developing designs to move most of the layer 3 down to the building layer. With a mixture of trunked p2p vlans, EIGRP peering, and VRF's, we believe we can adapt any building's current network topology to the new solution. In most cases, we hope to use EIGRP (we are currently testing the 9.X IOS for the ASA that will support EIGRP in multi-context mode). We hope to also expand our implementation of fabricpath.
3. We do segment users by vlan. When multiple departments inhabit a building (very common) and require firewalling, we currently put their layer 3 interface on the core ASA and trunk it down to the building as necessary. With layer 3 at the building, we use VRF's to direct traffic to/from the firewalled client out through the ASA to secure it from the other "local" SVI's.


Thank you,
Joe Marentette Network Engineer Washington University in St. Louis Network Services & Support 314-935-7031 jmarentette@wustl.edu
On 12/18/2012 1:35 PM, Bob Richman wrote:

So what I am looking for input from the group is:

1.       Do you dual-home buildings, and if so, to the same distribution layer device or  multiple? Does history of uptime and time to repair the single linked buildings come into consideration?

2.       If you dual home, do you have layer 3 at the building, or do you ether-channel or spanning tree?

3.       Do you provide segmented vlans for users, and if so, how do you apply the policy between them?

Any other lessons learned would also be interesting to me too!


We are in the finishing stage of a long (gradual migration, not a forklift) infrastructure upgrade.  We had a basic hub-and-spoke network originally with all L3 at the core.  We experimented with some new buildings and renovations doing L3 at the building, but it was a pure point-to-point link to the core.

The new network is L3 at the building, the network is very VRF oriented (gets the nasties and compliance things in their own world), and we've gone with private addressing campus-wide.  The "legacy" network was imported into the new one as a VRF, and the "new" VRFs grew up around it, taking their pieces out of the ever-shrinking legacy.

We have active/active ASA5585s on the border, active/passive (but suitable to cluster if load becomes an issue) 5540s for VPN, and active/passive FWSMs at the core that handle inter-VRF exceptions.  We run default-deny inbound, and with private addressing, if you don't have a static NAT and some access exceptions, nothing gets in.

Our goal is a VSS core but for now we are a single chassis (we're getting there...  dual Sups, and getting linecards suitable for VSS).  Routing is done on backbone "rings" from the core and aggregation points and not extended into the building beyond the L3 CE router.  The core and server farm top-of-racks do the PE functions and mesh certain VRFs (we try to keep services and users in separate VRFs, then import/export users to their designated services, and let the FWSM handle the exceptions).

We basically duplicate access/privilege "roles" as like-numbered vlans within a building where appropriate.  We're not doing heavy-duty 6500 MPLS for the VRFs, just VRF-lite with dedicated dot1q trunks (much cheaper, and you can back off to a 45xx/35xx/37xx CE in the buildings).

We also have split "cores" for campus, dorms, and wireless; each has their own path into the 5585s so the bulk of Internet traffic can bypass the core.  We have the option to home-run the server farms as well, but to this point that has not been an issue.

Jeff
********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/.

We are not dual-homing many buildings, but our design has been to route at the edge so we do intend to do this in the future.  We're using Juniper though, so "distribution" layer is blurred.  With regard to #3, everything goes back to the firewall.  We subscribe to a Zero Trust model.

-Brian

HI Bob,

 

1.       Do you dual-home buildings, and if so, to the same distribution layer device or  multiple? Does history of uptime and time to repair the single linked buildings come into consideration?

Try to dual-home/link when we can over diverse fiber paths out of same building – typically to the same “Primary” device.  Have had very little downtown due to a single “Primary” switch failing – almost never. Slowly moving from 1G to 10G uplink  We are about to do a stacked 3750-X to provide device diversity on a new large building.

2.       If you dual home, do you have layer 3 at the building, or do you ether-channel or spanning tree?

We do layer 3 at core – using etherchannel. I’ve debated layer 3 at the building…a lot more complexity, but I know of large designs that do this.

3.       Do you provide segmented vlans for users, and if so, how do you apply the policy between them?

Yes, segmented for users, etc. Using Router ACLs and or mainly NAC policy’s.  We are looking at changing this model a MPLS backbone enabled LAN design spanning multiple core boxes and doing some additional firewalling in key cores – nothing firm yet.

Thanks,

Chad

Assistant Director of Telecommunication Planning & Implementation

University Technology Services

University of Denver

2100 S. High St. #112

Denver, CO 80208

Desk Phone: 303-871-4441

Mobile Phone: 303-520-5657

 

 

 

********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/.

Our plan has been to dual home buildings, but don’t yet have the diverse fiber paths the the secondary core location.  In the past we have had a few buildings dual homed to 2 Catalyst 6500’s with layer 3 at the core and using using VRRP and spanning-tree.  It seemed to work well for years. 

 

We’ve recently upgrade to Nexus 7000’s in the core, and still route at the core.  The dual homed connections are now using VPC, and VRRP.  The plan is to use this model for all major building connections as the fiber is installed. We have a hand-full of VLANs that extend to multiple buildings that are trunked from the core to distribution.  We considered VRF and MPLS, but decided against it mainly due to complexity and support concerns.

 

A related issue I found is that you cannot route over a VPC link.  It is not supported by Cisco.  They require a separate physical link for routing.

 

We don’t do VLAN steering.

 

Ron

 

Close
Close


Annual Conference
September 29–October 2
Register Now!

Events for all Levels and Interests

Whether you're looking for a conference to attend face-to-face to connect with peers, or for an online event for team professional development, see what's upcoming.

Close

Digital Badges
Member recognition effort
Earn yours >

Career Center


Leadership and Management Programs

EDUCAUSE Institute
Project Management

 

 

Jump Start Your Career Growth

Explore EDUCAUSE professional development opportunities that match your career aspirations and desired level of time investment through our interactive online guide.

 

Close
EDUCAUSE organizes its efforts around three IT Focus Areas

 

 

Join These Programs If Your Focus Is

Close

Get on the Higher Ed IT Map

Employees of EDUCAUSE member institutions and organizations are invited to create individual profiles.
 

 

Close

2014 Strategic Priorities

  • Building the Profession
  • IT as a Game Changer
  • Foundations


Learn More >

Uncommon Thinking for the Common Good™

EDUCAUSE is the foremost community of higher education IT leaders and professionals.