Main Nav

It's been too quiet today. I hope everyone had a good Thanksgiving.

 

Switch installations require some many tasks (IP addressing, software upgrades, DNS hostnames, NMS, documentation, etc.) that I was wondering if any of you have your team fill out some sort of checklist to make sure that all these tasks actually happen. It's so easy to forget some of these things and sometimes you don't know that something was left off until you run into a problem. So, if you don't have a checklist, then how do you make sure that all these tasks are actually being done? How do you make sure that your team is adhering to the standards you are responsible for?

 

Thanks,

 

Hector Rios

Louisiana State University

********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/.

Comments

Message from cpolish@surewest.net

Hector J Rios wrote: > It's been too quiet today. I hope everyone had a good Thanksgiving. > Switch installations require some many tasks (IP addressing, > software upgrades, DNS hostnames, NMS, documentation, etc.) that > I was wondering if any of you have your team fill out some sort > of checklist to make sure that all these tasks actually happen. > It's so easy to forget some of these things and sometimes you > don't know that something was left off until you run into a > problem. So, if you don't have a checklist, then how do you make > sure that all these tasks are actually being done? How do you > make sure that your team is adhering to the standards you are > responsible for? Hi Hector, Possibly on-topic, here's a _similar_ checklist I threw together on a whim after a server move got mired in "issues". Perhaps it may help identify a step or two for a switch installation. Best regards, -- Charles Polisher =============================== Checklist: physical server move =============================== A. Power planning 1. Identify the rated mains load of the server at the voltage it will be plugged into at the destination (e.g., LRC is ?208?VAC). [Insert a quick-ref table here: HP Proliant 580/G5 = 20A@120VAC, 15A@208VAC..., PDU's in rack 1 max 20A per socket, max 150A per strip..., etc.] 2. Check that the increased load on the destination PDU (Power Distribution Unit) won't exceed the rating. 3. Check that the increased load on the circuit feeding the PDU isn't going to be exceeded. 4. Check that the increased load on the destination UPS won't exceed its rating. 5. Check that the expected decreased battery run-time on the destination won't fall below (Nagios-warning-threshold-on- run-time-in-minutes + 10% fudge factor). B. Networking 6. Check that the required VLAN is available at the aggregation switch that serves the destination rack, and whether any ACL's need to be altered. [Changes here could require firewall/ load balancer changes]. Identify and reserve one (could be 2+) available port(s) on that switch. 7. Check that the new rack location has an open kvm port. 8. Name the new switch port(s) on the destination switch. 9. Run a new cable from the new switch port to the new rack location. 10. Run a new kvm cable from the new rack location to the new kvm port. 11. Reserve the new kvm slot, update the label to: "sc123 Rack2:Pos5" C. Labeling 12. Label the switch end of the ethernet cable "sc123 Rack1-12" 13. Label the server end of the ethernet cable "SW1_A203_35" for switch SW1_A203, port 35. 14. Label the kvm end of the kvm cable "sc123 Rack1-12" D. Coordination 15. File a Change Request. Wait for approval. 16. Enter the maintenance window for down-time for the server in Nagios. E. Deployment 17.1 Check that the server's most recent backup completed successfully.. 17.2 Stop services/applications on the server. 17.3 Shut down the OS, and power down the server if not ACPI. 17.3 Move the server, reconnect cables, double-check they all seated properly. 18. Power up the server and boot into the BIOS. F. Console 19. Update the old kvm slot label to: "sc123 moved A201:1:35" (buildingroom:rack:position) 20. Hook up the server to the new kvm slot. 21. Update the rack location in the BIOS. 22. Continue booting, make sure the OS finishes booting. 23. Test connectivity from the KVM, the VLAN, and any other needed VLAN(s). G. Documentation 24. Check that the front/rear hostname labels didn't rub off in transit. 25. Update the rack location on the wiki. 26. Update the master portmap (or is this an automated, nightly step?). 27. Update the network map's physical location tag if the server was moved to a new building. H. Restore service 27. Bring up any applications that don't auto-start. 28. Make any required notifications. I. Post deployment networking 29. Assign the old switch port to the default non-routed VLAN. (We should have VLAN 0666 for this purpose]. Shut the port off. Erase the port label field in the switch config. 30. Remove the old cable that went from the old location to the old switch. 31. Close the Change Request. ================================= Why have a server move checklist? ================================= The pain points come when an outage occurs and (1) You can't restore service because the server is hung at a boot prompt, but you can't find the server because it's not labeled, the wiki doesn't say where it is, and the sysadmin who moved it to another building is on vacation. [occurred June 5, 2011] (2) Service restoration waits 30 minutes while you pull floor-tiles tracing a cable from a server to a switch. [Happened 2010, datacenter recabling] (3) You have to label 64 cables on a switch with the port number because the switch has to be swapped out but none of the cables was tagged. (4) After swapping out a switch you can't restore service because *one* cable wasn't tagged, and it could go several places, but they are on different VLANS. [happened 2010, datacenter recabling] With that in mind, which of the the steps proposed below save us time by *not doing them*, when these delays are figured in? How likely is it that all sysadmins called on to perform the manouver will remember to do each of them correctly without consulting a checklist? ********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/.
Message from dwcarder@wisc.edu

Close
Close


EDUCAUSE Connect
View dates and locations

Events for all Levels and Interests

Whether you're looking for a conference to attend face-to-face to connect with peers, or for an online event for team professional development, see what's upcoming.

Close

EDUCAUSE Institute
Leadership/Management Programs
Explore More

Career Center


Leadership and Management Programs

EDUCAUSE Institute
Project Management

 

 

Jump Start Your Career Growth

Explore EDUCAUSE professional development opportunities that match your career aspirations and desired level of time investment through our interactive online guide.

 

Close
EDUCAUSE organizes its efforts around three IT Focus Areas

 

 

Join These Programs If Your Focus Is

Close

Get on the Higher Ed IT Map

Employees of EDUCAUSE member institutions and organizations are invited to create individual profiles.
 

 

Close

2014 Strategic Priorities

  • Building the Profession
  • IT as a Game Changer
  • Foundations


Learn More >

Uncommon Thinking for the Common Good™

EDUCAUSE is the foremost community of higher education IT leaders and professionals.