{"id":23873,"date":"2026-03-13T10:12:16","date_gmt":"2026-03-13T14:12:16","guid":{"rendered":"https:\/\/carleton.ca\/scs\/?p=23873"},"modified":"2026-06-23T12:18:45","modified_gmt":"2026-06-23T16:18:45","slug":"moving-the-cloud-in-winter-how-80-servers-crossed-campus-without-shutting-down","status":"publish","type":"post","link":"https:\/\/carleton.ca\/scs\/2026\/moving-the-cloud-in-winter-how-80-servers-crossed-campus-without-shutting-down\/","title":{"rendered":"Moving the Cloud in Winter: How 55 Servers Crossed Campus Without Shutting Down"},"content":{"rendered":"\n<section class=\"w-screen px-6 cu-section cu-section--white ml-offset-center md:px-8 lg:px-14\">\n    <div class=\"space-y-6 cu-max-w-child-5xl  md:space-y-10 cu-prose-first-last\">\n\n            <div class=\"cu-textmedia flex flex-col lg:flex-row mx-auto gap-6 md:gap-10 my-6 md:my-12 first:mt-0 max-w-5xl\">\n        <div class=\"justify-start cu-textmedia-content cu-prose-first-last\" style=\"flex: 0 0 100%;\">\n            <header class=\"font-light prose-xl cu-pageheader md:prose-2xl cu-component-updated cu-prose-first-last\">\n                                    <h1 class=\"cu-prose-first-last font-semibold !mt-2 mb-4 md:mb-6 relative after:absolute after:h-px after:bottom-0 after:bg-cu-red after:left-px text-3xl md:text-4xl lg:text-5xl lg:leading-[3.5rem] pb-5 after:w-10 text-cu-black-700 not-prose\">\n                        Moving the Cloud in Winter: How 55 Servers Crossed Campus Without Shutting Down\n                    <\/h1>\n                \n                                \n                            <\/header>\n\n                    <\/div>\n\n            <\/div>\n\n    <\/div>\n<\/section>\n\n\n\n<p><span data-contrast=\"auto\">The School of Computer Science (SCS) facilities are currently undergoing major renovations, including the entire HP5100 wing of the Herzberg Building. To allow construction work to&nbsp;proceed, the wing must be temporarily vacated and&nbsp;relocated.<\/span><span data-ccp-props=\"{}\">&nbsp;<\/span><\/p>\n\n\n\n<p><span data-contrast=\"auto\">One of the most significant challenges in this process is moving the school\u2019s server room facility.&nbsp;This infrastructure supports departmental servers, specialized research equipment, and the OpenStack cloud platform used by the school.<\/span><span data-ccp-props=\"{}\">&nbsp;<\/span><\/p>\n\n\n\n<figure class=\"wp-block-image alignnone\"><a href=\"https:\/\/carleton.ca\/scs\/wp-content\/uploads\/sites\/260\/PXL_20250512_142335459-scaled.jpg\"><img decoding=\"async\" src=\"https:\/\/carleton.ca\/scs\/wp-content\/uploads\/sites\/260\/PXL_20250512_142335459-scaled.jpg\" alt=\"Andrew Pullin in a SCS server room\" class=\"wp-image-23874\"\/><\/a><figcaption class=\"wp-element-caption\">SCS server room relocation led by Andrew Pullin<\/figcaption><\/figure>\n\n\n\n<p><span data-contrast=\"auto\">The stakes are high. More than 2,000 undergraduate students rely on the OpenStack cloud for course assignments and laboratory work, with usage peaking during the fall and winter academic terms. At the same time, graduate students and researchers depend on the system around the clock to run Computer Science and Data Science experiments, simulations, and long-running computational workloads.<\/span><span data-ccp-props=\"{}\">&nbsp;<\/span><\/p>\n\n\n\n<p><span data-contrast=\"auto\">This creates&nbsp;a difficult question:<\/span><span data-ccp-props=\"{}\">&nbsp;<\/span><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>How do you relocate a critical server facility in the middle of the winter term while supporting more than 2,000 students, faculty, and staff who depend on it 24\/7, when the infrastructure runs one of the most complex cloud platforms ever created: OpenStack?<\/li>\n<\/ul>\n\n\n\n<p><span data-contrast=\"auto\">The School of Computer Science partnered with the university\u2019s Information Technology Services (ITS), which offered temporary server space in the Carleton Library to host the infrastructure during the renovation.<\/span><span data-ccp-props=\"{}\">&nbsp;<\/span><\/p>\n\n\n\n<p><span data-contrast=\"auto\">With a relocation site secured, the team considered two primary strategies for moving the server facility.<\/span><span data-ccp-props=\"{}\">&nbsp;<\/span><\/p>\n\n\n\n<p><span data-contrast=\"auto\">&nbsp;<\/span><span data-ccp-props=\"{}\">&nbsp;<\/span><\/p>\n\n\n\n<p><b><span data-contrast=\"auto\">Option 1: Full Shutdown and Rapid Relocation<\/span><\/b><span data-ccp-props=\"{}\">&nbsp;<\/span><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><span data-contrast=\"auto\">Under this approach, the entire server room would be powered down, physically moved to the new location in a single day, and then reassembled and brought back online.<\/span><span data-ccp-props=\"{}\">&nbsp;<\/span><\/li>\n\n\n\n<li><span data-contrast=\"auto\">The advantage of this strategy is speed. In the best-case scenario, the move could be completed within a day, resulting in only one to two days of downtime for users.<\/span><span data-ccp-props=\"{}\">&nbsp;<\/span><\/li>\n\n\n\n<li><span data-contrast=\"auto\">However, the risks were significant. If multiple servers&nbsp;failed to&nbsp;start after the move, or if a critical infrastructure node&nbsp;encountered&nbsp;problems, the entire OpenStack environment could remain offline for an extended period. In a worst-case scenario, service outages could stretch into&nbsp;days&nbsp;or even weeks, while systems were repaired and reconfigured.<\/span><span data-ccp-props=\"{}\">&nbsp;<\/span><\/li>\n<\/ul>\n\n\n\n<p><span data-contrast=\"auto\">&nbsp;<\/span><span data-ccp-props=\"{}\">&nbsp;<\/span><\/p>\n\n\n\n<p><b><span data-contrast=\"auto\">Option 2: Live Migration and Incremental Relocation<\/span><\/b><span data-ccp-props=\"{}\">&nbsp;<\/span><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><span data-contrast=\"auto\">&nbsp;The second&nbsp;option&nbsp;involved a slower, more deliberate process: migrating servers individually while gradually&nbsp;relocating&nbsp;hardware to the new facility.<\/span><span data-ccp-props=\"{}\">&nbsp;<\/span><\/li>\n\n\n\n<li><span data-contrast=\"auto\">Although this approach would take&nbsp;considerably longer, it offered a key advantage. Each server could be handled carefully and&nbsp;validated&nbsp;before proceeding to the&nbsp;next. If a problem occurred, it would affect only a single system rather than the entire infrastructure.<\/span><span data-ccp-props=\"{}\">&nbsp;<\/span><\/li>\n\n\n\n<li><span data-contrast=\"auto\">This incremental strategy significantly reduced the risk of a prolonged outage and ensured the OpenStack cloud could remain&nbsp;operational&nbsp;throughout the relocation.<\/span><span data-ccp-props=\"{}\">&nbsp;<\/span><\/li>\n<\/ul>\n\n\n\n<p><span data-ccp-props=\"{}\">&nbsp;<\/span><\/p>\n\n\n\n<p><span data-contrast=\"auto\">The SCS technical staff&nbsp;ultimately chose&nbsp;this incremental strategy. The effort was led by&nbsp;<\/span><b><span data-contrast=\"auto\">Andrew Pullin<\/span><\/b><span data-contrast=\"auto\">, who coordinated the migration plan and oversaw the relocation process.<\/span><\/p>\n\n\n\n<p><span data-contrast=\"auto\">Before any hardware was moved, the team first ensured the necessary network infrastructure was in place. The SCS subnet was extended between the Herzberg Building and the Carleton Library, effectively spanning both locations. Because the OpenStack cloud requires its infrastructure nodes to&nbsp;reside&nbsp;on the same subnet, this network configuration was critical.<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335551550&quot;:0,&quot;335551620&quot;:0,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">&nbsp;<\/span><\/p>\n\n\n\n<p><span data-contrast=\"auto\">With the subnet extended across both buildings, OpenStack could treat servers&nbsp;located&nbsp;in Herzberg and those in the library as part of the same environment, regardless of the physical distance between them. This allowed systems to be&nbsp;relocated&nbsp;gradually while&nbsp;remaining&nbsp;fully integrated with the existing cloud infrastructure.<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335551550&quot;:0,&quot;335551620&quot;:0,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">&nbsp;<\/span><\/p>\n\n\n\n<figure class=\"wp-block-image alignleft\"><a href=\"https:\/\/carleton.ca\/scs\/wp-content\/uploads\/sites\/260\/PXL_20250512_141449057-scaled.jpg\"><img decoding=\"async\" src=\"https:\/\/carleton.ca\/scs\/wp-content\/uploads\/sites\/260\/PXL_20250512_141449057-scaled.jpg\" alt=\"Karim Ismail and Andrew Pullin configuring a GPU server\" class=\"wp-image-23880\"\/><\/a><figcaption class=\"wp-element-caption\">SCS tech staff: Karim Ismail and Andrew Pullin configuring a GPU server<\/figcaption><\/figure>\n\n\n\n<p><span data-contrast=\"auto\">The OpenStack environment runs on a virtualized cloud infrastructure, where workloads exist as server images rather than being tied to specific physical machines. This architecture proved to be a major advantage during&nbsp;the relocation.<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335551550&quot;:0,&quot;335551620&quot;:0,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">&nbsp;<\/span><\/p>\n\n\n\n<p><span data-contrast=\"auto\">Virtual machine images could be migrated to the library facility ahead of the physical move.&nbsp;Each night, servers in&nbsp;Herzberg&nbsp;copied their images to the library location.&nbsp;By morning, once the workloads had successfully migrated, the now-vacant physical server in Herzberg could be safely powered down, removed, and transported to the new facility.<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335551550&quot;:0,&quot;335551620&quot;:0,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">&nbsp;<\/span><\/p>\n\n\n\n<p><span data-contrast=\"auto\">This approach significantly reduced risk. If an issue occurred during migration, it would affect only a single server rather than the entire cloud environment, allowing problems to be isolated and resolved without disrupting the broader system.<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335551550&quot;:0,&quot;335551620&quot;:0,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">&nbsp;<\/span><\/p>\n\n\n\n<p><span data-contrast=\"auto\">Winter, however, introduced an entirely different challenge.<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335551550&quot;:0,&quot;335551620&quot;:0,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">&nbsp;<\/span><\/p>\n\n\n\n<p><span data-contrast=\"auto\">This year Ottawa experienced a particularly harsh and snowy winter, so much so that the Rideau Canal remained open for&nbsp;<\/span><b><span data-contrast=\"auto\">56 days&nbsp;of skating&nbsp;<\/span><\/b><span data-contrast=\"auto\">(that\u2019s unusually long)<\/span><span data-contrast=\"auto\">. Moving more than 55 servers across campus in freezing conditions is not a trivial task.<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335551550&quot;:0,&quot;335551620&quot;:0,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">&nbsp;<\/span><\/p>\n\n\n\n<figure class=\"wp-block-image alignleft\"><a href=\"https:\/\/carleton.ca\/scs\/wp-content\/uploads\/sites\/260\/PANO_20200216_144437.vr_-scaled.jpg\"><img decoding=\"async\" src=\"https:\/\/carleton.ca\/scs\/wp-content\/uploads\/sites\/260\/PANO_20200216_144437.vr_-scaled.jpg\" alt=\"Rideau skating canal\" class=\"wp-image-23884\"\/><\/a><\/figure>\n\n\n\n<p><\/p>\n\n\n\n<p><span data-contrast=\"auto\">Fortunately, Carleton University has a unique advantage: its extensive underground tunnel system.<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335551550&quot;:0,&quot;335551620&quot;:0,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">&nbsp;<\/span><\/p>\n\n\n\n<p>The Supervisor of Operations at ITS, John MacGillivray, helped coordinate the move using the golf cart and trailer through the Carleton tunnels. Moving the equipment in small batches allowed the team to safely relocate servers between buildings without exposing them to the winter weather. What might have been a logistical nightmare outdoors instead became an efficient relocation route beneath the campus.<\/p>\n\n\n\n<p><span data-contrast=\"auto\">Over the course of less than two months,&nbsp;<\/span><b><span data-contrast=\"auto\">Andrew Pullin and the SCS technical team<\/span><\/b><span data-contrast=\"auto\">&nbsp;successfully&nbsp;migrated&nbsp;the entire environment during the middle of the winter academic term. The move included:<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335551550&quot;:0,&quot;335551620&quot;:0,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">&nbsp;<\/span><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>4 racks of equipment<\/strong><\/li>\n\n\n\n<li><b><span data-contrast=\"auto\">25 compute nodes<\/span><\/b><span data-contrast=\"auto\">&nbsp;totaling&nbsp;<\/span><b><span data-contrast=\"auto\">1,672 CPU cores<\/span><\/b><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335551550&quot;:0,&quot;335551620&quot;:0,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">&nbsp;<\/span><\/li>\n\n\n\n<li><b><span data-contrast=\"auto\">24 GPU servers<\/span><\/b><span data-contrast=\"auto\">&nbsp;containing&nbsp;<\/span><b><span data-contrast=\"auto\">138 GPUs<\/span><\/b><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335551550&quot;:0,&quot;335551620&quot;:0,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">&nbsp;<\/span><\/li>\n\n\n\n<li><span data-contrast=\"auto\">The full&nbsp;<\/span><b><span data-contrast=\"auto\">OpenStack infrastructure stack<\/span><\/b><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335551550&quot;:0,&quot;335551620&quot;:0,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">&nbsp;<\/span><\/li>\n\n\n\n<li><span data-contrast=\"auto\">Supporting&nbsp;<\/span><b><span data-contrast=\"auto\">networking and storage systems<\/span><\/b><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335551550&quot;:0,&quot;335551620&quot;:0,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">&nbsp;<\/span><\/li>\n<\/ul>\n\n\n\n<p>In the end, like magic, the SCS cloud was successfully relocated across campus in the middle of the winter term, transparent to end users, who continued their work without ever realising the servers themselves had physically moved across campus.<\/p>\n\n\n\n<figure class=\"wp-block-image alignnone\"><a href=\"https:\/\/carleton.ca\/scs\/wp-content\/uploads\/sites\/260\/ML-DC-Population-Video.gif\"><img loading=\"lazy\" decoding=\"async\" width=\"450\" height=\"450\" src=\"https:\/\/carleton.ca\/scs\/wp-content\/uploads\/sites\/260\/ML-DC-Population-Video.gif\" alt=\"Server room relocation timelapse\" class=\"wp-image-23886\"\/><\/a><figcaption class=\"wp-element-caption\">Server room relocation timelapse: front and back of each of the 4 racks<\/figcaption><\/figure>\n\n\n\n<p><span data-contrast=\"auto\">Modern network administration and virtualization technologies&nbsp;made&nbsp;this complex relocation possible.<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335551550&quot;:0,&quot;335551620&quot;:0,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">&nbsp;<\/span><\/p>\n\n\n\n<p><span data-contrast=\"auto\">Of course, there is one small catch.<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;335551550&quot;:0,&quot;335551620&quot;:0,&quot;335559738&quot;:240,&quot;335559739&quot;:240}\">&nbsp;<\/span><\/p>\n\n\n\n<p><span data-contrast=\"auto\">When the renovations are finished\u2026&nbsp;<\/span><b><span data-contrast=\"auto\">everything will have to be moved back.<\/span><\/b><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The School of Computer Science (SCS) facilities are currently undergoing major renovations, including the entire HP5100 wing of the Herzberg Building. To allow construction work to&nbsp;proceed, the wing must be temporarily vacated and&nbsp;relocated.&nbsp; One of the most significant challenges in this process is moving the school\u2019s server room facility.&nbsp;This infrastructure supports departmental servers, specialized research [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":23874,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":"","_links_to":"","_links_to_target":""},"categories":[137,53],"tags":[],"class_list":["post-23873","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-hp5100-wing-reno","category-tech-announcements"],"acf":{"cu_post_thumbnail":""},"_links":{"self":[{"href":"https:\/\/carleton.ca\/scs\/wp-json\/wp\/v2\/posts\/23873","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/carleton.ca\/scs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/carleton.ca\/scs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/carleton.ca\/scs\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/carleton.ca\/scs\/wp-json\/wp\/v2\/comments?post=23873"}],"version-history":[{"count":3,"href":"https:\/\/carleton.ca\/scs\/wp-json\/wp\/v2\/posts\/23873\/revisions"}],"predecessor-version":[{"id":24288,"href":"https:\/\/carleton.ca\/scs\/wp-json\/wp\/v2\/posts\/23873\/revisions\/24288"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/carleton.ca\/scs\/wp-json\/wp\/v2\/media\/23874"}],"wp:attachment":[{"href":"https:\/\/carleton.ca\/scs\/wp-json\/wp\/v2\/media?parent=23873"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/carleton.ca\/scs\/wp-json\/wp\/v2\/categories?post=23873"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/carleton.ca\/scs\/wp-json\/wp\/v2\/tags?post=23873"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}