By adopting a highly efficient 'chilled water' hybrid cooling technology that is unique amongst multi-user data centres within the University sector, the data centre is expected to significantly reduce power consumption and deliver to the University a 10% reduction in carbon emissions compared against its 2013 levels.
By early 2013, it was estimated that 90% of the world's digital data had been created during the previous 2 years. The amount of power required to store digital data accounts for 30-40% of any data centre's running costs, and McKinsey estimate that carbon emissions from data centres will quadruple by 2020, overtaking the airline industry in its impact on climate change. As a result, data centre design is rapidly evolving to place energy efficiency at the heart of the design process.
The University of Cambridge is committed to reducing its environmental impact. Its Carbon Management Plan for 2010-2020 commits to reducing the University's energy-related emissions by 34% by 2020 compared to 2005/06 base levels, so when it came to commissioning a new data centre, energy efficiency and carbon reduction was a prime motivator.
Rather than attempting to refurbish its existing diverse data storage infrastructures, the University and Cambridge Assessment formed a partnership to invest £20M in a bespoke world-class facility to support business operations, teaching and learning, and research communities for years to come.
“The West Cambridge Data Centre project will mark out the University as a clear early-adopter of the latest energy-efficient technology, showing what can be achieved and leading the way for other HE establishments to follow"
Ian Tasker, West Cambridge Data Centre manager.
Initially, the West Cambridge Data Centre will serve the current and future needs of UIS and the institutions for whom it manages IT infrastructure, the High Performance Computing Service (HPCS) supporting the University's research activities, and the administrative needs of Cambridge Assessment, that manages the University's three examination boards. Cambridge University Press are also planning to share the Cambridge Assessment Data Hall.
The different activities of these three user groups generate varying amounts of IT load, ranging from low-densities of 3.5kW per cabinet to a high IT density of 30kW per cabinet for intensive research-based data processing.
Determining the right approach for Cambridge
Traditional design approaches would advocate using three separate systems to support the three users' IT load types. After significant analysis, however, a 'one system' approach using the most appropriate new technology emerged as the best design solution.
The early design decision to supply air at the elevated ASHRAE A2 temperature range for all three user types unlocked the potential for creating our highly efficient design.
The 'chilledwater' solution pushes the industry towards a more flexible, yet still highly efficient system, delivering '100% free' cooling. This would not have been possible without the University's coming to trust that the benefits of using this novel 'chilledwater' system would far outweigh the risks – a decision many data centre clients would have shied away from.
Innovative hybrid cooling technology
Many approaches to cooling were explored, including all-air indirect evaporative systems. Power Utilization Effectiveness (PUE) and costing exercises were undertaken to assist with the tough decision-making. In order to meet the University's aspirations, however, it became obvious that the right solution for our data centre would need to go beyond the capabilities of all-air evaporative cooling.
In preference, a 'chilledwater' system was developed to deliver the same benefits of evaporative cooling, but without the use of chillers. This has allowed us to support both the low IT densities, which use hot-aisle containment and CRAH cooling, and the high IT density, which relies on rear-door cooling.
To support the high density cabinets for High Performance Computing, a back-of-rack cooling solution was adopted following the success of the system in a trial environment. Working with local supplier, Coldlogik, the University has been able to experiment and determine the optimum system settings to deliver the highest efficiency levels.
Designed for high availability and resilience
The centre has two independently routed Point of Presence rooms providing the incoming communications to the University. All the data cabinets, CRAHs, back-of-rack coolers, and electrical equipment in the building have dual power feeds.
Main power is supplied via dual 11,000KV feeds from UK Power Networks (UKPN) via separate substations, and a single 3,150Kva transformer. The centre has a 2,200Kva initial capacity from our provider, which can rise to 3,000Kva when this becomes available from UKPN.
Backup power is guaranteed by three 1,100Kva generator sets configured to N+1 (only two of which are required to supply sufficient backup power), with enough fuel to run for 72 hours. The three Hybrid Dry cooling towers are also designed for N+1 resilience, and only two of the three are required for normal operation.
Three 1,000Kva modular UPSs – also configured to N+1 – each comprising five 200Kva modules with intelligent controls, deliver 98% power efficiency. Two UPS output panels deliver separate A and B feeds to each cabinet via an overhead track busbar system, chosen for its flexibility. Power distribution is controlled and metered by intelligent cabinet power strips.
Four data halls
The new two storey 2,200m2 steel & block facility houses four data halls, designed for the different IT density requirements of its key stakeholders. There are currently 60 racks across three halls: Hall 1 accommodates the HPCS’s high density IT load of up to 900KW; Hall 2 provides 201KW for Cambridge Assessment’s needs, and Hall 3, 240KW for UIS' servers.
Hall 4 remains deliberately unallocated, and has not been fully fitted out. This forward-thinking decision allows us to install another 40-50 racks as future demand increases, leaving us with the maximum flexibility to incorporate the latest technologies as they emerge.
The purpose-built unit has a dedicated build room for engineering work, an operations room, security office and meeting room space. It also has a hoist for unloading deliveries, a large service elevator, designed to accommodate even the largest pieces of kit, and features an argonite gas fire suppression system.
Security is obviously a prime concern and the building itself has been designed with this in mind. It is manned by a dedicated security team who are on site 24/7, and has extensive CCTV coverage inside and out, and access to the site is tightly controlled.
Inside the building, security is no less important, since the facility must be accessible to IT staff from multiple University institutions. Access to the data halls is controlled by an anti-tailgating doorway, which can only be described as something out of Star Trek, featuring built-in weighing scales in its floor (leading some to recall Douglas Adams’ warnings to visitors of Bethselamin about the vital importance of remembering to get a receipt…). Once in the data halls, intelligent key cabinets monitor and control access to the individual cabinets' keys.
Low power, low carbon
Power usage effectiveness (PUE) is a measure of the ratio of power used by the data centre to the power consumed by the actual IT equipment; a perfect PUE would be 1. In January 2013, the European average was still in excess of 2.5.
Once fully operational, the West Cambridge Data Centre aims to deliver an overall PUE of 1.2, which is not far off that of large global players like Google (1.11 average for Q2 2014) and Facebook (1.06 and 1.08 averages for its two data centres, May 2013) who have made significant investments in their green data centres.
As a result of the power efficiency, we expect to see University carbon emissions reduce by 10% compared to current levels from its various machine rooms.
The data centre is currently undergoing final testing, and new technical staff are being recruited to operate the facility. Over the coming months a phased transition approach will see equipment relocated to the data centre, with much of it planned to be operational by year end.
"The substantial investment in the West Cambridge data centre reflects the University's commitment to excellent information services. This new facility will enable the university to benefit from information services that are of the highest quality, while substantially improving energy efficiency, and enabling valuable departmental space to be freed up to support teaching, learning and research."
Martin Bellamy, Director, University Information Services