Lead Australian Research Institute

Creation of Optimally Synergised Systems with Red Hat Ceph

ESTABLISHED IN 1987

  • Has a mandated role to advise the Australian Government on all nuclear and science technology matters

PRODUCTS USED

The preeminent research institution in Australia houses the country's most notable research landmark and national infrastructure. It leverages cutting-edge science and technologies to tackle intricate problems and promote human health, environmental well-being, and industrial progress. Through a research and development (R&D) approach, the institute employs sophisticated technologies to analyze the smallest molecular details. Consequently, it generates and stores vast amounts of data that necessitate high-performance infrastructure.

DiGiCOR worked with the Scientific Computing Specialist at Australian Research Institute, where on the organisation’s behalf, proposed the Greenfield project to DiGiCOR. In collaboration with our network of vendors and partners, we were able to negotiate and create a stream-lined solution that first and foremost satisfied all their technical, budgetary and reliability requirements and expectations.



Utilizing HPC for Scientific Research provides the following benefits that Australian Research Institute achieved using the cluster proposed to them.

  • Reducing physical testing: HPC can be used to create simulations, eliminating the need for physical tests. This can save time, money and resources.
  • Speeding up calculations: With the latest CPUs, GPUs, and low-latency networking fabrics such as RDMA, coupled with all-flash local and block storage devices, HPC can perform massive calculations in minutes instead of weeks or months.
  • Enabling data analysis: HPC can help researchers handle large and complex data sets, such as genomic sequencing, climate modeling, or astronomical observations. HPC can also support artificial intelligence and machine learning applications that require high computational power.
  • Enhancing collaboration: HPC can facilitate collaboration among researchers across different disciplines and locations by providing a common platform for sharing data and resources. HPC can also enable cloud-based services that offer flexibility and scalability.

    • HIGHLIGHTS

    OBJECTIVES

    • Required a full turnkey solution that could operate Australian Research Institute’s new science instruments
    • Workloads: able to run and store virtual machines, cache analysis, handling live scientific data and bursts of data, and analysis of data through a HPC system
    • Fulfil the technical specifications and budgetary limitations compared with other vendors in the market

    HOW WE HELPED

    • We brought our network of vendors to satisfy the organisation’s challenges, needs, and better provide offerings in the landscape of competition.
    • Educate the organisation based on the latest cutting-edge technologies and communicate our best simplified and vertically integrated solution options.

    FEATURED SOLUTIONS

    Intel R2224WFTZSR

    Supermicro SuperServer 1114S-WTRT

    Seagate J12C6XA16900DA

    RESULTS

    • Able to provide better alternatives solutions and options towards a solution that was more cost-effective and also highly reliable.
    • Integrated, finely tuned, and bundling Intel, Supermicro, and Seagate as a streamlined solution.

    Challenges

    Australian Research Institute required a full turnkey solution that could operate their newly adopted science instruments related to their Greenfield project during the growth period of the Synchrotron.

    This type of project required general high performance compute and storage to handle:

  • Running and storing virtual machines
  • Cache Analysis
  • Handling live scientific data and bursts of data
  • Analysis of data through a HPC system

    • Ron Bosworth, a Scientific Computing Specialist from Australian Research Institute and the research organisation had a clear set of technical specifications and budgetary limitations comparable with other vendors in the market.



    How We Helped

    We worked closely with Bosworth and affiliate Australian Research Institute project members, bringing our network of vendors into the conversation to provide a solution that not only satisfied the organisation’s challenges and needs, but provided better offerings in the landscape of competition.

    This solution was highlighted through the progressive relationship from years of reliable and effective solutions and services with Australian Research Institute. As a result, allowing us to better understand the fundamental challenges and needs of this project, educate the organisation based on the latest cutting-edge technologies and communicate our best simplified and vertically integrated solution options.


    To meet Australian Research Institute’s technical and budgetary specifications, we proposed an All-Flash NVMe-focused Turnkey solution based on the:

    1.Intel Server R2000WF Family (Formerly Wolf Pass)

    The purpose of (3) Intel 2U R2224WFTZSR server systems powered by Intel Xeon Scalable in this project is to establish high performance storage in running the required virtual machines, and performing cache analysis.

    2.Supermicro Back-up Server

    The purpose of the Supermicro 1U SuperServer 1114S-WTRT powered by AMD EPYC in this project is to deliver fast and efficient data transfers for the backing up of scientific data.

    3.Seagate JBOD

    The purpose of the Seagate 4U J12C6XA16900DA JBOD with 1.69PB 106 bays in this project is to effectively and speedily handle live scientific data capture with bursts of data and analyse the data within a HPC system.


    Results

    Being able to bring our trusted partners and vendors into discussion with the situation and competition landscape, Australian Research Institute provided many advantages against competitors in terms of pricing advantages and services.

    As a result,we were able to provide better alternatives solutions and options that shifted away from the initial proposed solution from Australian Research Institute towards a solution that was more cost-effective and also highly reliable. In addition to all parts being integrated, finely tuned, and bunding Intel,Supermicro and Seagate as a stream-lined solution.

    Due to the nature of the Greenfield project,satisfying the project objective of establishing high storage for virtual machines,analysis and data capture was simple as a result of the tight collaboration between our expert team of presales, sales and vendors . In addition to the solution meeting impact and performance expectations of Australian Research Institute, stated by Bosworth.

    Overall in the approach, Bosworth provided positive feedback throughout the engagement phase between DiGiCOR and Australian Research Institute as we provided “interesting” solutions and options of what Australian Research Institute required. This is complemented by the speed and effectiveness of providing a solution to the problem. In addition to the regular updates of delivery and expected time frames.


    Featured Solution: Intel R2000WF Family

    Intel R2224WFTZSR(MCT Storage server: 24x 8TB NVMe)
    Intel-R2224WFTZSR

    Barebone:

    • 2U, Spread Core Rack supporting dual Socket P 2nd Generation Intel® Xeon® Scalable Processors
    • up to 7.5 TB DDR4 ECC RDIMM/LRDIMM 2133/2400/2666/2933 & Intel® Optane™ DC Persistent Memory Supported
    • 1x PCI-E x4 Gen 3, 3x PCIe x8 Gen 3, 2x PCIe x16 Gen 3
    • 24x Hot-swap 2.5" SATA3 drive bays, 2x M.2 SSD
    • Integrated IPMI 2.0 and Redfish
    • 3x 1GbE (mgmt port) + Dual Port RJ45 10GbE
    • 1300W Redundant Power Supplies
    • CPU: 2x Intel Xeon SILVER 4215R Processor 8-core, 16 thread, 3.2 GHz 11MB Cache (130W)
    • RAM: 192GB 2933 MHz ECC RDIMM
    • Storage: 24x Intel® P4510 (8TB NVMe 2.5inch)
    • Networking: Intel® XXV710-DA2 25GbE SFP28
    • Warranty: 5y Intel Warranty

    Featured Solution: Intel R2000WF Family

    Intel R2224WFTZSR(IMBL Storage server: 12x 8TB NVMe)
    Intel-R2224WFTZSR

    Barebone:

    • 2U, Spread Core Rack supporting dual Socket P 2nd Generation Intel® Xeon® Scalable Processors
    • Up to 7.5 TB DDR4 ECC RDIMM/LRDIMM 2133/2400/2666/2933 & Intel® Optane™ DC Persistent Memory Supported
    • 1x PCI-E x4 Gen 3, 3x PCIe x8 Gen 3, 2x PCIe x16 Gen 3
    • 24x Hot-swap 2.5" SATA3 drive bays, 2x M.2 SSD
    • Integrated IPMI 2.0 and Redfish
    • 3x 1GbE (mgmt port) + Dual Port RJ45 10GbE
    • 1300W Redundant Power Supplies
    • CPU:2x Inter Xeon SILVER 4215R Processor 8-core,16 thread, 3.5GHz 11MB Cache(130W)
    • RAM: 192GB 2933 MHz ECC RDIMM
    • Storage: 12x Intel® P4510 (8TB NVMe 2.5inch)
    • Networking: Intel® XXV710-DA2 25GbE SFP28
    • Warranty: 5y Intel Warranty

    Featured Solution: Intel R2000WF Family

    Intel R2224WFTZSR(Spectroscopy Storage server: 4x 8TB NVMe)
    Intel-R2224WFTZSR

    Barebone:

    • 2U, Spread Core Rack supporting dual Socket P 2nd Generation Intel® Xeon® Scalable Processors
    • Up to 7.5 TB DDR4 ECC RDIMM/LRDIMM 2133/2400/2666/2933 & Intel® Optane™ DC Persistent Memory Supported
    • 1x PCI-E x4 Gen 3, 3x PCIe x8 Gen 3, 2x PCIe x16 Gen 3
    • 24x Hot-swap 2.5" SATA3 drive bays, 2x M.2 SSD
    • Integrated IPMI 2.0 and Redfish
    • 3x 1GbE (mgmt port) + Dual Port RJ45 10GbE
    • 1300W Redundant Power Supplies
    • CPU: 2x Intel Xeon SILVER 4215R Processor 8-core, 16 thread, 3.2 GHz 11MB Cache (130W)
    • RAM: 192GB 2933 MHz ECC RDIMM
    • Storage: 4x Intel® P4510 (8TB NVMe 2.5inch)
    • Networking: Intel® XXV710-DA2 25GbE SFP28
    • Warranty: 5y Intel Warranty

    Featured Solution: SUPERMICRO Mainstream Server

    Supermicro SuperServer 1114S-WTRT(Backup Server)
    Seagate-J12O6XA16900DA

    Barebone:

    • 1U Server supporting Single AMD EPYC™ 7002 Series Processor
    • Up to 2TB 3DS ECC DDR4-3200MHz RDIMM/LRDIMM
    • 2x PCI-E 4.0 x16 (FHFL) slot, 1x PCI-E 4.0 x16 (LP) slot
    • 10x Hot-swap 2.5" SATA3 drive bays, Optional 2 U.2 NVMe (PCI-E 3.0) drive support via additional kit for NVMe devices
    • Integrated IPMI 2.0 and KVM with Dedicated LAN
    • 2x 10GBase-T LAN ports via Broadcom BCM57416 Controller
    • 500W Redundant Power Supplies Platinum Level High-efficiency
    • CPU:1x AMD EPYC 7302P Processor 16-core 3.00GHz 128MB Cache (155W)
    • RAM: 256GB 3200 MHz ECC RDIMM
    • O.S.Drive: 2x Intel S4510 240GB
    • Networking: Intel® XXV710-DA2 25GbE SFP28
    • Warranty: 5y RTB

    Featured Solution: Seagate Expansion Shelves and JBODS EXOS E Series

    Seagate J12C6XA16900DA(106 Bay JBOD)
    Seagate-J12O6XA16900DA

    Barebone:

    • 4U Rackmount JBOD System
    • 1.69PB 106x Bay
    • Dual IO modules
    • 4 - 12Gb x4 mini-SAS HD host and expansion ports
    • 2x 1GbE Ethernet RJ-45 port
    • Redundant 2000W PSUs
    • Storage: 106x 16TB 3.5" SAS Exos X16 512e/4Kn Enterprise HDDs
    • Warranty: 5y Seagate Limited Warranty

    Contact Us