Gerstein Lab Computing in HPC

From GersteinInfo

(Difference between revisions)
Jump to: navigation, search
(OpenStack)
(26 intermediate revisions not shown)
Line 1: Line 1:
= Current Hardware =
= Current Hardware =
== Compute ==
== Compute ==
-
=== Grace ===
+
=== [https://research.computing.yale.edu/support/hpc/clusters/grace Grace] ===
33 nodes, 672 cores purchased in 2015
33 nodes, 672 cores purchased in 2015
-
=== Farnam ===
+
=== [https://research.computing.yale.edu/support/hpc/clusters/farnam Farnam] ===
-
32 nodes, 256 cores purchased in 2010-2011
+
-
 
+
2 nodes, 64 cores purchased in 2012 (will be shutdown in 2019)
2 nodes, 64 cores purchased in 2012 (will be shutdown in 2019)
Line 22: Line 20:
*32 nodes were shutdown in September 2018
*32 nodes were shutdown in September 2018
 +
 +
=== [http://info.gersteinlab.org/Gerstein_Lab_Openstack_2017 OpenStack] ===
 +
1 director node, 8 cores
 +
 +
3 controllers nodes, 24 cores
 +
 +
3 ceph nodes, 24 cores
 +
 +
5 compute nodes, 40 cores
== Storage ==
== Storage ==
-
=== Loomis ===
+
=== Loomis (mounted on grace) ===
3 TB default allocation
3 TB default allocation
-
300 TB purchased in 2015
+
130 TB purchased in 2014
 +
 
 +
170 TB purchased in 2015
100 TB purchased in 2016
100 TB purchased in 2016
Line 33: Line 42:
Total: 403 TB 93% used (27 TB free)
Total: 403 TB 93% used (27 TB free)
-
=== Farnam ===
+
* 30 TB loan from HPC (ending Jan 2019)
 +
 
 +
=== Farnam (mounted on both farnam and grace) ===
4 TB default allocation
4 TB default allocation
-
90 TB purchased in 2014
+
90 TB purchased in 2013 (will retire in July 2019)
-
276 TB purchased in 2017
+
276 TB  purchased in 2016
757 TB purchased in 2018
757 TB purchased in 2018
Line 44: Line 55:
Total: 1127 TB 83% used (193 TB free)
Total: 1127 TB 83% used (193 TB free)
 +
* Due to the limit of storage capacity, the actual will be 1017 TB + 90 TB (loan from HPC). The loan will be taken away in July 2019.
-
+
=== OpenStack ===
-
Actual CPU Usage - FY ‘19 Q1
+
Ceph nodes, 163TB
-
Grace Shared 258,948 h (equivalent to ~120 cores at 100% utilization)
+
Compute nodes, 2TB
-
Grace Dedicated 371,982 h (equivalent to ~175 cores at 100% utilization)
+
-
Grace Scavenge 362,890 h (equivalent to ~165 cores at 100% utilization)
+
-
Grace Total 993,820 h (~462 cores but own 672)
+
-
Farnam Shared 108,789 h (equivalent to ~50 cores at 100% utilization)
+
Director node, 2.2TB
-
Farnam Dedicated 262,642 h (equivalent to ~120 cores at 100% utilization)
+
 
-
Farnam Dedicated - GPU 13,250 h
+
Controller node, 2TB
-
Farnam Scavenge 29,527 h (equivalent to 13 cores at 100% utilization)
+
 
-
Farnam Total 414,208 h (~183 cores but own 536)
+
70 TB with 10GB connection to farnam
 +
 
 +
= Actual CPU Usage =
 +
=== Grace ===
 +
Grace Shared 258,948 h (equivalent to ~120 cores at 100% utilization)
 +
 
 +
Grace Dedicated 371,982 h (equivalent to ~175 cores at 100% utilization)
 +
 
 +
Grace Scavenge 362,890 h (equivalent to ~165 cores at 100% utilization)
 +
 
 +
Grace Total 993,820 h (~462/672 cores)
 +
 
 +
=== Farnam ===
 +
Farnam Shared 108,789 h (equivalent to ~50 cores at 100% utilization)
-
Ruddle Total 1,486 h�
+
Farnam Dedicated 262,642 h (equivalent to ~120 cores at 100% utilization)
-
Available Options (Rough Pricing and subject to change)
+
-
Standard Compute Node (28 cores, 192GB RAM) $12,000
+
Farnam Dedicated - GPU 13,250 h
-
GPU Node (28 cores, 192GB RAM, 1 V100) $22,000
+
-
Large Memory Node (28 cores, 768GB RAM) $19,000
+
-
Thinksystem D2 chassis (1 per 4 std nodes/2 GPU nodes) $2,000
+
-
Next Steps from 10/19/2018 Meeting
+
Farnam Scavenge 29,527 h (equivalent to 13 cores at 100% utilization)
-
$55-75k available - Dave to contact Janine Martinez for details of grant(s) and any additional detail on funding available.
+
Farnam Total 414,208 h (~183/536 cores)
-
Purchasing additional storage will be impossible in the time-frame due to the need to purchase in ~1PB increments.
+
-
The nature of the work suggests that Farnam is the appropriate place to put new equipment.
+
-
Prices for:
+
-
Large Memory Node (28 cores, 768GB RAM)
+
-
Large Memory Node (28 cores, 1536GB RAM)
+
-
Standard Compute Node (28 cores, 192GB RAM)
+
-
Standard Compute Node (28 cores, 384GB RAM)
+

Revision as of 19:49, 5 December 2018

Contents

Current Hardware

Compute

Grace

33 nodes, 672 cores purchased in 2015

Farnam

2 nodes, 64 cores purchased in 2012 (will be shutdown in 2019)

11 nodes, 308 cores purchased in 2018

1 large memory node (1.5TB RAM), 40 cores purchased in 2018

3 GPU nodes (2xNVIDIA K80), 60 cores purchased in 2016

2 GPU nodes (2xNVIDIA P100), 56 cores purchased in 2018

1 GPU node(4xNVIDIA TITAN V), 8 cores purchased in 2018

Total: 20 nodes, 536 cores

  • 32 nodes were shutdown in September 2018

OpenStack

1 director node, 8 cores

3 controllers nodes, 24 cores

3 ceph nodes, 24 cores

5 compute nodes, 40 cores

Storage

Loomis (mounted on grace)

3 TB default allocation

130 TB purchased in 2014

170 TB purchased in 2015

100 TB purchased in 2016

Total: 403 TB 93% used (27 TB free)

  • 30 TB loan from HPC (ending Jan 2019)

Farnam (mounted on both farnam and grace)

4 TB default allocation

90 TB purchased in 2013 (will retire in July 2019)

276 TB purchased in 2016

757 TB purchased in 2018

Total: 1127 TB 83% used (193 TB free)

  • Due to the limit of storage capacity, the actual will be 1017 TB + 90 TB (loan from HPC). The loan will be taken away in July 2019.

OpenStack

Ceph nodes, 163TB

Compute nodes, 2TB

Director node, 2.2TB

Controller node, 2TB

70 TB with 10GB connection to farnam

Actual CPU Usage

Grace

Grace Shared 258,948 h (equivalent to ~120 cores at 100% utilization)

Grace Dedicated 371,982 h (equivalent to ~175 cores at 100% utilization)

Grace Scavenge 362,890 h (equivalent to ~165 cores at 100% utilization)

Grace Total 993,820 h (~462/672 cores)

Farnam

Farnam Shared 108,789 h (equivalent to ~50 cores at 100% utilization)

Farnam Dedicated 262,642 h (equivalent to ~120 cores at 100% utilization)

Farnam Dedicated - GPU 13,250 h

Farnam Scavenge 29,527 h (equivalent to 13 cores at 100% utilization)

Farnam Total 414,208 h (~183/536 cores)

Personal tools