Gerstein Lab Computing in HPC

From GersteinInfo

(Difference between revisions)
Jump to: navigation, search
(OpenStack)
(29 intermediate revisions not shown)
Line 1: Line 1:
= Current Hardware =
= Current Hardware =
== Compute ==
== Compute ==
-
=== Grace ===
+
=== [https://research.computing.yale.edu/support/hpc/clusters/grace Grace] ===
33 nodes, 672 cores purchased in 2015
33 nodes, 672 cores purchased in 2015
 +
=== [https://research.computing.yale.edu/support/hpc/clusters/farnam Farnam] ===
 +
2 nodes, 64 cores purchased in 2012 (will be shutdown in 2019)
 +
 +
11 nodes, 308 cores purchased in 2018
 +
 +
1 large memory node (1.5TB RAM), 40 cores purchased in 2018
 +
 +
3 GPU nodes (2xNVIDIA K80), 60 cores purchased in 2016
-
Farnam 32 nodes, 256 cores purchased in 2010-2011
 
-
2 nodes, 64 cores purchased in 2012 (will be shutdown in 2019)
 
-
11 nodes, 308 cores purchased in 2018
 
-
1 large memory node (1.5TB RAM), 40 cores purchased in 2018
 
-
3 GPU nodes (2xNVIDIA K80), 60 cores purchased in 2016
 
2 GPU nodes (2xNVIDIA P100), 56 cores purchased in 2018
2 GPU nodes (2xNVIDIA P100), 56 cores purchased in 2018
 +
1 GPU node(4xNVIDIA TITAN V), 8 cores purchased in 2018
1 GPU node(4xNVIDIA TITAN V), 8 cores purchased in 2018
-
Total: 20 nodes, 536 cores
+
 
 +
Total: 20 nodes, 536 cores  
 +
 
*32 nodes were shutdown in September 2018
*32 nodes were shutdown in September 2018
-
Storage
+
=== [http://info.gersteinlab.org/Gerstein_Lab_Openstack_2017 OpenStack] ===
-
Loomis 3 TB default allocation
+
1 director node, 8 cores
-
300 TB purchased in 2015
+
 
-
100 TB purchased in 2016
+
3 controllers nodes, 24 cores
-
Total: 403 TB 93% used (27 TB free)
+
 
-
Farnam 4 TB default allocation
+
3 ceph nodes, 24 cores
-
90 TB purchased in 2014
+
 
-
276 TB purchased in 2017
+
5 compute nodes, 40 cores
-
757 TB purchased in 2018
+
 
-
Total: 1127 TB 83% used (193 TB free)
+
== Storage ==
-
*35 TB in /project on ysm-gpfs appears to be a free, undocumented loan?
+
=== Loomis (mounted on grace) ===
 +
3 TB default allocation
 +
 
 +
130 TB purchased in 2014
 +
 
 +
170 TB purchased in 2015
 +
 
 +
100 TB purchased in 2016
 +
 
 +
Total: 403 TB 93% used (27 TB free)
 +
 
 +
* 30 TB loan from HPC (ending Jan 2019)
 +
 
 +
=== Farnam (mounted on both farnam and grace) ===
 +
4 TB default allocation
 +
 
 +
90 TB purchased in 2013 (will retire in July 2019)
 +
 
 +
276 TB  purchased in 2016
 +
 
 +
757 TB purchased in 2018
 +
 
 +
Total: 1127 TB 83% used (193 TB free)
 +
 
 +
* Due to the limit of storage capacity, the actual will be 1017 TB + 90 TB (loan from HPC). The loan will be taken away in July 2019.
 +
 
 +
=== OpenStack ===
 +
Ceph nodes, 163TB
 +
 
 +
Compute nodes, 2TB
 +
 
 +
Director node, 2.2TB
 +
 
 +
Controller node, 2TB
 +
 
 +
70 TB with 10GB connection to farnam
 +
 
 +
= Actual CPU Usage =
 +
=== Grace ===
 +
Grace Shared 258,948 h (equivalent to ~120 cores at 100% utilization)
 +
 
 +
Grace Dedicated 371,982 h (equivalent to ~175 cores at 100% utilization)
-
+
Grace Scavenge 362,890 h (equivalent to ~165 cores at 100% utilization)
-
Actual CPU Usage - FY ‘19 Q1
+
-
Grace Shared 258,948 h (equivalent to ~120 cores at 100% utilization)
+
Grace Total 993,820 h (~462/672 cores)
-
Grace Dedicated 371,982 h (equivalent to ~175 cores at 100% utilization)
+
-
Grace Scavenge 362,890 h (equivalent to ~165 cores at 100% utilization)
+
-
Grace Total 993,820 h (~462 cores but own 672)
+
-
Farnam Shared 108,789 h (equivalent to ~50 cores at 100% utilization)
+
=== Farnam ===
-
Farnam Dedicated 262,642 h (equivalent to ~120 cores at 100% utilization)
+
Farnam Shared 108,789 h (equivalent to ~50 cores at 100% utilization)
-
Farnam Dedicated - GPU 13,250 h
+
-
Farnam Scavenge 29,527 h (equivalent to 13 cores at 100% utilization)
+
-
Farnam Total 414,208 h (~183 cores but own 536)
+
-
Ruddle Total 1,486 h�
+
Farnam Dedicated 262,642 h (equivalent to ~120 cores at 100% utilization)
-
Available Options (Rough Pricing and subject to change)
+
-
Standard Compute Node (28 cores, 192GB RAM) $12,000
+
Farnam Dedicated - GPU 13,250 h
-
GPU Node (28 cores, 192GB RAM, 1 V100) $22,000
+
-
Large Memory Node (28 cores, 768GB RAM) $19,000
+
-
Thinksystem D2 chassis (1 per 4 std nodes/2 GPU nodes) $2,000
+
-
Next Steps from 10/19/2018 Meeting
+
Farnam Scavenge 29,527 h (equivalent to 13 cores at 100% utilization)
-
$55-75k available - Dave to contact Janine Martinez for details of grant(s) and any additional detail on funding available.
+
Farnam Total 414,208 h (~183/536 cores)
-
Purchasing additional storage will be impossible in the time-frame due to the need to purchase in ~1PB increments.
+
-
The nature of the work suggests that Farnam is the appropriate place to put new equipment.
+
-
Prices for:
+
-
Large Memory Node (28 cores, 768GB RAM)
+
-
Large Memory Node (28 cores, 1536GB RAM)
+
-
Standard Compute Node (28 cores, 192GB RAM)
+
-
Standard Compute Node (28 cores, 384GB RAM)
+

Revision as of 19:49, 5 December 2018

Contents

Current Hardware

Compute

Grace

33 nodes, 672 cores purchased in 2015

Farnam

2 nodes, 64 cores purchased in 2012 (will be shutdown in 2019)

11 nodes, 308 cores purchased in 2018

1 large memory node (1.5TB RAM), 40 cores purchased in 2018

3 GPU nodes (2xNVIDIA K80), 60 cores purchased in 2016

2 GPU nodes (2xNVIDIA P100), 56 cores purchased in 2018

1 GPU node(4xNVIDIA TITAN V), 8 cores purchased in 2018

Total: 20 nodes, 536 cores

  • 32 nodes were shutdown in September 2018

OpenStack

1 director node, 8 cores

3 controllers nodes, 24 cores

3 ceph nodes, 24 cores

5 compute nodes, 40 cores

Storage

Loomis (mounted on grace)

3 TB default allocation

130 TB purchased in 2014

170 TB purchased in 2015

100 TB purchased in 2016

Total: 403 TB 93% used (27 TB free)

  • 30 TB loan from HPC (ending Jan 2019)

Farnam (mounted on both farnam and grace)

4 TB default allocation

90 TB purchased in 2013 (will retire in July 2019)

276 TB purchased in 2016

757 TB purchased in 2018

Total: 1127 TB 83% used (193 TB free)

  • Due to the limit of storage capacity, the actual will be 1017 TB + 90 TB (loan from HPC). The loan will be taken away in July 2019.

OpenStack

Ceph nodes, 163TB

Compute nodes, 2TB

Director node, 2.2TB

Controller node, 2TB

70 TB with 10GB connection to farnam

Actual CPU Usage

Grace

Grace Shared 258,948 h (equivalent to ~120 cores at 100% utilization)

Grace Dedicated 371,982 h (equivalent to ~175 cores at 100% utilization)

Grace Scavenge 362,890 h (equivalent to ~165 cores at 100% utilization)

Grace Total 993,820 h (~462/672 cores)

Farnam

Farnam Shared 108,789 h (equivalent to ~50 cores at 100% utilization)

Farnam Dedicated 262,642 h (equivalent to ~120 cores at 100% utilization)

Farnam Dedicated - GPU 13,250 h

Farnam Scavenge 29,527 h (equivalent to 13 cores at 100% utilization)

Farnam Total 414,208 h (~183/536 cores)

Personal tools