The Grendel-cluster hardware

The cluster consists of:

293 SUN X2200 compute-nodes, each w. 2 quadcore Opteron 2.3 GHz, 16 GB memory and 500 GB disk.
25 SUN X2200 compute-nodes, each w. 2 quadcore Opteron 2.3 GHz, 32 GB memory and 2 TB disk.
94 Dell SC1435 compute-nodes, each w. 2 dualcore Opteron 2.6 GHz, 8 GB memory and 250 GB disk.
2 SUN X4100 fileserver, 2 EonStor Raid system; contributes w. ca. 48 TB for user filesystems.
1 Dell SC1435 fileserver, 1 EonStor Raid system; contributes w. ca. 5 TB for user filesystems.
1 SUN X2200 infrastructure server
1 SUN X4600 frontend server w. 4 Quadcore CPUs.
20 Supermicro compute-nodes, each w.
2 quadcore Nehalem 2.6 GHz, 24 GB memory and 1 TB disk, 2 Nvidia Tesla M1060 GPUs, Infiniband.
196 HP DL170h G6 compute-nodes, each w.
2 quadcore Nehalem 2.6 GHz, 24 GB memory and 1 TB disk, Infiniband.
Cluster interconnect: 17 Nortel Baystack 5510 1 Gbps switches.
10 Voltaire 4036 QDR Infiniband switches.
Management LAN: Ethernet, 100 Mb/s
Gateway PC, providing internet access for the clusternodes.

The 1 Gbps Ethernet network topologi of Grendel:

NB: In ring r1 the SUN x2100-nodes have been retired, and replaced by the HP DL170h G6 nodes. These nodes are also connected via a 40 Gbps Infiniband network.

Absent from the picture above are the 20 Supermicro GPU-nodes. These nodes are connected to a 1 Gbps Ethernet switch attached to r3, and a 40 Gbps Infiniband switch.

The 1 Gbps rack-switches are stacked in a ring-topologi with 40 Gbps (full duplex) backbone. The thick lines represent each a 4x1 Gbps Multilink Trunk between two stacks.
The ring desgination (r1, r2, r3, r4) can be used as a feature in PBS resource requests.

At Danmarks Statistik in Copenhagen, is placed a minor clone of the Grendel-cluster. This cluster, called DS-Grendel, works on classified data from DS, and is surrounded by very high security regulations. It is running as a autonomous system, not connected to Grendel at all. DS-Grendel is only available for the researchgroup, who got the DCSC-grant to acquire it. DS-Grendel consists of:

16 SUN X2200 compute-nodes, each w. 2 quadcore Opteron 2.3 GHz, 16 GB memory and 500 GB disk.
1 SUN X2200 file- and infrastructure server, w. ca. 2 TB disk for user filesystems.
Cluster interconnect: 1 HP Procurve-2824 switch, 1 Gbps.

Here are some pictures from the assembling of Grendel (the first x2100 nodes) at the SUN factory.


		ABOUT-DCSC	DCSC/SDU	DCSC/AU	DCSC/AAU	DCSC/DTU	DCSC/KU

	+Open all -Close all About DCSC /AU Home Information About DCSC /AU Organisation News Current News News Archive Resources Grendel cluster Grendel GPU cluster Huge IBM p6-cluster Freke SGI Altix Sleipner IBM p4-cluster Karlsen SGI Origin 2000 The Archive Grid Computing Statistics Utilization Graphs System Load Support How to Get Help How to Get a Grant How to Get an Account Utillities Links Contact DCSC /AU Contact DCSC Central Office Other DCSC Centers
			Overview Hardware Software Batchjobs Hints The Grendel-cluster hardware The cluster consists of: 293 SUN X2200 compute-nodes, each w. 2 quadcore Opteron 2.3 GHz, 16 GB memory and 500 GB disk. 25 SUN X2200 compute-nodes, each w. 2 quadcore Opteron 2.3 GHz, 32 GB memory and 2 TB disk. 94 Dell SC1435 compute-nodes, each w. 2 dualcore Opteron 2.6 GHz, 8 GB memory and 250 GB disk. 2 SUN X4100 fileserver, 2 EonStor Raid system; contributes w. ca. 48 TB for user filesystems. 1 Dell SC1435 fileserver, 1 EonStor Raid system; contributes w. ca. 5 TB for user filesystems. 1 SUN X2200 infrastructure server 1 SUN X4600 frontend server w. 4 Quadcore CPUs. 20 Supermicro compute-nodes, each w. 2 quadcore Nehalem 2.6 GHz, 24 GB memory and 1 TB disk, 2 Nvidia Tesla M1060 GPUs, Infiniband. 196 HP DL170h G6 compute-nodes, each w. 2 quadcore Nehalem 2.6 GHz, 24 GB memory and 1 TB disk, Infiniband. Cluster interconnect: 17 Nortel Baystack 5510 1 Gbps switches. 10 Voltaire 4036 QDR Infiniband switches. Management LAN: Ethernet, 100 Mb/s Gateway PC, providing internet access for the clusternodes. The 1 Gbps Ethernet network topologi of Grendel: NB: In ring r1 the SUN x2100-nodes have been retired, and replaced by the HP DL170h G6 nodes. These nodes are also connected via a 40 Gbps Infiniband network. Absent from the picture above are the 20 Supermicro GPU-nodes. These nodes are connected to a 1 Gbps Ethernet switch attached to r3, and a 40 Gbps Infiniband switch. The 1 Gbps rack-switches are stacked in a ring-topologi with 40 Gbps (full duplex) backbone. The thick lines represent each a 4x1 Gbps Multilink Trunk between two stacks. The ring desgination (r1, r2, r3, r4) can be used as a feature in PBS resource requests. At Danmarks Statistik in Copenhagen, is placed a minor clone of the Grendel-cluster. This cluster, called DS-Grendel, works on classified data from DS, and is surrounded by very high security regulations. It is running as a autonomous system, not connected to Grendel at all. DS-Grendel is only available for the researchgroup, who got the DCSC-grant to acquire it. DS-Grendel consists of: 16 SUN X2200 compute-nodes, each w. 2 quadcore Opteron 2.3 GHz, 16 GB memory and 500 GB disk. 1 SUN X2200 file- and infrastructure server, w. ca. 2 TB disk for user filesystems. Cluster interconnect: 1 HP Procurve-2824 switch, 1 Gbps. Here are some pictures from the assembling of Grendel (the first x2100 nodes) at the SUN factory.