Big Data Cluster
The Big Data Cluster, which has been operated at the Faculty of Computer Science as part of the Big Data Competence Centre since 2012 and was expanded in 2015, 2017, 2019 and 2020, makes it possible to teach the topic of big data in a practical and realistic way in courses, internships and student project work. In addition, students can acquire expertise in carrying out comparative studies with realistic data sets and application scenarios as part of their final theses.
The computers of the Big Data Cluster were procured from central and decentralised QSL funds (funds for quality assurance of studies and teaching) of Darmstadt University of Applied Sciences and the Faculty of Computer Science.
Update July 2020: 4 high-performance Dell PowerEdge C6420 computers were procured from central QSL funds (see below for equipment details).
Update December 2019: 4 high-performance Dell PowerEdge C6420 (see below for equipment details) were procured from funds of the Research Centre for Applied Computer Science.
Funding
Funding was provided in accordance with the law on improving the quality of study conditions and teaching in the state of Hesse.
Hardware
The core of the Big Data Cluster currently consists of 36 nodes that can be dynamically configured for different application scenarios as well as a corresponding network environment. The nodes differ in terms of processors, RAM and storage media in order to offer a suitable infrastructure for a wide range of big data scenarios. The technical details of the equipment can be found below.
Software
The following big data software products are currently offered on the cluster:
- Apache Spark
- Apache Hadoop
- MongoDB
- Couchbase
- Apache Cassandra
- Neo4j
- SAP HANA
Further information on the installed software and how to use the cluster can be found in the Big Data Cluster wiki (log in with h-da account). Further software can be supported if required.
Technical equipment in detail
The 36 nodes differ in terms of processors, RAM and storage media in order to offer a suitable infrastructure for a wide range of big data scenarios:
- 16 x Dell PowerEdge C6220
- 2 Intel Xeon E5-2609 (4 cores each)
- 64 GB RAM
- 4 x 1 TB SATA 7.2 k
- 8 x Dell PowerEdge C6320
- 2 Intel Xeon E5-2620v2 (6 cores each)
- 128 GB RAM
- 4 x 1 TB SATA 7.2 k
- 2 x Dell PowerEdge C6320
- 2 Intel Xeon E5-2620v2 (6 cores each)
- 128 GB RAM
- 2 x 1 TB SATA 7.2 k
- 2 x 200 GB SATA SSD
- 2 x Dell PowerEdge C6320
- 2 Intel Xeon E5-2695v2 (12 cores each)
- 128 GB RAM
- 2 x 1 TB SATA 7.2 k
- 2 x 200 GB SATA SSD
- 4 x Dell PowerEdge C6420
- 2 x Intel Xeon Gold 6254(18 cores each)
- 384 GB RAM
- 4 x 480 GB SSD
- 2 x 960 NVMe hard drive
- New: July 2020: 4 x Dell PowerEdge C6420
- 2x Intel Xeon Gold 6226 (12 cores each)
- 384 GB RAM
- 4x 512 GB NVMe
