WT5.1 – Complex Networks and Massive Data Analysis Laboratory
One of the goals of the ENGINE project is the establishment of a new complex networks and massive data analysis laboratory that will support the ENGINE team in the research by providing scalable computing platform. The laboratory reached now fully functional state and is already used by research groups. It consists of three main components: utility servers, computing servers and storage.
Utility servers are multi-purpose blade servers that are able to run variety of operating systems and software and are fully customizable for end users. The laboratory has 10 of those servers and each of them has at least 144 GB of RAM and 12 cores. Computing servers (53 servers) are suited for high-performance computing (R, Python and others). They have either 64 or 128 GB RAM, 2 CPUs (12 cores each) and are running Linux operating system. They are installed as a part of the Bem supercomputing cluster located at Wrocław Centrer for Networking and Supercomputing in order to be able to make full use of their power. This cluster is listed among TOP 500 supercomputers in the world placed 137 (TOP500 June 2015 list) and the ENGINE servers within it provide abt. 7.3% of its computing power. Lastly, storage consists of 118 TB (gross) of SAS and NLSAS hard drives for storing and processing large datasets.
To sum up, the main goals of the complex networks and massive data analysis laboratory are the following:
- to provide a powerful and scalable environment for massive data analysis, modelling and visualisation, e.g. for analysing complex networks including social networks, text analysis, machine learning or processing signal streams,
- to provide huge and fast storage for storing the datasets and performing the on-disk operations (if too large for in-memory operations),
- to support other ENGINE project laboratories with computational power if needed.
Currently, the most of the research work run in this laboratory is devoted to the following areas: diffusion of information, sentiment analysis and group evolution prediction, as well as general machine learning tasks are also being run.
In case of any questions regarding the laboratory, please contact the task leader responsible for establishing and running that laboratory, Dr. Radosław Michalski (email@example.com).