Past HPC School Exercises: HPC workflow with sequential jobs

Copyright (c) 2013-2019 UL HPC Team <hpc-sysadmins@uni.lu>

Prerequisites

Make sure you have followed the tutorial "Getting started" and the HPC Management of Sequential and Embarrassingly Parallel Jobs tutorials.

For many users, the typical usage of the HPC facilities is to execute 1 program with many parameters. On your local machine, you can just start your program 100 times sequentially. However, you will obtain better results if you parallelize the executions on a HPC Cluster.

You will see below 3 use cases mentioned in past training events:

• Exercise 1: Use the serial launcher (1 node, in sequential and parallel mode);
  • application to Object recognition with Tensorflow and Python Imageai
• Exercise 2: Use the generic launcher, distribute your executions on several nodes (python script);
  • Illustration on an Image Watermarking process in Python
• Exercise 3: Advanced use case, using a Java program: "JCell".