Running on HPC

This guide covers three scenarios for running the SCOP Framework on HPC systems using SLURM:

1. Java and Maven are available on the HPC system.
2. Maven is not available on the HPC system (requiring a locally built fat JAR).
3. Both Java and Maven are not available on the HPC system (requiring Java installation with EasyBuild).

Checking Available Modules

To verify if the required modules (Java/21.0.2 and Maven/3.9.9) are installed on the HPC, follow these steps:

Check for Java

module avail Java

This command will list all available Java modules. Look for Java/21.0.2 in the output.

Check for Maven

module avail Maven

This command will list all available Maven modules. Look for Maven/3.9.9 in the output.

Case 1: Java and Maven Are Available on the HPC System

In this scenario, both Java and Maven are installed on the HPC environment, allowing you to build and run the SCOP Framework directly on the cluster.

Clone the Repository and Build the Project

• Clone the repository of your model and build it using Maven:

  git clone https://github.com/your-repo/your-model.git
  cd your-model
  mvn clean install
  

• This step compiles the Java code and packages the application into a runnable JAR file (typically target/your-model.jar).

Create a SLURM Job Script

• Write a job script to run the application, specifying the Java JAR file:

  #!/bin/bash
  
  ## Set up the job  
  # Note: Adjust the sbatch settings as needed 
  #SBATCH --nodes=8
  #SBATCH --time=00:05:00
  #SBATCH --qos=devel
  #SBATCH --account=account_name
  
  # Load Java and Maven modules
  module --quiet purge    # Reset the modules to the system default
  module load Java/21.0.2 # Loading Java 21
  module load Maven/3.9.9 # Loading Maven 3.9.9
  
  # Navigate to the your model's directory
  cd /path/to/your-model
  
  # Run the simulation using the JAR file
  # Note: Adjust the memory settings (-Xms and -Xmx) as needed
  java -Xms1g -Xmx32g -jar target/your-model.jar exec-cli
  
  echo $SLURM_SUBMIT_DIR
  
  exit 0
  

• Save this script as run_your_model_on_hpc.sh.

Submit the Job

• Submit the job using the sbatch command:

  sbatch run_your_model_on_hpc.sh
  

Check the Job

• Check the status of the job using the squeue command:

  squeue -u your_username
  

Case 2: Maven Is Not Available on the HPC System

In this scenario, Maven is not available on the HPC, so the build process must be done locally (on a development machine), and then the resulting “fat JAR” is transferred to the HPC system.

Build a Fat JAR Locally

• On your local machine (with Maven installed), navigate to the project repository and build a fat JAR that includes all dependencies:

  mvn clean compile package -Pshade -DskipTests
  

• This will generate a “fat JAR” file that includes all required libraries and dependencies, typically found in the target folder, e.g., target/your-model-version-fat.jar.

Transfer the Fat JAR to the HPC

• Use scp or rsync to copy the fat JAR to the HPC machine:

  scp target/your-model-version-fat.jar your_username@hpc.server.com:/path/to/hpc/folder
  

Create a SLURM Job Script

• On the HPC, write a job script to run the fat JAR:

  #!/bin/bash
  
  ## Set up the job  
  # Note: Adjust the sbatch settings as needed 
  #SBATCH --nodes=8
  #SBATCH --time=00:05:00
  #SBATCH --qos=devel
  #SBATCH --account=account_name
  
  # Load Java and Maven modules
  module --quiet purge    # Reset the modules to the system default
  module load Java/21.0.2 # Loading Java 21
  
  # Navigate to the your model's directory
  cd /path/to/your-model
  
  # Run the simulation using the JAR file
  # Note: Adjust the memory settings (-Xms and -Xmx) as needed
  java -Xms1g -Xmx32g -jar target/your-model-version-fat.jar exec-cli
  
  echo $SLURM_SUBMIT_DIR
  
  exit 0
  

• Save this script as run_your_model_on_hpc.sh.

Submit the Job

• Submit the job using the sbatch command:

  sbatch run_your_model_on_hpc.sh
  

Check the Job

• Check the status of the job using the squeue command:

  squeue -u your_username
  

Case 3: Java and Maven Are Not Available on the HPC System

In this scenario, neither Java nor Maven is available on the HPC system, so Java needs to be installed using EasyBuild, and a fat JAR must be generated locally.

Create an EasyBuild Script to in Java 21

• On the HPC, create an EasyBuild script to install Java 21:

  easyblock = 'Binary'
  
  name = 'JDK'
  version = '21'
  toolchain = {'name': 'foss', 'version': '2023b'}  # Adjust based on your available toolchains.
  
  homepage = 'https://www.oracle.com/java/'
  description = "Oracle JDK 21 - Java Development Kit"
  
  source_urls = ['https://download.oracle.com/java/23/latest/']
  sources = ['jdk-21_linux-x64_bin.tar.gz']
  
  # Adjust the installation prefix as necessary
  #install_path = '/cluster/home/ogurcan/software/OracleJDK-23'
  
  # Dependencies if needed
  dependencies = []
  
  # License (if applicable)
  license_file = 'LICENCE'
  
  # Adjust this to ensure proper extraction and installation
  install_cmd = 'tar -xzf jdk-21_linux-x64_bin.tar.gz -C %(installdir)s'
  
  sanity_check_paths = {
       'files': ['jdk-21.0.1/bin/java', 'jdk-21.0.1/bin/javac'],
       'dirs': ['jdk-21.0.1/lib']
  }
  
  moduleclass = 'lang'
  

• Save this script as JDK-21.eb.

Install Java 21 Using EasyBuild

• Use the EasyBuild script to generate the module file:

  eb JDK-21.eb --prefix=/path/to/install/directory
  

Load the Installed Java Module

• Load the newly installed Java 21 module:

  module use /path/to/install/directory/modules/all
  module load Java/21
  

Add Java to your PATH

• Manually add the JDK 23 bin directory to your PATH: bash export PATH=/path/to/install/directory/JDK/21.0.1/jdk-21.0.1/bin/:$PATH
• Verify that Java is installed correctly:

  java -version
  

Continue from Case 2

• Now that Java loaded, go to Case 2, and follow the steps.

Conclusion

In all cases, adjust paths and module names according to the specific setup of your HPC environment.

For more information on SLURM, refer to the SLURM documentation.