Slurm API

This documentation provides comprehensive guidance on interfacing with the SLURM API for job submission within the PowerSlurm cluster. These instructions can be used for submitting jobs that originate from a web interface.

For a detailed understanding of the API's capabilities and functionalities, refer to the official SLURM documentation at SLURM REST API Documentation. https://slurm.schedmd.com/rest_api.html

Authentication

Introduction

Secure access to the SLURM REST API is managed through JWT (JSON Web Tokens). This section provides a step-by-step guide on how to obtain a JWT token, which is essential for authenticating and authorizing API requests.

Prerequisites

• An API key provided by the High-Performance Computing (HPC) team.

Constants

• Base URL for the SLURM REST API: https://slurmtron.tau.ac.il
• Endpoint for token generation: https://slurmtron.tau.ac.il/slurmapi/generate-token/

Python Example for creating a JWT token

import requests

def get_api_token(username, api_key):
    """
    Retrieves a JWT token for SLURM REST API access for powerslurm cluster.

    Parameters:
    username (str): The username of the user requesting the token.
    api_key (str): The API key provided by the HPC team.

    Returns:
    str: The API token if the request is successful.

    Raises:
    Exception: If the request fails with a non-200 status code.
    """

    generate_token_url = 'https://slurmtron.tau.ac.il/slurmapi/generate-token/'

    payload = {
        'username': username,
        'api_key': api_key
    }
    
    response = requests.post(generate_token_url, data=payload)

    if response.status_code == 200:
        # Extracting the token from the JSON response
        return response.json()['SlurmJWT']
    else:
        raise Exception(f"Error: {response.status_code}, {response.text}")

# Example usage
# Replace 'your_username' and 'your_api_key' with actual values
# token = get_api_token('your_username', 'your_api_key')

Manual Token Generation (General Method)

1. Send a POST request to https://slurmtron.tau.ac.il/slurmapi/generate-token/ with your username and API key.
2. The request should be in the format of a JSON payload containing your credentials.
3. On success, the server will return a JSON response in the format: { "SlurmJWT": "token" }.
4. Extract the SlurmJWT value from the response. This is your required JWT token.

Security and Best Practices

• Keep your API key and JWT token confidential.
• Use the JWT token in the header of your API requests for authorized access to the SLURM REST API.

Job Submission to SLURM REST API:

Introduction

This section covers the process of submitting a job to the SLURM REST API at Tel Aviv University

Prerequisites

• Access to the SLURM REST API.
• An API key and username, provided by the Tel Aviv University HPC team.
• A tool or library for making HTTP requests (e.g., requests in Python).

Python Example:

#!/usr/bin/python3

import requests

# Base URL for authentication and token generation
base_url_auth = 'https://slurmtron.tau.ac.il'
generate_token_url = f"{base_url_auth}/slurmapi/generate-token/"

# User credentials
current_user = "user"
api_key = "token"

def get_api_token(url, username, api_key):
    """
    Retrieves an API token for SLURM REST API access.

    Parameters:
    url (str): The URL endpoint for obtaining the API token.
    username (str): The username of the user requesting the token.
    api_key (str): The API key provided by the HPC team.

    Returns:
    str: The API token if the request is successful.

    Raises:
    Exception: If the request fails with a non-200 status code.
    """
    payload = {
        'username': username,
        'api_key': api_key
    }
    response = requests.post(url, data=payload)

    if response.status_code == 200:
        return response.json()  # Assuming the token is returned in JSON format
    else:
        raise Exception(f"Error: {response.status_code}, {response.text}")

# Base URL for job submission
base_url = "https://slurmtron.tau.ac.il/slurmrestd"

# Job submission URL
job_url = f'{base_url}/slurm/v0.0.40/job/submit'

# Authorization headers with the obtained token
headers = {
    'X-SLURM-USER-NAME': current_user,
    'X-SLURM-USER-TOKEN': get_api_token(generate_token_url, current_user, api_key)['SlurmJWT']
}

# Job submission request
jobs_request = requests.post(
    job_url,
    headers=headers,
    json={
        # Example job script
        "script": "#!/bin/bash\n\n"
                  "srun hostname\n"
                  "echo \"hello world444\"\n"
                  "sleep 30",
        "job": {
            "partition": "< queue/partition_name >",
            "tasks": 1,
            "name": "< job_name> ",
            "account": "< account_name >",
            "nodes": "1",
            "cpus_per_task": < cpu_number >,
            "memory_per_node": {
                "number": <ram in MB >,
                "set": True,
                "infinite": False
            },
            # Full path to your error/output file.
            "standard_output": "/path/to/your/output.txt",
            "standard_error": "/path/to/your/error.txt",
            "current_working_directory": "/tmp/",
            # Environment modules (module load) should not be used directly under the script parameter. Instead, set all necessary environment variables under the environment parameter.
            "environment": [
                "PATH=/bin:/usr/bin/:/usr/local/bin/",
                "LD_LIBRARY_PATH=/lib/:/lib64/:/usr/local/lib"
            ],
        },
    }
)

# Processing the job submission result
jobs_result = jobs_request.json()['result']
for key, value in jobs_result.items():
    print(key, value)

Important Notes

• The script parameter in the job submission request is an example. Customize this script to fit your specific job requirements.
• Use full and appropriate paths for "standard_output" and "standard_error". Replace the placeholders with actual paths where you want the output and error files to be stored.
• Environment modules (module load) should not be used directly under the script parameter. Instead, set all necessary environment variables under the environment parameter.

Security and Best Practices

• Securely handle your API key and other sensitive information.
• Regularly review and update your scripts to align with updates in the SLURM REST API.

More Examples:

https://docs.lxp.lu/cloud/slurmrestd/