HPC Guide - User contributions [en]

Submitting a job to a slurm queue

2025-10-23T06:20:08Z

Shosh:

== Accessing the System ==

To submit jobs to SLURM at Tel Aviv University, you need to access the system through one of the following login nodes:

* slurmlogin.tau.ac.il

=== Requirements for Access ===

* '''Group Membership''': You must be part of the "power" group to access the resources.
* '''University Credentials''': Use your Tel Aviv University username and password to log in.

These login nodes are your starting point for submitting jobs, checking job status, and managing your SLURM tasks.

=== SSH Example ===

To access the system using SSH, use the following example:

<syntaxhighlight lang="bash">
# Replace 'your_username' with your actual Tel Aviv University username
ssh your_username@slurmlogin.tau.ac.il
</syntaxhighlight>

Your connection will be automatically routed to one of the login nodes:
powerslurm-login, powerslurm-login2, or powerslurm-login3.

If you have an SSH key set up for password-less login, you can specify it like this:

<syntaxhighlight lang="bash">
# Replace 'your_username' and '/path/to/your/private_key' accordingly
ssh -i /path/to/your/private_key your_username@slurmlogin.tau.ac.il
</syntaxhighlight>

== Environment Modules ==

Environment Modules in SLURM allow users to dynamically modify their shell environment, providing an easy way to load and unload different software applications, libraries, and their dependencies. This system helps avoid conflicts between software versions and ensures the correct environment for running specific applications.

Here are some common commands to work with environment modules:<syntaxhighlight lang="bash">
#List Available Modules: To see all the modules available on the system, use:
module avail

#To search for a specific module by name (e.g., `gcc`), use:
module avail gcc/gcc-12.1.0

#Get Detailed Information About a Module: The `module spider` command provides detailed information about a module, including versions, dependencies, and descriptions:
module spider gcc/gcc-12.1.0

#View Module Settings: To see what environment variables and settings will be modified by a module, use:
module show gcc/gcc-12.1.0

#Load a Module: To set up the environment for a specific software, use the `module load` command. For example, to load GCC version 12.1.0:
module load gcc/gcc-12.1.0

#List Loaded Modules: To view all currently loaded modules in your session, use:
module list

#Unload a Module: To unload a specific module from your environment, use:
module unload gcc/gcc-12.1.0

#Unload All Modules:** If you need to clear your environment of all loaded modules, use:
module purge

</syntaxhighlight>By using these commands, you can easily manage the software environments needed for different tasks, ensuring compatibility and reducing potential conflicts between software versions.

== Basic Job Submission Commands ==

=== Finding Your Account and Partition ===

Before submitting a job, you need to know which partitions you have permission to use.

Run the command `<code>check_my_partitions</code>` to view a list of all the partitions you have permission to send jobs to.

== Submitting Jobs==
sbatch: Submits a job script for batch processing.

'''Example''':<syntaxhighlight lang="bash">

sbatch --ntasks=1 --time=10 -p power-general-shared-pool -A public-users_v2 --qos=public pre_process.bash
# This command submits pre_process.bash to the power-general partition for 10 minutes.
# With 1 GPU:
sbatch --gres=gpu:1 -p gpu-general-pool -A public-users_v2 --qos=public gpu_job.sh

</syntaxhighlight>

=== Submitting Multiple Jobs ===

If you need to submit many similar jobs (hundreds or more), you should use a **Slurm job array**. Submitting each job individually using separate `sbatch` commands places a heavy load on the scheduler, slowing down job processing across the cluster. Job arrays allow you to bundle many related jobs together as a single submission. This is more efficient and easier to manage.

Each task in the array runs independently like a separate job, but the array is submitted as a single job ID for scheduling and tracking purposes.
You can customize the behavior of each task using the environment variable <code>SLURM_ARRAY_TASK_ID</code>.

==== Script Example: Job Array ====

This script submits a job array with 100 tasks, each processing a different input file. The array reduces scheduler load and simplifies job tracking.

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --job-name=array_job # Job name
#SBATCH --account=public-users_v2 # Account name
#SBATCH --partition=power-general-shared-pool # Partition name
#SBATCH --qos=public # qos type
#SBATCH --time=02:00:00 # Max run time (hh:mm:ss)
#SBATCH --ntasks=1 # Number of tasks per array job
#SBATCH --nodes=1 # Number of nodes
#SBATCH --cpus-per-task=1 # CPUs per task
#SBATCH --mem-per-cpu=4G # Memory per CPU
#SBATCH --array=1-100 # Array range: 100 tasks
#SBATCH --output=array_job_%A_%a.out # Output file: Job ID and array task ID
#SBATCH --error=array_job_%A_%a.err # Error file: Job ID and array task ID

echo "Starting SLURM array task"
echo "Job ID: $SLURM_JOB_ID"
echo "Array Task ID: $SLURM_ARRAY_TASK_ID"
echo "Running on node(s): $SLURM_JOB_NODELIST"
echo "Allocated CPUs: $SLURM_JOB_CPUS_PER_NODE"

# Your application commands go here
# You can use $SLURM_ARRAY_TASK_ID to customize behavior per task
# ./my_program input_${SLURM_ARRAY_TASK_ID}.txt
echo "Task completed"
</syntaxhighlight>

In this example:
* The job array consists of 100 tasks.
* Each task runs the same script but with a different input file.
* You access the task ID using the environment variable <code>SLURM_ARRAY_TASK_ID</code>.
* The output and error logs are separated per task using <code>%A</code> (job ID) and <code>%a</code> (array task ID).

==== Script Example: Job Array with different parameters per task ====

This script submits a job array with 3 tasks. Each task runs the same program with a different input file: `data1.txt`, `data2.txt`, and `data3.txt`.

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --job-name=array_job # Job name
#SBATCH --account=public-users_v2 # Account name
#SBATCH --partition=power-general-shared-pool # Partition name
#SBATCH --qos=public # qos type
#SBATCH --time=01:00:00 # Max run time (hh:mm:ss)
#SBATCH --ntasks=1 # Number of tasks per array job
#SBATCH --nodes=1 # Number of nodes
#SBATCH --cpus-per-task=1 # CPUs per task
#SBATCH --mem-per-cpu=2G # Memory per CPU
#SBATCH --array=1-3 # Run 3 tasks with IDs 1, 2, 3
#SBATCH --output=array_%A_%a.out # Output file: Job ID and task ID
#SBATCH --error=array_%A_%a.err # Error file: Job ID and task ID

echo "Starting SLURM array task"
echo "Job ID: $SLURM_JOB_ID"
echo "Array Task ID: $SLURM_ARRAY_TASK_ID"

# Each task runs the program with a different input file
./my_program data${SLURM_ARRAY_TASK_ID}.txt

echo "Task completed"
</syntaxhighlight>
===Writing Single SLURM Job Scripts===

Here is a simple job script example:

==== Basic Script====
<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --job-name=my_job # Job name
#SBATCH --account=public-users_v2 # Account name
#SBATCH --partition=power-general-shared-pool # Partition name
#SBATCH --qos=public # qos type
#SBATCH --time=02:00:00 # Max run time (hh:mm:ss)
#SBATCH --ntasks=1 # Number of tasks
#SBATCH --nodes=1 # Number of nodes
#SBATCH --cpus-per-task=1 # CPUs per task
#SBATCH --mem-per-cpu=4G # Memory per CPU
#SBATCH --output=my_job_%j.out # Output file
#SBATCH --error=my_job_%j.err # Error file
#SBATCH --mail-user=<your email> # Your mail address to receive an email
#SBATCH --mail-type=END,FAIL # The mail will be sent upon ending the script successfully or not

echo "Starting my SLURM job"
echo "Job ID: $SLURM_JOB_ID"
echo "Running on nodes: $SLURM_JOB_NODELIST"
echo "Allocated CPUs: $SLURM_JOB_CPUS_PER_NODE"

# Your application commands go here
# ./my_program

echo "Job completed"
</syntaxhighlight>

To ask for x cores interactively:
<pre>
srun --ntasks=1 --cpus-per-task=x --partition=power-general-public-pool --account=public-users_v2 --qos=public --nodes=1 --pty bash
</pre>

However, need for now also to set slurm parameters inside the script, or within the interactive job:
<pre>
export SLURM_TASKS_PER_NODE=48
export SLURM_CPUS_ON_NODE=48
</pre>

==== Script for 1 GPU ====
<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --job-name=gpu_job # Job name
#SBATCH --account=my_account # Account name
#SBATCH --partition=gpu-general-pool # Partition name
#SBATCH --qos=my_qos # qos type
#SBATCH --time=02:00:00 # Max run time
#SBATCH --ntasks=1 # Number of tasks
#SBATCH --nodes=1 # Number of nodes
#SBATCH --cpus-per-task=1 # CPUs per task
#SBATCH --gres=gpu:1 # Number of GPUs
#SBATCH --mem-per-cpu=4G # Memory per CPU
#SBATCH --output=my_job_%j.out # Output file
#SBATCH --error=my_job_%j.err # Error file

module load python/python-3.8

echo "Starting GPU job"
echo "Job ID: $SLURM_JOB_ID"
echo "Running on nodes: $SLURM_JOB_NODELIST"
echo "Allocated CPUs: $SLURM_JOB_CPUS_PER_NODE"

# Your GPU commands go here

echo "Job completed"
</syntaxhighlight>
For excluding a node, one may add the following
<syntaxhighlight>
#SBATCH --exclude=compute-0-[100-103],compute-0-67
</syntaxhighlight>

===Importance of Correct RAM Usage in Jobs===

When writing SLURM job scripts, it's crucial to understand and correctly specify the memory requirements for your job.

Proper memory allocation ensures efficient resource usage and prevents job failures due to out-of-memory (OOM) errors.

==== Why Correct RAM Usage Matters ====

* '''Resource Efficiency''': Allocating the right amount of memory helps in optimal resource utilization, allowing more jobs to run simultaneously on the cluster.
* '''Job Stability''': Underestimating memory requirements can lead to OOM errors, causing your job to fail and waste computational resources.
* '''Performance''': Overestimating memory needs can lead to underutilization of resources, potentially delaying other jobs in the queue.

==== How to Specify Memory in SLURM ====

* '''--mem''': Specifies the total memory required for the job.
* '''--mem-per-cpu''': Specifies the memory required per CPU.

'''Example''':<syntaxhighlight lang="bash">
#SBATCH --mem=4G # Total memory for the job
#SBATCH --mem-per-cpu=2G # Memory per CPU
</syntaxhighlight>

===Interactive Jobs===
<syntaxhighlight lang="bash">
#Start an interactive session:
srun --ntasks=1 -p power-general-shared-pool -A public-users_v2 --qos=public --pty bash

#Specify a compute node:
srun --ntasks=1 -p power-general-shared-pool -A public-users_v2 --qos=public --nodelist="compute-0-12" --pty bash

#Using GUI:
srun --ntasks=1 -p power-general-shared-pool -A public-users_v2 --qos=public --x11 /bin/bash

</syntaxhighlight>

=== Submitting RELION Jobs===

To submit a RELION job interactively on the <code>gpu-relion</code> queue with X11 forwarding, use the following steps:<syntaxhighlight lang="bash">
#Start an interactive session with X11:
srun --ntasks=1 -p gpu-relion-pool -A gpu-relion-users_v2 --qos=owner --x11 --pty bash
#Load the RELION module:
module load relion/relion-4.0.1
#Launch RELION:
relion

</syntaxhighlight>

==Running matlab example==
In this example there are 3 files:

myTable.m ⇒ This matlab file calculates something
<pre>
fprintf('=======================================\n');
fprintf(' a b c d \n');
fprintf('=======================================\n');
while 1
for j = 1:10
a = sin(10*j);
b = a*cos(10*j);
c = a + b;
d = a - b;
fprintf('%+6.5f %+6.5f %+6.5f %+6.5f \n',a,b,c,d);
end
end
fprintf('=======================================\n');
</pre>

my_table_script.sh ⇒ This script executes the matlab program. Need just to run qsub with this script
<pre>
#!/bin/bash

#SBATCH --mem=50mg
#SBATCH --partition power-general-shared-pool
#SBATCH -A public-users_v2
hostname

cd /a/home/cc/tree/taucc/staff/dvory/matlab

matlab -nodisplay -nosplash -nodesktop -r "run(myTable());exit;"
</pre>

run_in_loop.sh ⇒ However, one may also generate many jobs with this file
<pre>
#!/bin/bash

for i in {1..100}

do

sbatch my_table_script.sh

done
</pre>

Running my job is with the command (after doing chmod +x 'run_in_loop.sh'):
<pre>
./run_in_loop.sh
</pre>

==AlphaFold==

AlphaFold is a deep learning tool designed for predicting protein structures.

'''Guides:'''

[https://hpcguide.tau.ac.il/index.php?title=Alphafold AlphaFold Guide]

[https://hpcguide.tau.ac.il/index.php?title=Alphafold3 AlphaFold3 Guide]

==Common SLURM Commands==
<syntaxhighlight lang="bash">
#View all queues (partitions):
sinfo
#View all jobs:
squeue
#View details of a specific job:
scontrol show job <job_number>
#Get information about partitions:
scontrol show partition
</syntaxhighlight>

== Troubleshooting & Tips ==

=== Common Errors ===

# <code>srun: error: Unable to allocate resources: No partition specified or system default partition</code> <br />'''Solution:''' Always specify a partition. Example: <code>srun --pty -c 1 --mem=2G -p power-general /bin/bash</code>
# Job failed, and upon doing scontrol show job job_id or when running sacct -j job_id -o JobID,JobName,State%20 <br />you see: <code>JobState=OUT_OF_MEMORY Reason=OutOfMemory</code> or :<syntaxhighlight lang="bash">
JobID JobName State
------------ ---------- --------------------
71 oom_test OUT_OF_MEMORY
71.batch batch OUT_OF_MEMORY
71.extern extern COMPLETED

</syntaxhighlight>it means that the ram requested for the job was not enough, please resubmit the job again with more ram. see [https://wikihpc.tau.ac.il/index.php?title=Slurm_user_guide#Estimating_RAM_Usage below] for help with understanding how much ram your job may need.

=== Chain Jobs ===
Use the <code>--depend</code> flag to set job dependencies.

'''Example:'''
<syntaxhighlight lang="bash">
sbatch --ntasks=1 --time=60 -p power-general -A power-general-users --depend=45001 do_work.bash
</syntaxhighlight>

=== Always Specify Resources ===
When submitting jobs, ensure you include all required resources like partition, memory, and CPUs to avoid job failures.

=== Attaching to Running Jobs ===
If you need to monitor or interact with a running job, use <code>sattach</code>. This command allows you to attach to a job's input, output, and error streams in real-time.

'''Example:'''
<syntaxhighlight lang="bash">
sattach <job_id>
</syntaxhighlight>

To view job steps of a specific job, use the following command:

<syntaxhighlight lang="bash">
scontrol show job <job_id>
</syntaxhighlight>

Look for sections labeled "StepId" within the output.

'''For specific job steps, use:'''
<syntaxhighlight lang="bash">
sattach <job_id.step_id>
</syntaxhighlight>

'''Note:''' <code>sattach</code> is particularly useful for interactive jobs, where you can provide input directly. For non-interactive jobs, it acts like <code>tail -f</code>, allowing you to monitor the output stream.

=== Estimating RAM Usage ===

When writing SLURM job scripts, it's crucial to understand and correctly specify the memory requirements for your job. Proper memory allocation ensures efficient resource usage and prevents job failures due to out-of-memory (OOM) errors.

==== Tips for Estimating RAM Usage ====

* Check Application Documentation: Refer to the official documentation or user guides for memory-related information.
* Run a Small Test Job: Submit a smaller version of your job and monitor its memory usage using commands like `free -m`, `top`, or `htop`.
* Use Profiling Tools: Tools like `valgrind`, `gprof`, or built-in profilers can help you understand memory usage.
* Analyze Previous Jobs: Review SLURM logs and job statistics for insights into memory consumption of past jobs.
* Consult with Peers or Experts: Ask colleagues or experts who have experience with similar workloads.

==== Example: Monitoring Memory Usage ====
<syntaxhighlight lang="bash">
#!/bin/bash

#SBATCH --job-name=memory_test
#SBATCH --account=your_account
#SBATCH --partition=your_partition
#SBATCH --qos=your_qos
#SBATCH --time=01:00:00
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH --mem=4G
#SBATCH --output=memory_test.out
#SBATCH --error=memory_test.err

# Monitor memory usage
echo "Memory usage before running the job:"
free -m

# Your application commands go here
# ./your_application

# Monitor memory usage after running the job
echo "Memory usage after running the job:"
free -m
</syntaxhighlight>

==== General Tips ====

* Start Small: Begin with a conservative memory request and increase it based on observed usage.
* Consider Peak Usage: Plan for peak memory usage to avoid OOM errors.
* Use SLURM's Memory Reporting: Use `sacct` to view memory usage statistics.

'''Example:'''
<syntaxhighlight lang="bash">
sacct -j <job_id> --format=JobID,JobName,MaxRSS,Elapsed
</syntaxhighlight>

Submitting a job to a slurm queue

2025-10-23T06:18:53Z

Shosh:

== Accessing the System ==

To submit jobs to SLURM at Tel Aviv University, you need to access the system through one of the following login nodes:

* slurmlogin.tau.ac.il

=== Requirements for Access ===

* '''Group Membership''': You must be part of the "power" group to access the resources.
* '''University Credentials''': Use your Tel Aviv University username and password to log in.

These login nodes are your starting point for submitting jobs, checking job status, and managing your SLURM tasks.

=== SSH Example ===

To access the system using SSH, use the following example:

<syntaxhighlight lang="bash">
# Replace 'your_username' with your actual Tel Aviv University username
ssh your_username@slurmlogin.tau.ac.il
</syntaxhighlight>

Your connection will be automatically routed to one of the login nodes:
powerslurm-login, powerslurm-login2, or powerslurm-login3.

If you have an SSH key set up for password-less login, you can specify it like this:

<syntaxhighlight lang="bash">
# Replace 'your_username' and '/path/to/your/private_key' accordingly
ssh -i /path/to/your/private_key your_username@slurmlogin.tau.ac.il
</syntaxhighlight>

== Environment Modules ==

Environment Modules in SLURM allow users to dynamically modify their shell environment, providing an easy way to load and unload different software applications, libraries, and their dependencies. This system helps avoid conflicts between software versions and ensures the correct environment for running specific applications.

Here are some common commands to work with environment modules:<syntaxhighlight lang="bash">
#List Available Modules: To see all the modules available on the system, use:
module avail

#To search for a specific module by name (e.g., `gcc`), use:
module avail gcc/gcc-12.1.0

#Get Detailed Information About a Module: The `module spider` command provides detailed information about a module, including versions, dependencies, and descriptions:
module spider gcc/gcc-12.1.0

#View Module Settings: To see what environment variables and settings will be modified by a module, use:
module show gcc/gcc-12.1.0

#Load a Module: To set up the environment for a specific software, use the `module load` command. For example, to load GCC version 12.1.0:
module load gcc/gcc-12.1.0

#List Loaded Modules: To view all currently loaded modules in your session, use:
module list

#Unload a Module: To unload a specific module from your environment, use:
module unload gcc/gcc-12.1.0

#Unload All Modules:** If you need to clear your environment of all loaded modules, use:
module purge

</syntaxhighlight>By using these commands, you can easily manage the software environments needed for different tasks, ensuring compatibility and reducing potential conflicts between software versions.

== Basic Job Submission Commands ==

=== Finding Your Account and Partition ===

Before submitting a job, you need to know which partitions you have permission to use.

Run the command `<code>check_my_partitions</code>` to view a list of all the partitions you have permission to send jobs to.

== Submitting Jobs==
sbatch: Submits a job script for batch processing.

'''Example''':<syntaxhighlight lang="bash">

sbatch --ntasks=1 --time=10 -p power-general -A power-general-users pre_process.bash
# This command submits pre_process.bash to the power-general partition for 10 minutes.
# With 1 GPU:
sbatch --gres=gpu:1 -p gpu-general -A gpu-general-users gpu_job.sh

</syntaxhighlight>

=== Submitting Multiple Jobs ===

If you need to submit many similar jobs (hundreds or more), you should use a **Slurm job array**. Submitting each job individually using separate `sbatch` commands places a heavy load on the scheduler, slowing down job processing across the cluster. Job arrays allow you to bundle many related jobs together as a single submission. This is more efficient and easier to manage.

Each task in the array runs independently like a separate job, but the array is submitted as a single job ID for scheduling and tracking purposes.
You can customize the behavior of each task using the environment variable <code>SLURM_ARRAY_TASK_ID</code>.

==== Script Example: Job Array ====

This script submits a job array with 100 tasks, each processing a different input file. The array reduces scheduler load and simplifies job tracking.

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --job-name=array_job # Job name
#SBATCH --account=public-users_v2 # Account name
#SBATCH --partition=power-general-shared-pool # Partition name
#SBATCH --qos=public # qos type
#SBATCH --time=02:00:00 # Max run time (hh:mm:ss)
#SBATCH --ntasks=1 # Number of tasks per array job
#SBATCH --nodes=1 # Number of nodes
#SBATCH --cpus-per-task=1 # CPUs per task
#SBATCH --mem-per-cpu=4G # Memory per CPU
#SBATCH --array=1-100 # Array range: 100 tasks
#SBATCH --output=array_job_%A_%a.out # Output file: Job ID and array task ID
#SBATCH --error=array_job_%A_%a.err # Error file: Job ID and array task ID

echo "Starting SLURM array task"
echo "Job ID: $SLURM_JOB_ID"
echo "Array Task ID: $SLURM_ARRAY_TASK_ID"
echo "Running on node(s): $SLURM_JOB_NODELIST"
echo "Allocated CPUs: $SLURM_JOB_CPUS_PER_NODE"

# Your application commands go here
# You can use $SLURM_ARRAY_TASK_ID to customize behavior per task
# ./my_program input_${SLURM_ARRAY_TASK_ID}.txt
echo "Task completed"
</syntaxhighlight>

In this example:
* The job array consists of 100 tasks.
* Each task runs the same script but with a different input file.
* You access the task ID using the environment variable <code>SLURM_ARRAY_TASK_ID</code>.
* The output and error logs are separated per task using <code>%A</code> (job ID) and <code>%a</code> (array task ID).

==== Script Example: Job Array with different parameters per task ====

This script submits a job array with 3 tasks. Each task runs the same program with a different input file: `data1.txt`, `data2.txt`, and `data3.txt`.

<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --job-name=array_job # Job name
#SBATCH --account=public-users_v2 # Account name
#SBATCH --partition=power-general-shared-pool # Partition name
#SBATCH --qos=public # qos type
#SBATCH --time=01:00:00 # Max run time (hh:mm:ss)
#SBATCH --ntasks=1 # Number of tasks per array job
#SBATCH --nodes=1 # Number of nodes
#SBATCH --cpus-per-task=1 # CPUs per task
#SBATCH --mem-per-cpu=2G # Memory per CPU
#SBATCH --array=1-3 # Run 3 tasks with IDs 1, 2, 3
#SBATCH --output=array_%A_%a.out # Output file: Job ID and task ID
#SBATCH --error=array_%A_%a.err # Error file: Job ID and task ID

echo "Starting SLURM array task"
echo "Job ID: $SLURM_JOB_ID"
echo "Array Task ID: $SLURM_ARRAY_TASK_ID"

# Each task runs the program with a different input file
./my_program data${SLURM_ARRAY_TASK_ID}.txt

echo "Task completed"
</syntaxhighlight>
===Writing Single SLURM Job Scripts===

Here is a simple job script example:

==== Basic Script====
<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --job-name=my_job # Job name
#SBATCH --account=public-users_v2 # Account name
#SBATCH --partition=power-general-shared-pool # Partition name
#SBATCH --qos=public # qos type
#SBATCH --time=02:00:00 # Max run time (hh:mm:ss)
#SBATCH --ntasks=1 # Number of tasks
#SBATCH --nodes=1 # Number of nodes
#SBATCH --cpus-per-task=1 # CPUs per task
#SBATCH --mem-per-cpu=4G # Memory per CPU
#SBATCH --output=my_job_%j.out # Output file
#SBATCH --error=my_job_%j.err # Error file
#SBATCH --mail-user=<your email> # Your mail address to receive an email
#SBATCH --mail-type=END,FAIL # The mail will be sent upon ending the script successfully or not

echo "Starting my SLURM job"
echo "Job ID: $SLURM_JOB_ID"
echo "Running on nodes: $SLURM_JOB_NODELIST"
echo "Allocated CPUs: $SLURM_JOB_CPUS_PER_NODE"

# Your application commands go here
# ./my_program

echo "Job completed"
</syntaxhighlight>

To ask for x cores interactively:
<pre>
srun --ntasks=1 --cpus-per-task=x --partition=power-general-public-pool --account=public-users_v2 --qos=public --nodes=1 --pty bash
</pre>

However, need for now also to set slurm parameters inside the script, or within the interactive job:
<pre>
export SLURM_TASKS_PER_NODE=48
export SLURM_CPUS_ON_NODE=48
</pre>

==== Script for 1 GPU ====
<syntaxhighlight lang="bash">
#!/bin/bash
#SBATCH --job-name=gpu_job # Job name
#SBATCH --account=my_account # Account name
#SBATCH --partition=gpu-general-pool # Partition name
#SBATCH --qos=my_qos # qos type
#SBATCH --time=02:00:00 # Max run time
#SBATCH --ntasks=1 # Number of tasks
#SBATCH --nodes=1 # Number of nodes
#SBATCH --cpus-per-task=1 # CPUs per task
#SBATCH --gres=gpu:1 # Number of GPUs
#SBATCH --mem-per-cpu=4G # Memory per CPU
#SBATCH --output=my_job_%j.out # Output file
#SBATCH --error=my_job_%j.err # Error file

module load python/python-3.8

echo "Starting GPU job"
echo "Job ID: $SLURM_JOB_ID"
echo "Running on nodes: $SLURM_JOB_NODELIST"
echo "Allocated CPUs: $SLURM_JOB_CPUS_PER_NODE"

# Your GPU commands go here

echo "Job completed"
</syntaxhighlight>
For excluding a node, one may add the following
<syntaxhighlight>
#SBATCH --exclude=compute-0-[100-103],compute-0-67
</syntaxhighlight>

===Importance of Correct RAM Usage in Jobs===

When writing SLURM job scripts, it's crucial to understand and correctly specify the memory requirements for your job.

Proper memory allocation ensures efficient resource usage and prevents job failures due to out-of-memory (OOM) errors.

==== Why Correct RAM Usage Matters ====

* '''Resource Efficiency''': Allocating the right amount of memory helps in optimal resource utilization, allowing more jobs to run simultaneously on the cluster.
* '''Job Stability''': Underestimating memory requirements can lead to OOM errors, causing your job to fail and waste computational resources.
* '''Performance''': Overestimating memory needs can lead to underutilization of resources, potentially delaying other jobs in the queue.

==== How to Specify Memory in SLURM ====

* '''--mem''': Specifies the total memory required for the job.
* '''--mem-per-cpu''': Specifies the memory required per CPU.

'''Example''':<syntaxhighlight lang="bash">
#SBATCH --mem=4G # Total memory for the job
#SBATCH --mem-per-cpu=2G # Memory per CPU
</syntaxhighlight>

===Interactive Jobs===
<syntaxhighlight lang="bash">
#Start an interactive session:
srun --ntasks=1 -p power-general-shared-pool -A public-users_v2 --qos=public --pty bash

#Specify a compute node:
srun --ntasks=1 -p power-general-shared-pool -A public-users_v2 --qos=public --nodelist="compute-0-12" --pty bash

#Using GUI:
srun --ntasks=1 -p power-general-shared-pool -A public-users_v2 --qos=public --x11 /bin/bash

</syntaxhighlight>

=== Submitting RELION Jobs===

To submit a RELION job interactively on the <code>gpu-relion</code> queue with X11 forwarding, use the following steps:<syntaxhighlight lang="bash">
#Start an interactive session with X11:
srun --ntasks=1 -p gpu-relion-pool -A gpu-relion-users_v2 --qos=owner --x11 --pty bash
#Load the RELION module:
module load relion/relion-4.0.1
#Launch RELION:
relion

</syntaxhighlight>

==Running matlab example==
In this example there are 3 files:

myTable.m ⇒ This matlab file calculates something
<pre>
fprintf('=======================================\n');
fprintf(' a b c d \n');
fprintf('=======================================\n');
while 1
for j = 1:10
a = sin(10*j);
b = a*cos(10*j);
c = a + b;
d = a - b;
fprintf('%+6.5f %+6.5f %+6.5f %+6.5f \n',a,b,c,d);
end
end
fprintf('=======================================\n');
</pre>

my_table_script.sh ⇒ This script executes the matlab program. Need just to run qsub with this script
<pre>
#!/bin/bash

#SBATCH --mem=50mg
#SBATCH --partition power-general-shared-pool
#SBATCH -A public-users_v2
hostname

cd /a/home/cc/tree/taucc/staff/dvory/matlab

matlab -nodisplay -nosplash -nodesktop -r "run(myTable());exit;"
</pre>

run_in_loop.sh ⇒ However, one may also generate many jobs with this file
<pre>
#!/bin/bash

for i in {1..100}

do

sbatch my_table_script.sh

done
</pre>

Running my job is with the command (after doing chmod +x 'run_in_loop.sh'):
<pre>
./run_in_loop.sh
</pre>

==AlphaFold==

AlphaFold is a deep learning tool designed for predicting protein structures.

'''Guides:'''

[https://hpcguide.tau.ac.il/index.php?title=Alphafold AlphaFold Guide]

[https://hpcguide.tau.ac.il/index.php?title=Alphafold3 AlphaFold3 Guide]

==Common SLURM Commands==
<syntaxhighlight lang="bash">
#View all queues (partitions):
sinfo
#View all jobs:
squeue
#View details of a specific job:
scontrol show job <job_number>
#Get information about partitions:
scontrol show partition
</syntaxhighlight>

== Troubleshooting & Tips ==

=== Common Errors ===

# <code>srun: error: Unable to allocate resources: No partition specified or system default partition</code> <br />'''Solution:''' Always specify a partition. Example: <code>srun --pty -c 1 --mem=2G -p power-general /bin/bash</code>
# Job failed, and upon doing scontrol show job job_id or when running sacct -j job_id -o JobID,JobName,State%20 <br />you see: <code>JobState=OUT_OF_MEMORY Reason=OutOfMemory</code> or :<syntaxhighlight lang="bash">
JobID JobName State
------------ ---------- --------------------
71 oom_test OUT_OF_MEMORY
71.batch batch OUT_OF_MEMORY
71.extern extern COMPLETED

</syntaxhighlight>it means that the ram requested for the job was not enough, please resubmit the job again with more ram. see [https://wikihpc.tau.ac.il/index.php?title=Slurm_user_guide#Estimating_RAM_Usage below] for help with understanding how much ram your job may need.

=== Chain Jobs ===
Use the <code>--depend</code> flag to set job dependencies.

'''Example:'''
<syntaxhighlight lang="bash">
sbatch --ntasks=1 --time=60 -p power-general -A power-general-users --depend=45001 do_work.bash
</syntaxhighlight>

=== Always Specify Resources ===
When submitting jobs, ensure you include all required resources like partition, memory, and CPUs to avoid job failures.

=== Attaching to Running Jobs ===
If you need to monitor or interact with a running job, use <code>sattach</code>. This command allows you to attach to a job's input, output, and error streams in real-time.

'''Example:'''
<syntaxhighlight lang="bash">
sattach <job_id>
</syntaxhighlight>

To view job steps of a specific job, use the following command:

<syntaxhighlight lang="bash">
scontrol show job <job_id>
</syntaxhighlight>

Look for sections labeled "StepId" within the output.

'''For specific job steps, use:'''
<syntaxhighlight lang="bash">
sattach <job_id.step_id>
</syntaxhighlight>

'''Note:''' <code>sattach</code> is particularly useful for interactive jobs, where you can provide input directly. For non-interactive jobs, it acts like <code>tail -f</code>, allowing you to monitor the output stream.

=== Estimating RAM Usage ===

When writing SLURM job scripts, it's crucial to understand and correctly specify the memory requirements for your job. Proper memory allocation ensures efficient resource usage and prevents job failures due to out-of-memory (OOM) errors.

==== Tips for Estimating RAM Usage ====

* Check Application Documentation: Refer to the official documentation or user guides for memory-related information.
* Run a Small Test Job: Submit a smaller version of your job and monitor its memory usage using commands like `free -m`, `top`, or `htop`.
* Use Profiling Tools: Tools like `valgrind`, `gprof`, or built-in profilers can help you understand memory usage.
* Analyze Previous Jobs: Review SLURM logs and job statistics for insights into memory consumption of past jobs.
* Consult with Peers or Experts: Ask colleagues or experts who have experience with similar workloads.

==== Example: Monitoring Memory Usage ====
<syntaxhighlight lang="bash">
#!/bin/bash

#SBATCH --job-name=memory_test
#SBATCH --account=your_account
#SBATCH --partition=your_partition
#SBATCH --qos=your_qos
#SBATCH --time=01:00:00
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH --mem=4G
#SBATCH --output=memory_test.out
#SBATCH --error=memory_test.err

# Monitor memory usage
echo "Memory usage before running the job:"
free -m

# Your application commands go here
# ./your_application

# Monitor memory usage after running the job
echo "Memory usage after running the job:"
free -m
</syntaxhighlight>

==== General Tips ====

* Start Small: Begin with a conservative memory request and increase it based on observed usage.
* Consider Peak Usage: Plan for peak memory usage to avoid OOM errors.
* Use SLURM's Memory Reporting: Use `sacct` to view memory usage statistics.

'''Example:'''
<syntaxhighlight lang="bash">
sacct -j <job_id> --format=JobID,JobName,MaxRSS,Elapsed
</syntaxhighlight>

Security installations

2024-09-26T12:25:59Z

Shosh:

== NAC - Forescout security software for Linux ==

Download and install the Forescout client:

[http://hpcguide.tau.ac.il/nac/ForeScoutSecureConnector_64_visible_daemon.tar.gz ForeScoutSecureConnector_64_visible_daemon.tar.gz]

== Installation ==
tar -zxvf ForeScoutSecureConnector_64_visible_daemon.tar.gz

cd secure_connector

./install.sh

systemctl start SecureConnector.service

== EDR - Falcon (for ubuntu 22 and ubuntu 24) ==

Download and install the Falcon client:

[http://hpcguide.tau.ac.il/edr/falcon-sensor_7.18.0-17106_amd64.deb falcon-sensor_7.18.0-17106_amd64.deb]

== Installation ==

dpkg -i falcon-sensor_7.18.0-17106_amd64.deb

systemctl restart falcon-sensor.service

Need to run below command to register the program (Only for new installation):

/opt/CrowdStrike/falconctl -s --cid=24E96F03F42840719A7F1DC3287E03B9-59

Security installations

2024-09-26T12:22:33Z

Shosh:

== NAC - Forescout security software for Linux ==

Download and install the Forescout client:

[http://hpcguide.tau.ac.il/nac/ForeScoutSecureConnector_64_visible_daemon.tar.gz ForeScoutSecureConnector_64_visible_daemon.tar.gz]

== Installation ==
tar -zxvf ForeScoutSecureConnector_64_visible_daemon.tar.gz

cd secure_connector

./install.sh

systemctl start SecureConnector.service

== EDR - Falcon (for ubuntu 22 and ubuntu 24) ==

Download and install the Falcon client:

[http://hpcguide.tau.ac.il/edr/falcon-sensor_7.18.0-17106_amd64.deb falcon-sensor_7.18.0-17106_amd64.deb]

== Installation ==

dpkg -i falcon-sensor_7.18.0-17106_amd64.deb

Now you need to run below command to register the program:

/opt/CrowdStrike/falconctl -s --cid=24E96F03F42840719A7F1DC3287E03B9-59

systemctl restart falcon-sensor.service

Security installations

2024-09-26T12:21:11Z

Shosh:

== NAC - Forescout security software for Linux ==

Download and install the Forescout client:

[http://hpcguide.tau.ac.il/nac/ForeScoutSecureConnector_64_visible_daemon.tar.gz ForeScoutSecureConnector_64_visible_daemon.tar.gz]

== Installation ==
tar -zxvf ForeScoutSecureConnector_64_visible_daemon.tar.gz

cd secure_connector

./install.sh

systemctl start SecureConnector.service

== EDR - Falcon (for ubuntu 22 and ubuntu 24) ==

Download and install the Falcon client:

[http://hpcguide.tau.ac.il/edr/falcon-sensor_7.18.0-17106_amd64.deb falcon-sensor_7.18.0-17106_amd64.deb]

== Installation ==

dpkg -i falcon-sensor_7.18.0-17106_amd64.deb

Now you need to run below command to register the program:

/opt/CrowdStrike/falconctl -s --cid=24E96F03F42840719A7F1DC3287E03B9-59

Security installations

2024-09-26T12:19:51Z

Shosh:

Security installations

2024-09-26T12:18:32Z

Shosh: Created page with "== NAC - Forescout security software for Linux == Download and install the Forescout client: [http://hpcguide.tau.ac.il/nac/ForeScoutSecureConnector_64_visible_daemon.tar..."

== NAC - Forescout security software for Linux ==

Download and install the Forescout client:

[http://hpcguide.tau.ac.il/nac/ForeScoutSecureConnector_64_visible_daemon.tar.gz ForeScoutSecureConnector_64_visible_daemon.tar.gz]

== Installation ==
tar -zxvf ForeScoutSecureConnector_64_visible_daemon.tar.gz

cd secure_connector

./install.sh

systemctl start SecureConnector.service

== EDR - Falcon ==

Download and install the Falcon client:

[http://hpcguide.tau.ac.il/edr/falcon-sensor_7.18.0-17106_amd64.deb falcon-sensor_7.18.0-17106_amd64.deb]

== Installation ==

dpkg -i falcon-sensor_7.18.0-17106_amd64.deb

NAC and EDR for Linux

2024-09-26T12:17:53Z

Shosh:

== NAC - Forescout security software for Linux ==

Download and install the Forescout client:

[http://hpcguide.tau.ac.il/nac/ForeScoutSecureConnector_64_visible_daemon.tar.gz ForeScoutSecureConnector_64_visible_daemon.tar.gz]

== Installation ==
tar -zxvf ForeScoutSecureConnector_64_visible_daemon.tar.gz

cd secure_connector

./install.sh

systemctl start SecureConnector.service

== EDR - Falcon ==

Download and install the Falcon client:

[http://hpcguide.tau.ac.il/edr/falcon-sensor_7.18.0-17106_amd64.deb falcon-sensor_7.18.0-17106_amd64.deb]

== Installation ==

dpkg -i falcon-sensor_7.18.0-17106_amd64.deb

NAC and EDR for Linux

2024-09-26T12:17:36Z

Shosh:

== NAC - Forescout security software for Linux ==

Download and install the Forescout client:

[http://hpcguide.tau.ac.il/nac/ForeScoutSecureConnector_64_visible_daemon.tar.gz ForeScoutSecureConnector_64_visible_daemon.tar.gz]

== Installation ==
tar -zxvf ForeScoutSecureConnector_64_visible_daemon.tar.gz

cd secure_connector

./install.sh

systemctl start SecureConnector.service

== EDR - Falcon ==

Download and install the Falcon client:

[http://hpcguide.tau.ac.il/edr/falcon-sensor_7.18.0-17106_amd64.deb falcon-sensor_7.18.0-17106_amd64.deb]

== Installation ==

dpkg -i falcon-sensor_7.18.0-17106_amd64.deb

NAC and EDR for Linux

2024-09-23T12:39:18Z

Shosh:

NAC and EDR for Linux

2024-09-23T12:38:57Z

Shosh:

NAC and EDR for Linux

2024-09-23T12:38:06Z

Shosh:

NAC and EDR for Linux

2024-09-23T12:37:11Z

Shosh:

NAC and EDR for Linux

2024-09-23T12:25:52Z

Shosh: Created page with "== NAC - Forescout security software for Linux == Download and install the Forescout client: hpcguide.tau.ac.il/nac/ForeScoutSecureConnector_64_visible_daemon.tar.gz|For..."

== NAC - Forescout security software for Linux ==

Download and install the Forescout client:
[[hpcguide.tau.ac.il/nac/ForeScoutSecureConnector_64_visible_daemon.tar.gz|ForeScoutSecureConnector_64_visible_daemon.tar.gz]]

Submitting a job to a slurm queue

2024-09-18T06:07:48Z

Shosh:

SLURM (Simple Linux Utility for Resource Management) is a job scheduler used on many high-performance computing systems. It manages and allocates resources such as compute nodes and controls job execution.

=== Accessing the System ===
To submit jobs to the SLURM scheduler at Tel Aviv University, you must access the system through one of the designated login nodes. These nodes act as the gateway for submitting and managing your SLURM jobs. The available login nodes are:

* <code>powerslurm-login.tau.ac.il</code>
* <code>powerslurm-login2.tau.ac.il</code>

==== Login Requirements: ====

# Membership in the "power" group: Ensure you are a part of the "power" group which grants the necessary permissions for accessing the HPC resources.
# University Credentials: Log in using your Tel Aviv University credentials. This ensures secure access and that your job submissions are appropriately accounted for under your user profile.

Remember, these login nodes are the initial point of contact for all your job management tasks, including job submission, monitoring, and other SLURM-related operations.

=== Basic Job Submission Commands ===
====Finding your account and partition====
In order to submit jobs to slurm, one needs to know the accounts and partitions she belongs to. Each account may belong to one or more partitions.

To see the account I belong to, please type:
<pre>
sacctmgr show associations where user=dvory format=Account%20
</pre>
If you know your partition, and would like to know which account you need to specify when using it, please do (on powerslurm-login)
<pre>
check_allowed_account -p <partition>
</pre>
For example:
<pre>
check_allowed_account -p power-general
</pre>

====Example for submitting jobs====
# sbatch: Submit a batch job script.
#* Example: <code>sbatch --ntasks=1 --time=10 -p power-general -A power-general-users pre_process.bash</code>
#* This submits <code>pre_process.bash</code> with 1 task for 10 minutes.
#* Example of chaining jobs: <code>sbatch --ntasks=128 --time=60 -p power-general -A power-general-users --depend=45001 do_work.bash</code>
#* Example with GPU: <code>sbatch --ntasks=1 --time=10 --gres=gpu:2 -p gpu-general -A gpu-general-users pre_process.bash</code> or <code>sbatch --gres=gpu:1 -p gpu-general -A gpu-general-users gpu_job.sh</code>
# srun: Submit an interactive job with MPI (Message Passing Interface), often called a "job step."
#* Example: <code>srun --ntasks=2 -p power-general -A power-general-users --label hostname</code>
#* With MPI: <code>srun -intasks=2 -p power-general -A power-general-users --label hostname</code>
# sattach: Attach stdin/out/err to an existing job or job step.

=== Interactive Job Examples ===
* Opening a bash shell: <code>srun --ntasks=56 -p power-general -A power-general-users --pty bash</code>
* Specifying compute nodes: <code>srun --ntasks=56 -p power-general -A power-general-users --nodelist="compute-0-12" --pty bash</code>
* Using GUI: <code>srun --ntasks=56 -p power-general -A power-general-users --x11 /bin/bash</code>

=== Script Examples: ===
<syntaxhighlight lang="bash">
#!/bin/bash

#SBATCH --job-name=my_job # Job name
#SBATCH --account=power-general-users # Account name for billing
#SBATCH --partition=power-general # Partition name
#SBATCH --time=02:00:00 # Time allotted for the job (hh:mm:ss)
#SBATCH --ntasks=4 # Number of tasks (processes)
#SBATCH --cpus-per-task=1 # Number of CPU cores per task
#SBATCH --mem-per-cpu=4G # Memory per CPU core
#SBATCH --output=my_job_%j.out # Standard output and error log (%j expands to jobId)
#SBATCH --error=my_job_%j.err # Separate file for standard error

# Load modules or software if required
# module load python/python-3.8

# Print some information about the job
echo "Starting my SLURM job"
echo "Job ID: $SLURM_JOB_ID"
echo "Running on nodes: $SLURM_JOB_NODELIST"
echo "Allocated CPUs: $SLURM_JOB_CPUS_PER_NODE"

# Run your application, this could be anything from a custom script to standard applications
# ./my_program
# python my_script.py

# End of script
echo "Job completed"

</syntaxhighlight>

==== Script example with GPU ====
<syntaxhighlight lang="bash">
#!/bin/bash

#SBATCH --job-name=my_job # Job name
#SBATCH --account=my_account # Account name for billing
#SBATCH --partition=long # Partition name
#SBATCH --time=02:00:00 # Time allotted for the job (hh:mm:ss)
#SBATCH --ntasks=4 # Number of tasks (processes)
#SBATCH --cpus-per-task=1 # Number of CPU cores per task
#SBATCH --gres=gpu:NUMBER_OF_GPUS # number of GPU's to use in the job
#SBATCH --mem-per-cpu=4G # Memory per CPU core
#SBATCH --output=my_job_%j.out # Standard output and error log (%j expands to jobId)
#SBATCH --error=my_job_%j.err # Separate file for standard error

# Load modules or software if required
module load python/python-3.8

# Print some information about the job
echo "Starting my SLURM job"
echo "Job ID: $SLURM_JOB_ID"
echo "Running on nodes: $SLURM_JOB_NODELIST"
echo "Allocated CPUs: $SLURM_JOB_CPUS_PER_NODE"

# Run your application, this could be anything from a custom script to standard applications
# ./my_program
# python my_script.py

# End of script
echo "Job completed"
</syntaxhighlight>

Example with GUI
<syntaxhighlight lang="bash">

srun --pty -c 1 --mem=2G -p <power-xxx> --x11 /bin/bash

</syntaxhighlight>

=== Error Handling ===
* On some clusters, specifying resources is necessary. Without them, the job may fail.
** Example error: <code>srun: error: Unable to allocate resources: No partition specified or system default partition</code>
** Correct usage: <code>srun --pty -c 1 --mem=2G -p power-yoren /bin/bash</code>
* Be aware that specifying GPU resources is crucial for jobs

=== SLURM Information Commands ===

* sinfo: View all queues (partitions).
* squeue: View all jobs.
* scontrol show partition: View all partitions.
* scontrol show job <job_number>: View a job's attributes.

=== Tips for Managing SLURM Jobs ===

* Chain jobs by using the <code>--depend</code> flag in <code>sbatch</code>.
* Use <code>salloc</code> for interactive jobs that require specific resources for a limited time.
* <code>srun</code> is versatile for both interactive and batch jobs, especially with MPI.
* Always specify necessary resources in clusters where defaults are not set.

Submitting a job to a slurm queue

2024-09-18T06:07:13Z

Shosh:

SLURM (Simple Linux Utility for Resource Management) is a job scheduler used on many high-performance computing systems. It manages and allocates resources such as compute nodes and controls job execution.

=== Accessing the System ===
To submit jobs to the SLURM scheduler at Tel Aviv University, you must access the system through one of the designated login nodes. These nodes act as the gateway for submitting and managing your SLURM jobs. The available login nodes are:

* <code>powerslurm-login.tau.ac.il</code>
* <code>powerslurm-login2.tau.ac.il</code>

==== Login Requirements: ====

# Membership in the "power" group: Ensure you are a part of the "power" group which grants the necessary permissions for accessing the HPC resources.
# University Credentials: Log in using your Tel Aviv University credentials. This ensures secure access and that your job submissions are appropriately accounted for under your user profile.

Remember, these login nodes are the initial point of contact for all your job management tasks, including job submission, monitoring, and other SLURM-related operations.

=== Basic Job Submission Commands ===
====Finding your account and partition====
In order to submit jobs to slurm, one needs to know the accounts and partitions she belongs to. Each account may belong to one or more partitions.

To see the account I belong to, please type:
<pre>
sacctmgr show associations where user=dvory format=Account%20
</pre>
If you know your partition, and would like to know which account you need to specify when using it, please do (on powerslurm-login)
<pre>
check_allowed_account -p <partition>
</pre>
For example:
<pre>
check_allowed_account -p power-general
</pre>

====Example for submitting jobs====
# sbatch: Submit a batch job script.
#* Example: <code>sbatch --ntasks=1 --time=10 -p power-general -A power-general-users pre_process.bash</code>
#* This submits <code>pre_process.bash</code> with 1 task for 10 minutes.
#* Example of chaining jobs: <code>sbatch --ntasks=128 --time=60 -p power-general -A power-general-users --depend=45001 do_work.bash</code>
#* Example with GPU: <code>sbatch --ntasks=1 --time=10 --gres=gpu:2 -p gpu-general -A gpu-general-users pre_process.bash</code> or <code>sbatch --gres=gpu:1 -p gpu-general -A gpu-general-users gpu_job.sh</code>
# srun: Submit an interactive job with MPI (Message Passing Interface), often called a "job step."
#* Example: <code>srun --ntasks=2 -p power-general -A power-general-users --label hostname</code>
#* With MPI: <code>srun -intasks=2 -p power-general -A power-general-users --label hostname</code>
# sattach: Attach stdin/out/err to an existing job or job step.

=== Interactive Job Examples ===
* Opening a bash shell: <code>srun --ntasks=56 -p power-general -A power-general-users --pty bash</code>
* Specifying compute nodes: <code>srun --ntasks=56 -p power-general -A power-general-users --nodelist="compute-0-12" --pty bash</code>
* Using GUI: <code>srun --ntasks=56 -p power-general -A power-general-users --x11 /bin/bash</code>

=== Script Examples: ===
<syntaxhighlight lang="bash">
#!/bin/bash

#SBATCH --job-name=my_job # Job name
#SBATCH --account=power-general-users # Account name for billing
#SBATCH --partition=power-general # Partition name
#SBATCH --time=02:00:00 # Time allotted for the job (hh:mm:ss)
#SBATCH --ntasks=4 # Number of tasks (processes)
#SBATCH --cpus-per-task=1 # Number of CPU cores per task
#SBATCH --mem-per-cpu=4G # Memory per CPU core
#SBATCH --output=my_job_%j.out # Standard output and error log (%j expands to jobId)
#SBATCH --error=my_job_%j.err # Separate file for standard error

# Load modules or software if required
# module load python/python-3.8

# Print some information about the job
echo "Starting my SLURM job"
echo "Job ID: $SLURM_JOB_ID"
echo "Running on nodes: $SLURM_JOB_NODELIST"
echo "Allocated CPUs: $SLURM_JOB_CPUS_PER_NODE"

# Run your application, this could be anything from a custom script to standard applications
# ./my_program
# python my_script.py

# End of script
echo "Job completed"

</syntaxhighlight>

==== Script example with GPU ====
<syntaxhighlight lang="bash">
#!/bin/bash

#SBATCH --job-name=my_job # Job name
#SBATCH --account=my_account # Account name for billing
#SBATCH --partition=long # Partition name
#SBATCH --time=02:00:00 # Time allotted for the job (hh:mm:ss)
#SBATCH --ntasks=4 # Number of tasks (processes)
#SBATCH --cpus-per-task=1 # Number of CPU cores per task
#SBATCH --gres=gpu:NUMBER_OF_GPUS # number of GPU's to use in the job
#SBATCH --mem-per-cpu=4G # Memory per CPU core
#SBATCH --output=my_job_%j.out # Standard output and error log (%j expands to jobId)
#SBATCH --error=my_job_%j.err # Separate file for standard error

# Load modules or software if required
module load python/3.8

# Print some information about the job
echo "Starting my SLURM job"
echo "Job ID: $SLURM_JOB_ID"
echo "Running on nodes: $SLURM_JOB_NODELIST"
echo "Allocated CPUs: $SLURM_JOB_CPUS_PER_NODE"

# Run your application, this could be anything from a custom script to standard applications
# ./my_program
# python my_script.py

# End of script
echo "Job completed"
</syntaxhighlight>

Example with GUI
<syntaxhighlight lang="bash">

srun --pty -c 1 --mem=2G -p <power-xxx> --x11 /bin/bash

</syntaxhighlight>

=== Error Handling ===
* On some clusters, specifying resources is necessary. Without them, the job may fail.
** Example error: <code>srun: error: Unable to allocate resources: No partition specified or system default partition</code>
** Correct usage: <code>srun --pty -c 1 --mem=2G -p power-yoren /bin/bash</code>
* Be aware that specifying GPU resources is crucial for jobs

=== SLURM Information Commands ===

* sinfo: View all queues (partitions).
* squeue: View all jobs.
* scontrol show partition: View all partitions.
* scontrol show job <job_number>: View a job's attributes.

=== Tips for Managing SLURM Jobs ===

* Chain jobs by using the <code>--depend</code> flag in <code>sbatch</code>.
* Use <code>salloc</code> for interactive jobs that require specific resources for a limited time.
* <code>srun</code> is versatile for both interactive and batch jobs, especially with MPI.
* Always specify necessary resources in clusters where defaults are not set.

Palo Alto VPN for linux

2024-01-18T08:58:50Z

Shosh:

For security reason TelAviv University starts a VPN with double authentication standard.

In order to do that users have to check/fill in their mobile phone at myTAU page
(https://mytau.tau.ac.il/GetResource.php) and enroll to the service.
Then you need install GoogleAuthenticator on you mobile device and register it at TAU.

After that you may download and install PaloAlto GlobalProtect VPN client on your device (all
operation systems are supported: IOS, Android, Linux MAC and even Window)

The steps:
==Enrollment==
Go to https://mytau.tau.ac.il/GetResource.php

Choose the “1” then “2” :

Then you will receive SMS with 2-minute code and enter it immediately to the filed:
Then you will be redirected to the QR code for GoogleAuthenticator account setup:
Scan it using your mobile Google Authenticator app using “+” on bottom right corner of mobile device
and enter the generated code from mobile GoogleAuthenticator to the field and press the green button.

==Download==
Download and install VPN client, from the browser, go to:

[https://hpcguide.tau.ac.il/vpn/PanGPLinux-5.3.4-c5.tgz GlobalProtect-5.3.4]

[https://hpcguide.tau.ac.il/vpn/PanGPLinux-6.0.1-c6.tgz GlobalProtect-6.0.1]

[https://hpcguide.tau.ac.il/vpn/PanGPLinux-6.1.1-c4.tgz GlobalProtect-6.1.1]

Linux package should be extracted and installed appropriated version:

'''Debian/Ubuntu'''

<code>dpkg -i GlobalProtect_UI_deb-6.0.1.1-6.deb</code>

'''Redhat/Centos'''

<code>yum localinstall GlobalProtect_UI_rpm-6.0.1.1-6.rpm</code>

==Configure==

[[File:Paloalto3.PNG|thumb|right]]

Execute and configure VPN client on Linux (another OS are similar) :

Open client by pressing on the relevant icon ("1" as in the picture on the right)

And enter address '''vpn.tau.ac.il''' ("2" as in the picture on the right)

==Errors==
===SSL Error===
On latest ubuntu version, ubuntu 22.04, after installing and configuring globalprotect VPN, you get this error:

[[File:784px-Vpn ssl error.png|none|thumb]]

====Fix only for globalprotect====
create new <code>ssl.conf</code> file on your pc with the following content:
vim ~/ssl.conf
<syntaxhighlight lang="bash">
openssl_conf = openssl_init
[openssl_init]
ssl_conf = ssl_sect
[ssl_sect]
system_default = system_default_sect
[system_default_sect]
Options = UnsafeLegacyRenegotiation

</syntaxhighlight>
Then find this file:
<code>sudo find / -name PanGPUI.desktop -type f</code>
or
<code>locate PanGPUI.desktop</code> (may need to do sudo updatedb before running this one)
there should be at least 2 path with this file, ignore this one --> <code>/opt/paloaltonetworks/globalprotect/PanGPUI.desktop</code>

On my linux, kubuntu 22.04 the file is here: <code>/etc/xdg/autostart/PanGPUI.desktop</code>
enter this file and change it from:
<syntaxhighlight lang="bash">
[Desktop Entry]
Name=PanGPUI
Type=Application
Exec=/opt/paloaltonetworks/globalprotect/PanGPUI
Terminal=false
</syntaxhighlight>

to
<syntaxhighlight lang="bash">
[Desktop Entry]
Name=PanGPUI
Type=Application
Exec=OPENSSL_CONF=~/ssl.conf /opt/paloaltonetworks/globalprotect/PanGPUI
Terminal=false
</syntaxhighlight>

After restarting you pc, globalprotect will autostart with the custom ssl settings

====Global fix====
here is how to workaround it:

open <code>/usr/lib/ssl/openssl.cnf</code>

comment out this section:<syntaxhighlight lang="bash">
# [openssl_init]

# providers = provider_sect
</syntaxhighlight>add this new section under the commented one from earlier:<syntaxhighlight lang="bash">
[openssl_init]
ssl_conf = ssl_sect

[ssl_sect]
system_default = system_default_sect

[system_default_sect]
Options = UnsafeLegacyRenegotiation

</syntaxhighlight>reboot globalprotect app and the error should be fixed.

<blockquote>source:https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/1960268</blockquote>

==TAU credentials==
[[File:Paloalto4.PNG|thumb|right]]
Fill in pop-upped windows with your TAU credentials:

Open your mobile GoogleAuthenticator and enter code from there

Congratulations: you are done!