PowerIDE User Guide

From HPC Guide
Revision as of 07:36, 26 May 2026 by Levk (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

PowerIDE User Guide

PowerIDE provides interactive access to the HPC cluster through a web browser. You can run Jupyter notebooks, VS Code, and RStudio directly on compute nodes without needing SSH access.

---

Getting Started

1. Access PowerIDE

Open your web browser and navigate to:

https://poweride.tau.ac.il/jupyter

2. Login

Log in with your TAU university credentials:

  • Username: Your TAU username
  • Password: Your TAU password

This is the same login you use for email and other university services.

---

Starting Your Server

After logging in, you'll see a page with a large orange button:

Click "Start My Server"

You will then be presented with a Server Options form where you configure your compute resources.

How It Works

When you start your server, PowerIDE submits a Slurm job to the PowerSlurm cluster. Your Jupyter session runs on a compute node, not on the PowerIDE server itself.

This means:

  • You get dedicated resources (CPUs, memory, GPUs) on a compute node
  • Your job runs through the same Slurm scheduler as other HPC jobs
  • The PowerIDE server is only the web interface - all computation happens on cluster nodes
  • Your session will queue if the cluster is busy (just like regular batch jobs)

---

Configuring Resources

File:Server Options Form.png
Server Options form showing resource selection

The form includes the following fields:

Partition

Select which partition (queue) to run on. The dropdown will only show partitions you have access to based on your Slurm account permissions.

Common partitions:

  • `power-general-shared-pool` - General purpose computing
  • `gpu-general-pool` - GPU-enabled partition (if available)
  • Check with your PI or HPC admin for which partitions you should use

QOS (Quality of Service)

Select the QOS for your job. This controls priority and resource limits.

  • Default (owner) - Usually the best choice (uses your group's default QOS)
  • Other options may be available based on your partition selection

The form will automatically show only valid QOS options for your selected partition.

GPUs

If you select a GPU-enabled partition, a GPUs field will appear. Specify how many GPUs you need (0 if none).

The maximum number of GPUs is automatically limited based on the partition's capabilities.

Time (D-HH:MM:SS)

Specify how long your session should run. Default is `04:00:00` (4 hours).

Formats accepted:

  • `HH:MM:SS` - e.g., `02:30:00` for 2.5 hours
  • `D-HH:MM:SS` - e.g., `1-12:00:00` for 1 day and 12 hours

Important: Your session will be terminated when time runs out. Save your work regularly!

CPUs per task

Number of CPU cores for your session. Default is `2`.

Increase this if you're running multi-threaded code.

Memory

Amount of RAM to allocate. Default is `4G`.

Examples:

  • `4G` - 4 gigabytes
  • `16G` - 16 gigabytes
  • `64G` - 64 gigabytes

Tip: Start with less and increase if needed. Over-requesting resources may delay job start.

Working directory

Default: Your home directory.

Recommendation: Change this to your project directory to save navigation time after launch. For example:

  • `/a/home/cc/students/yourgroup/project1`
  • `/scratch/yourusername/analysis`

Stdout / Stderr directory

Where to write job output and error logs. Default: your home directory.

Usually fine to leave as default.

---

Starting Your Session

After filling out the form, click the Start button at the bottom.

What happens next:

  1. PowerIDE submits a Slurm job with your requested resources
  2. You'll see a progress page saying "Your server is starting up..."
  3. Wait for a compute node to become available (usually 10-60 seconds)
  4. Once started, you'll automatically be redirected to JupyterLab

Note: If the cluster is busy, it may take longer. You can close the browser and come back — your session will start when resources are available.

---

Using JupyterLab

Once your server starts, you'll land in JupyterLab — a web-based development environment.

JupyterLab Interface

  • Left sidebar: File browser, running kernels, extensions
  • Main area: Notebooks, text files, terminals
  • Launcher: Click the + button to see available tools

Common Tasks

Create a new notebook:

  1. Click the + button (or File → New Launcher)
  2. Click on a kernel (e.g., "Python 3")
  3. Start coding!

Open a terminal:

  1. Click the + button
  2. Click "Terminal" in the launcher
  3. You now have a bash shell on the compute node

Upload files:

  • Drag and drop files into the file browser, OR
  • Click the upload button (↑ icon) in the file browser

Download files:

  • Right-click file → Download

Stopping JupyterLab

Always stop your server when you're done to free up resources for others!

---

Using VS Code

VS Code runs as a separate service with its own Slurm job, independent of JupyterLab.

Starting VS Code

  1. Go to https://poweride.tau.ac.il/jupyter/services/vscode/
  2. Log in if prompted
  3. Fill in the launch form and click Launch VS Code

The button will spin while your job is being submitted. Once the job starts on a compute node you'll be redirected automatically.

Launch form fields

Same resource fields as JupyterLab (Partition, QOS, CPUs, Memory, Time, GPUs), plus:

Field What it does
Working directory Directory VS Code opens in (default: home)
VS Code data dir Where extensions and settings are stored (default: home). Tip: set to `/scratch/username` if your home directory is slow or has limited quota.
Node constraint Require specific hardware (e.g. `intel`, `amd`)
Specific node Pin job to one node (e.g. `compute-0-5`)

Stopping VS Code

To free cluster resources, always stop your session when done.

Method How
Top bar button Click ⏹ Stop current job → confirm
URL Navigate to `https://poweride.tau.ac.il/jupyter/services/vscode/stop`
Terminal `scancel --name=vscode-<yourusername>`

Note: Closing the browser tab does not stop the job. It keeps running until the time limit.

Reconnecting

Go to the same URL. If your job is still running you'll be taken straight back in.

---

Using RStudio

RStudio runs as a separate service with its own Slurm job, independent of JupyterLab and VS Code.

Starting RStudio

  1. Go to https://poweride.tau.ac.il/jupyter/services/rstudio/
  2. Log in if prompted
  3. Choose your R environment and resource settings, then click Launch RStudio

Launch form fields

Same resource fields as the other services, plus:

Field What it does
R environment The R module to load (sets R version and available packages). Contact HPC support if your module is not listed.

Installing R packages

Install packages directly from the RStudio console: 

install.packages("ggplot2")

Packages install to your personal R library (`~/R/…`) and persist across sessions.

Stopping RStudio

Important: "File → Quit Session" inside RStudio only ends the R session — it does not stop your Slurm job. RStudio will reload with a fresh empty session and your job keeps running.

To fully stop the job:

Method How
Top bar button Click ⏹ Stop current job → confirm
URL Navigate to `https://poweride.tau.ac.il/jupyter/services/rstudio/stop`
Terminal `scancel --name=rstudio-<yourusername>`

Note: Closing the browser tab does not stop the job either.

Reconnecting

Go to the same URL while your job is running and you'll be taken straight back in.

---

Python Environments

What is a Python Kernel?

A kernel is simply a Python interpreter that JupyterLab uses to run your code. When you create a notebook and select "Python 3.12 (Base)", you're choosing which Python environment to use.

Default Kernel

PowerIDE provides one default kernel:

  • Python 3.12 (Base) - Standard Python with JupyterLab and common packages

Creating Your Own Kernels

You can register your own conda/mamba environments as kernels! This lets you:

  • Use different Python versions (3.9, 3.10, 3.11, etc.)
  • Install custom packages without affecting others
  • Have multiple project-specific environments

Example: 

# From a JupyterLab terminal (or any cluster node):
module load mamba/mamba-2.1.1
mamba create -n my-project python=3.11 pandas matplotlib
mamba activate my-project
mamba install ipykernel

# Register as kernel (--user means only you will see it)
python -m ipykernel install --user --name my-project --display-name "My Project (Python 3.11)"

# Done! Refresh your browser and look for "My Project (Python 3.11)" in the launcher

Important notes:

  • Kernels are just small config files (~1 KB) — they don't use your disk quota
  • Each user only sees their own kernels (plus system defaults)
  • Remove a kernel: `jupyter kernelspec uninstall kernel-name`

Contact hpc@tauex.tau.ac.il if you need help setting up a kernel.

---

Best Practices

Resource Allocation

  • Start small: Request fewer resources initially. You can always restart with more.
  • Be realistic: Only request what you actually need
  • Time limits: Set a reasonable time limit. You can always restart if you need more time.
  • GPU usage: Only request GPUs if your code actually uses them

File Management

  • Working directory: Set it to your project folder to save navigation time
  • Save frequently: Your session will end when time runs out
  • Large files: Store large datasets in scratch space, not your home directory
  • VS Code extensions: Store in `/scratch/username` if home quota is limited

Data and Code

  • Home directory: Personal files, small projects
  • Scratch space: Large temporary datasets
  • Project directories: Shared group work (varies by group)

Tip: Use Git to version control your code, not for large data files.

---

Troubleshooting

My server won't start

Possible reasons:

  • Cluster is full: Wait a few minutes and try again
  • Invalid partition: Make sure you selected a partition you have access to
  • Too many resources: Try requesting fewer CPUs/memory
  • No QOS access: You may not have any QOS configured for your account

What to do:

  1. Wait 2-3 minutes
  2. If still pending, go to Hub Control Panel and click "Stop My Server"
  3. Try again with fewer resources or a different partition
  4. If QOS dropdown is empty, contact HPC support

Stuck on "Starting…" for a long time

Your job is queued. Check your position: 

squeue -u $USER

If it's been more than 10–15 minutes, try a partition with more availability or reduce resource requests.

Page shows "Slurm state: NONE" right after submitting

The job submitted but exited immediately. Check the log file: 

cat ~/vscode-<jobid>.out
cat ~/rstudio-<jobid>.out

"Service unavailable" / blank page

The backend is still starting or the job ended. Wait a few seconds and refresh. If it persists, go back to the service URL — if the job died you'll see the launch form.

I see "404: Not Found" (JupyterLab)

This usually means your job didn't start successfully.

  1. Go to your home directory on a login node
  2. Look for files named `jupyterhub-JOBID.err`
  3. Check the file for error messages
  4. Contact HPC support if you can't resolve it

My session was killed

Common reasons:

  • Time limit reached: Your session ran for the full time you requested
  • Out of memory: Your code used more RAM than allocated
  • Node failure: Rare, but compute nodes can crash

Check the log for details: 

cat ~/vscode-<jobid>.err
cat ~/rstudio-<jobid>.out

I accidentally launched two jobs

Run `scancel --me` in a terminal to cancel all your jobs, or cancel by name: 

scancel --name=vscode-<yourusername>
scancel --name=rstudio-<yourusername>

R module not in the list?

R environments are set up by HPC staff and refreshed weekly. If your module is not listed, contact hpc@tauex.tau.ac.il.

---

Getting Help

For technical issues, email: hpc@tauex.tau.ac.il

When asking for help, include:

  • Your username
  • What you were trying to do
  • Error messages (copy/paste or screenshot)
  • Job ID if available

---

FAQ

Q: Can I run multiple services at once?

A: Yes. Jupyter, VS Code, and RStudio each run as independent Slurm jobs. You can have all three running simultaneously, limited only by your account's resource quota.

Q: Can I run multiple Jupyter servers at once?

A: No, you can only have one Jupyter server running at a time.

Q: How long can my session run?

A: Varies by partition — most partitions allow up to 7 days maximum.

Q: Can I install Python packages?

A: Yes! Create your own conda/mamba environment, install whatever packages you need, and register it as a kernel. See the Python Environments section.

Q: Why is my QOS dropdown empty?

A: You don't have any QOS associations configured in Slurm. Contact HPC support — they need to add you to a Slurm account with appropriate QOS access.

Q: What happens to my files when I stop my server?

A: Your files are safe! Only the running session is terminated. All files in your home directory and project directories remain intact.

Q: Can I share my session with a colleague?

A: No, sessions are personal. However, you can share notebooks and code files through the filesystem or Git.

Q: Is PowerIDE the same as the login nodes?

A: No! PowerIDE runs on compute nodes through the Slurm scheduler, giving you dedicated resources. Login nodes are shared by everyone.

Q: How do I get access to different partitions?

A: Partition access is controlled by Slurm account associations. Contact your PI or HPC admin to request access to specific partitions.

---

Quick Reference Card

Action How-To
Access PowerIDE https://poweride.tau.ac.il/jupyter
Start Jupyter server Click "Start My Server" → Fill form → Click "Start"
Open notebook Click + → Choose Python kernel
Open terminal Click + → Click "Terminal"
Stop Jupyter server File → Hub Control Panel → "Stop My Server"
Start VS Code Go to /services/vscode/ → Fill form → Click "Launch VS Code"
Stop VS Code Click ⏹ Stop current job in top bar, or `scancel --name=vscode-<username>`
Start RStudio Go to /services/rstudio/ → Select R module → Fill form → Click "Launch RStudio"
Stop RStudio Click ⏹ Stop current job in top bar, or `scancel --name=rstudio-<username>`
Upload files Drag & drop into JupyterLab file browser
Download files Right-click file → Download
Get help Email hpc@tauex.tau.ac.il

---

Happy computing! 🚀

Retrieved from "https://hpcguide.tau.ac.il/index.php?title=PowerIDE_User_Guide&oldid=1562"