Getting Started on Mjolnir

Mjolnir access goes through the UCPH VPN, then SSH with your KU ID. Logging in doesn't put you on the machine that runs analyses — it puts you on a shared login node. The first habit to build, every single session, is confirming who you are and where you've landed before doing anything else.

Connect to Mjolnir, then run three commands: one that prints your username, one that prints the machine's name, and one that prints your current location.

The connection itself is one SSH command using your KU ID. Once in, whoami, hostname, and pwd each answer one of the three questions above.

Does whoami match your KU-ID? What command or piece of information would you use to confirm which machine/node you are currently on?

An HPC (High-Performance Computing) cluster is many networked computers sharing storage and a scheduler, pooled together so jobs too large for a single laptop — and jobs from many people at once — can run efficiently. Mjolnir is UCPH's cluster for exactly this kind of research. Crucially, you are never the only person using it: everything about how you're expected to behave on it (which you'll meet across this module) follows from that one fact.

In your own words, write a short note explaining what makes a cluster different from your own laptop, and why sharing it with many other researchers changes how you're expected to use it.

Think about what would happen if every user on a shared machine ran heavy, long-running work wherever and whenever they liked.

Can you name one concrete rule a shared cluster needs that your own laptop never would?

Mjolnir's login node is a shared entry point for everyone connected at that moment — meant for lightweight tasks: organising files, writing scripts, submitting and checking jobs. Actual analyses run on compute nodes, scheduled through Slurm. Running real computation on the login node slows down every other user's session, and persistent misuse can lead to your access being withdrawn.

List three tasks that belong on a login node, and three that must go to a compute node instead. Think ahead to the kind of work you'll do in later modules (downloading a small reference file vs. running an alignment, for example).

The deciding factor isn't "is this a command I type" — it's how much CPU, memory, or time it's likely to need.

What do you think actually happens — technically and socially — if everyone treated the login node like a compute node?

These four commands are the absolute foundation of working on any Linux server, and you'll run them more than anything else in this entire project: pwd (where am I), ls (what's here), ls -lh (the same, with detail and readable sizes), and cd (go there).

Starting from wherever you land after logging in, check your location, list what's there in detail, move into any subdirectory you can see, and confirm you arrived.

cd takes the name (or path) of where you want to go as its argument.

How can you tell from ls -lh output which entries are directories and which are files?

. means "this directory," .. means "the parent directory," and ~ means "my home directory" — all three save you from typing full paths constantly, and all three appear again and again in other commands, not just cd.

From wherever you currently are, jump straight home, then move up one level from there, then return to exactly where you started using the shortcut, not a typed-out path.

cd ~ works from anywhere. cd - is a bonus shortcut that returns you to wherever you were before your last cd.

Where might . be useful as an argument to a command other than cd — for example, as a destination for cp?

An absolute path starts from the root of the filesystem (/) and is unambiguous no matter where you currently are. A relative path is interpreted relative to your current directory — shorter to type, but it only means what you think it means if you know your starting point. Mixing these up is one of the most common sources of "file not found" errors, and worse, scripts that silently touch the wrong location.

Print the absolute path of your home directory without moving there. Then, from a subdirectory of your choice, write down a relative path that points back to that same home directory.

echo ~ expands and prints your home directory's absolute path without changing your location. An absolute path always starts with /; a relative one never does.

If you moved to a completely different directory first, would the same relative path (e.g. ../data) still point to the same place? Why not — and why does an absolute path not have this problem?

Your home directory (/home/<KU-ID>) is personal but capped at 100 GB, and is meant for configuration files and small scripts — not datasets or analysis output. Actual project data belongs under /projects/<project>/, which has a standard layout: people/<KU-ID>/ (your own working space), data/ (shared datasets used by the whole project), and scratch/ (temporary working space — not backed up, so treat anything there as disposable). If your people/ folder doesn't exist for you yet, ask your supervisor or Mjolnir support rather than improvising a location yourself.

Find both locations: your home directory, and your personal folder within the project's shared space. Don't create anything yet — just confirm both exist and note the difference.

echo ~ gives you the first. The second has the shape /projects/<project>/people/<KU-ID>/ — ls /projects/ shows you which projects you can see.

Which of these two locations should hold your actual working files for this course, and why?

Now that you know where project work belongs, create a dedicated space for the exercises in this module — kept separate from the Great Auk project directory you'll set up in Module 01.

Inside your personal project folder, create a directory for practising these HPC exercises, and move into it.

A short, clear, lowercase name with no spaces — something like hpc_intro.

Does pwd confirm you're inside the new directory, under /projects/?

/projects/ storage isn't currently quota-enforced the way your home directory is — but it's still a shared, finite resource used by many research groups, not an unlimited personal allocation. Checking usage periodically is good practice, not a one-off task.

Check how much space your personal project folder is using, and which subfolders (if any) are using the most.

The flag that matters for accurate reporting is --apparent-size — it reports actual data size rather than filesystem allocation, which is what you want for capacity planning.

Does the first command give you one total? What does the second one show you that the first doesn't?

cp copies a file, leaving the original in place. mv moves it — or renames it, since there's no separate "rename" command. rm deletes a file permanently: there is no undo and no recycle bin, so always double-check the path (and any wildcard) before pressing enter.

Create a small text file, make a copy of it, rename that copy, then delete the renamed copy — leaving the original untouched.

echo "some text" > filename writes that text into a new file. mv used with a new name in the same directory renames a file.

Is notes_copy.txt recoverable after rm? Does notes.txt still exist? What's one habit worth building before running rm, especially with a wildcard like *?

Files on a cluster can be huge — opening one fully is slow at best, and can freeze your session at worst. cat prints a whole (small) file. head prints just the start. less lets you page through interactively without loading everything at once. file reports what a file actually is, regardless of its extension.

Using the notes.txt file from the previous exercise, try all four commands.

less is interactive — press q to quit back to the prompt.

What does file report for this plain text file? You'll see it report something very different — "gzip compressed data" — once you start working with real genomic files in Module 01. Why is checking this before you assume a file's format a good habit?

Mjolnir provides shared research software through Environment Modules. Rather than everyone installing their own copy of every bioinformatics tool, pre-installed software is loaded into your shell's (or job's) environment on demand with a single command. Much of it is packaged internally as Conda environments, but you access it through the module system, not by managing Conda yourself.

Before touching any commands, think about and note down: why would a shared cluster prefer this system over letting every user install and manage their own software?

Think about disk space (your home directory's 100 GB cap), reproducibility, and what happens if 50 people each install a slightly different version of the same tool.

Why might installing your own Conda environment in your home directory specifically be a bad idea on Mjolnir?

module avail lists everything installed; adding a search term narrows it. module whatis and module display show details before you commit to loading something. module load brings a tool into your environment — pin an exact version for work you intend to repeat. module list shows what's currently loaded, and module purge clears everything.

Modules required by a batch job must be loaded inside the sbatch script itself — a batch job starts in its own environment on a different machine, so anything you loaded interactively in your login shell does not carry over.

Note: Mjolnir does not support module spider, even though you may see it mentioned elsewhere — use module avail and module apropos instead.

Search for samtools (a very common bioinformatics tool you'll likely use later), check what versions exist, load one, confirm it's active, then clear your environment again.

which samtools confirms a loaded tool is actually on your PATH.

What does module list show immediately after module purge? Why might it be good practice to start every batch script with module purge before loading what you actually need?

Slurm is the scheduler that decides which job runs on which compute node, and when. Every submitted job gets a job ID, and must declare the CPU, memory, and maximum wall time it needs, plus which partition (a pool of nodes for a particular purpose) and QoS (quality of service — a priority/limits tier) it should run under. On Mjolnir, --qos is mandatory: jobs submitted without one are rejected outright. cpuqueue is the general-purpose default partition; normal is the everyday default QoS. Mjolnir also has other partitions and QoS tiers for other situations (GPU work, low-priority background jobs, and more) — cpuqueue/normal is simply the everyday default you'll use for now.

Before writing any script, note down: why does a shared cluster require everyone to declare their resource needs upfront, rather than just letting jobs run and seeing what happens?

Think about what information the scheduler needs to fit many users' jobs onto a limited number of machines without conflicts.

What do you think happens if you request far more memory than your program actually ends up using? Hold that thought for Exercise 19.

A Slurm batch script is an ordinary bash script with #SBATCH comment lines at the top telling Slurm what resources and settings to use before your actual commands run. --output/--error with %j let Slurm substitute the real job ID into the log filenames once it's submitted.

Create a file called hello_mjolnir.sbatch that requests the minimum sensible resources for a job that only prints a hostname, a date, and a message — then prints that message.

A job that just runs hostname/date/echo needs barely any CPU, memory, or time — resist the urge to request more "just in case." You can create a multi-line file directly from the command line with a cat <<'EOF' ... EOF block, or use a terminal editor like nano if you prefer.

Why is requesting 1 CPU and 500 MB the right call here, compared to a heavier example you might see elsewhere in the documentation (e.g. 4 CPUs and 2 GB per CPU)?

sbatch hands your script to Slurm, which immediately replies with a job ID. You'll need that ID for every remaining exercise in this module — monitoring, logs, and efficiency checks are all keyed on it.

Submit hello_mjolnir.sbatch, and record the job ID it returns in your notes below.

One command, the script's filename as its only argument.

What job ID did you get?

squeue -u $USER shows your own queued and running jobs. A job's state (column ST) moves through PD (pending), R (running), CG (completing), and finally disappears from this live view once it's COMPLETED (or FAILED/TIMEOUT/CANCELLED).

Run squeue -u $USER a few times, quickly — your job only takes seconds, so you're watching for it to appear running and then disappear.

While it's running, the last column of squeue's output shows the compute node's name. While pending, it shows the reason it's waiting instead.

Which compute node did your job run on, according to squeue? Write it down — the final challenge will ask you to confirm this a different way.

Because your script set --output/--error with %j, Slurm wrote two files named after your job ID. Normal output (like echo and date) goes to the .out file; errors and warnings go to the .err file. Telling these apart matters hugely once you're running real bioinformatics tools that report problems this way.

List your directory, find the two log files matching your job ID, and inspect both.

A wildcard with your job ID will match both files at once in ls.

Does the hostname printed inside the .out file match the compute node name squeue showed you? Is the .err file empty — what would you expect to see there if something had gone wrong?

seff <jobid> gives a quick summary of CPU and memory use as a percentage of what you requested. sacct gives more detail. Right-sizing requests matters for reasons beyond your own job: fairshare is charged on what you request, not what you consume, so over-requesting hurts your own future priority; memory you reserve but don't use is unavailable to anyone else for the whole run, even while the CPUs sit idle; and smaller, realistic requests let the scheduler backfill jobs into gaps far more easily.

Check your finished job's efficiency, and compare what you requested against what it actually used.

Both commands take your job ID as an argument.

Did your job use anywhere near the 1 CPU / 500 MB you requested? If you'd requested 8 CPUs and 16 GB "just to be safe" instead, what would that have cost — to you and to everyone else sharing Mjolnir right now?

scancel <jobid> requests termination — it doesn't guarantee an instant stop. Running jobs get a brief grace period before being forced to stop; pending jobs are simply removed from the queue. Appropriate reasons to cancel include wrong input files or arguments, wrong resource requests, an accidental duplicate submission, or a job that's clearly stuck. Repeatedly cancelling and resubmitting to try to "jump the queue" doesn't work — fairshare priority isn't affected by that.

Submit a second, deliberately slow test job (one that just sleeps for a while), then cancel it before it finishes.

This needs its own script and its own job ID — your first job already finished, so there's nothing left to cancel there.

After running scancel, how long did it take for the job to actually disappear from squeue? Was it instant?

This is where every piece of this module comes together, independently, exactly as you'll need to work for every real analysis step from Module 01 onward. Nothing here is new — the only new part is doing it without a template in front of you.

Without copying a ready-made script, independently:

• Create (or navigate to) an appropriate working directory
• Write a small .sbatch script requesting sensible, minimal resources
• Load a module inside the script if your job actually uses one (e.g. print a tool's version)
• Submit the job and record the job ID it returns
• Monitor it until it finishes
• Identify which compute node ran it
• Locate and read its stdout and stderr files
• Check its completed-job resource usage
• Record requested vs. actual resource use
• In your notes, briefly explain why this job belonged on a compute node rather than the login node

Every single step here was covered somewhere in Exercises 08 and 13–20. If you get stuck on one specific step, that's the exercise to revisit — rather than reaching for the reveal below immediately.

Can you point to, right now: your job ID, the compute node's name, the contents of your .out file, and at least one actual-vs-requested resource number? If any of those are missing, that's a sign to revisit the relevant exercise above rather than mark this complete.