Using Large Language Models (LLMs) on Hopper

The Office of Research Computing (ORC) offers two ways to use a large language model (LLM) from your editor, your terminal, or your own scripts:

The ORC LLM gateway: a model ORC already runs for you. You get a base URL and a personal key, and any tool that speaks the OpenAI API (or, for Claude Code, the Anthropic API) can use it. There is nothing to launch and no GPU to reserve.
Your own llama.cpp server: you reserve a slice of an A100 GPU through Open OnDemand and run a model of your choice for the length of your job.

Most people use the gateway for day-to-day coding help and start their own server only when a specific model, a private endpoint, or control over serving parameters is the point of the work.

How the pieces fit together

 Your laptop                         Hopper
 ┌───────────────────────┐           ┌──────────────────────────────────────┐
 │ VS Code · Claude Code │──SSH────▶ │ Login node ──ProxyJump──▶ Compute node│
 └──────────┬────────────┘           └──────────────────────────────────────┘
            │
            │  HTTP + your virtual key
            ▼
 ┌───────────────────────────────┐    ┌──────────────────────────────────────┐
 │ A · ORC LLM gateway           │ or │ B · Your own llama.cpp server        │
 │ always on, nothing to launch  │    │ Open OnDemand job on an A100 slice   │
 │ model: qwen-3.8-27B           │    │ Gemma · Qwen3 · Qwen3.6              │
 └───────────────────────────────┘    └──────────────────────────────────────┘

The gateway can be reached from your laptop and from Hopper, so you do not need a Hopper session at all to use it. A Slurm allocation is only needed if you are also editing code on a compute node (Running VS Code) or running your own model.

Which option should I use?

Use the gateway when…	Run your own server when…
You want an assistant in your editor today, with no setup.	You need a particular model or quantization.
`qwen-3.8-27B` is good enough for the task.	Your prompts and data shouldn't leave your own job.
You work from your laptop as often as from Hopper.	You are benchmarking or changing serving parameters.
You don't want to hold a GPU reservation.	You can work within the 12-hour GPU session limit.

Before you start

You need:

An active Hopper account (your Mason NetID plus an approved ORC account request). See Logging into Hopper.
The Mason VPN if you are off campus.
VS Code installed on your own computer, if you plan to use an editor.
An LLM virtual key for the gateway. Email orchelp@gmu.edu to request one.

You don't need:

A GPU reservation to use the gateway.
An Anthropic, OpenAI, or GitHub Copilot subscription. The gateway replaces them.
Admin rights on Hopper.

Pick your tool

Page	Use it for
The ORC LLM Gateway	Start here. Base URL, model name, getting and protecting your key, testing the connection.
VS Code built-in chat (BYOK)	Chat and agent mode inside VS Code, with no extra extension.
VS Code + Continue	Chat and inline autocomplete from the ORC model.
VS Code + Cline	An autonomous coding agent that plans and edits files step by step.
Claude Code	A terminal coding agent pointed at the gateway.
Run your own model with llama.cpp	Your own model on an A100 slice through Open OnDemand.
Responsible Use of LLMs	What never goes in a prompt, and how to run an agent safely.

Quick comparison of the editor tools

	VS Code BYOK	Continue	Cline	Claude Code
Where it runs	VS Code	VS Code	VS Code	Terminal (any machine)
Chat	Yes	Yes	Yes	Yes
Agent (edits files, runs commands)	Yes	Yes	Yes	Yes
Inline autocomplete from the ORC model	No	Yes	No	No
Extra extension needed	No	Yes	Yes	CLI install