Skip to content

The ORC LLM Gateway

ORC runs an LLM gateway that serves a hosted model to anyone with a personal key. It speaks the OpenAI-compatible API, so any tool that lets you set a base URL and an API key can use it. No Anthropic, OpenAI, or GitHub Copilot account is involved.

At a glance

Setting Value
Base URL http://129.174.189.202
Model qwen-3.8-27B
API style OpenAI-compatible (chat completions); Anthropic-compatible for Claude Code
Authentication Your personal virtual key (sk-…), sent as Authorization: Bearer <key>
Reachable from Your laptop (on campus or on the Mason VPN) and from Hopper nodes

Tip

The model name must match what the gateway serves exactly, including capitalization. If a tool reports "model not found", list the models your key can use (see Test your key) and copy the name from there.

Get a key

Keys are issued one per person. Email orchelp@gmu.edu with your NetID and a sentence about what you'll use it for, and ORC will set one up for you. If you attended an ORC LLM workshop, you were given a key for the session.

Test your key

Test the key once from a terminal before configuring any editor. If this works, any later problem is in the tool's configuration, not in your key or network.

macOS / Linux / Hopper

# Read the key without echoing it to the screen or saving it in your shell history
read -rs -p "ORC LLM key: " ORC_LLM_KEY; echo
export ORC_LLM_KEY

# 1. Which models can this key use?
curl -s http://129.174.189.202/v1/models \
  -H "Authorization: Bearer $ORC_LLM_KEY"

# 2. Send one short message
curl -s http://129.174.189.202/v1/chat/completions \
  -H "Authorization: Bearer $ORC_LLM_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "qwen-3.8-27B",
       "messages": [{"role": "user", "content": "Say hello in five words."}],
       "max_tokens": 50}'

Windows PowerShell

$env:ORC_LLM_KEY = Read-Host "ORC LLM key" -MaskInput   # PowerShell 7+; on 5.1 paste it into the quotes: $env:ORC_LLM_KEY = "sk-..."

Invoke-RestMethod -Uri "http://129.174.189.202/v1/models" `
  -Headers @{ Authorization = "Bearer $env:ORC_LLM_KEY" }

What the result means

You see Meaning
JSON listing one or more models Key and network are fine. Use the model id shown.
A JSON reply with "choices" and a message Everything works end to end.
401 / Authentication Error The key is wrong, expired, or revoked. Check for extra spaces; otherwise contact ORC.
400 mentioning the model The model name doesn't match. Use the name from /v1/models.
Timeout or "connection refused" You are off campus without the Mason VPN, or the gateway is down. Connect to the VPN and retry; if it persists, contact ORC.

Keep your key safe

Your virtual key is a credential, exactly like a password.

  • One key, one person. Don't share it; collaborators can request their own.
  • Never commit it. Keep it out of Git repositories. Add any file that contains it (for example .env, or a project-level config) to .gitignore.
  • Never paste it into a wiki page, a ticket, a chat, a slide, or a prompt. Screenshots count; crop before sharing.
  • Prefer environment variables or your OS keychain over plain-text files. VS Code's built-in BYOK stores the key in your OS keychain automatically.
  • If it leaks, rotate it. Email orchelp@gmu.edu and ORC will revoke it and issue a new one.

Use the gateway from your own code

Because the gateway is OpenAI-compatible, the official openai Python package works with only the base URL changed. Install it (pip install openai) in a Python virtual environment on Hopper or on your laptop:

import os
from openai import OpenAI

client = OpenAI(
    base_url="http://129.174.189.202/v1",
    api_key=os.environ["ORC_LLM_KEY"],   # never hard-code the key
)

reply = client.chat.completions.create(
    model="qwen-3.8-27B",
    messages=[{"role": "user", "content": "Explain Slurm's --time flag in one sentence."}],
)
print(reply.choices[0].message.content)

Next steps

Configure the tool you want to use:

See Also