Skip to content

VS Code Built-in Chat with the ORC Gateway (BYOK)

VS Code's chat panel can use your own model through Bring Your Own Key (BYOK). You don't need an extra extension or a paid Copilot plan. Once it's set up, the ORC model handles chat, agent mode, and small utility tasks such as chat titles and commit messages.

Time needed: about 5 minutes. Prerequisites: VS Code installed locally, and an ORC LLM key that passes the key test.

Note

Configure this in the copy of VS Code on your own computer. The setting follows you into Remote-SSH sessions on Hopper, so you don't repeat it on the cluster. To edit code on a compute node, see Running VS Code.

Step 1 · Open "Manage Language Models"

Open the Command Palette with Ctrl+Shift+P (Windows/Linux) or Cmd+Shift+P (macOS), type Chat: Manage Language Models, and press Enter.

Step 2 · Add a custom OpenAI-compatible endpoint

Choose the option to add a model provider and select the custom OpenAI-compatible endpoint option.

Step 3 · Enter the gateway details

Answer the prompts with:

Prompt Enter
Base URL http://129.174.189.202
API key Your ORC virtual key
Model ID qwen-3.8-27B

VS Code stores the key in your operating system's keychain, not in a settings file.

Step 4 · Pick the model

In the Chat view, open the model picker at the bottom of the input box and select the ORC model. Send a short test message such as "Reply with OK" before doing real work.

Without this step, background tasks such as generating chat titles and commit messages fall back to Copilot's own models, or fail. Open Preferences: Open User Settings (JSON) and add:

"chat.utilityModel": "qwen-3.8-27B",
"chat.utilitySmallModel": "qwen-3.8-27B"

What VS Code saves for you

VS Code writes your answers to a file called chatLanguageModels.json in your VS Code user folder. You don't type this by hand, but it's useful to recognize when troubleshooting. It looks like this (the key itself is a keychain reference, not the real value):

[
  {
    "name": "ORC LLM",
    "vendor": "customendpoint",
    "apiKey": "${input:chat.lm.secret.xxxxxxxx}",
    "apiType": "chat-completions",
    "models": [
      {
        "id": "qwen-3.8-27B",
        "name": "Qwen 3.8 27B (ORC)",
        "url": "http://129.174.189.202",
        "toolCalling": true,
        "vision": true,
        "maxInputTokens": 128000,
        "maxOutputTokens": 16000
      }
    ]
  }
]

toolCalling: true is what allows agent mode to edit files and run commands. If agent mode doesn't offer the ORC model, check that this line is present.

What BYOK does not cover: inline autocomplete

BYOK covers chat, agent mode, and utility tasks. The grey inline suggestions that appear as you type are a separate feature that always runs on GitHub's own service; a custom endpoint can't take it over, however it is configured. If you want the ORC model to do autocomplete too, use Continue.

Troubleshooting

Problem Fix
The custom endpoint option never appears If your GitHub account belongs to an organization with a Copilot policy, an admin may need to enable BYOK. Update VS Code to the latest version, then contact orchelp@gmu.edu if it's still missing.
"Model not found" or 404 The model ID must match the gateway exactly. List the valid names with the key test. If the name is right, try the base URL with /v1 appended.
401 / authentication error Re-run Chat: Manage Language Models, remove the ORC entry, and add it again with the correct key.
Requests time out Connect to the Mason VPN if you are off campus.
Agent mode can't edit files Confirm "toolCalling": true in chatLanguageModels.json.

See Also