{"id":1058,"date":"2025-10-07T13:39:22","date_gmt":"2025-10-07T17:39:22","guid":{"rendered":"https:\/\/carleton.ca\/rcs\/?page_id=1058"},"modified":"2026-07-06T13:57:31","modified_gmt":"2026-07-06T17:57:31","slug":"llm-access","status":"publish","type":"page","link":"https:\/\/carleton.ca\/rcs\/llm-access\/","title":{"rendered":"LLM Access"},"content":{"rendered":"\n<section class=\"w-screen px-6 cu-section cu-section--white ml-offset-center md:px-8 lg:px-14\">\n    <div class=\"space-y-6 cu-max-w-child-5xl  md:space-y-10 cu-prose-first-last\">\n\n            <div class=\"cu-textmedia flex flex-col lg:flex-row mx-auto gap-6 md:gap-10 my-6 md:my-12 first:mt-0 max-w-5xl\">\n        <div class=\"justify-start cu-textmedia-content cu-prose-first-last\" style=\"flex: 0 0 100%;\">\n            <header class=\"font-light prose-xl cu-pageheader md:prose-2xl cu-component-updated cu-prose-first-last\">\n                                    <h1 class=\"cu-prose-first-last font-semibold !mt-2 mb-4 md:mb-6 relative after:absolute after:h-px after:bottom-0 after:bg-cu-red after:left-px text-3xl md:text-4xl lg:text-5xl lg:leading-[3.5rem] pb-5 after:w-10 text-cu-black-700 not-prose\">\n                        LLM Access\n                    <\/h1>\n                \n                                \n                            <\/header>\n\n                    <\/div>\n\n            <\/div>\n\n    <\/div>\n<\/section>\n\n\n\n<p>The Research and Computing Services created a Large Language Model server to provide access to the Carleton University community. REST API endpoints can be accessed upon request. There are many models deployed on this server, including:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>all-minilm:33m<\/li>\n\n\n\n<li>all-minilm:latest<\/li>\n\n\n\n<li>command-r-plus:latest<\/li>\n\n\n\n<li>command-r:latest<\/li>\n\n\n\n<li>embeddinggemma:latest<\/li>\n\n\n\n<li>gemma4:26b<\/li>\n\n\n\n<li>gemma4:latest<\/li>\n\n\n\n<li>gpt-oss:120b<\/li>\n\n\n\n<li>gpt-oss:20b<\/li>\n\n\n\n<li>llama3.1:latest<\/li>\n\n\n\n<li>llama3.2-vision:90b<\/li>\n\n\n\n<li>llama3.2:latest<\/li>\n\n\n\n<li>llama3.3:latest<\/li>\n\n\n\n<li>llama4:scout<\/li>\n\n\n\n<li>medgemma1.5:4b<\/li>\n\n\n\n<li>medgemma:27b<\/li>\n\n\n\n<li>medgemma:4b<\/li>\n\n\n\n<li>mistral-large:latest<\/li>\n\n\n\n<li>mixtral:8x22b<\/li>\n\n\n\n<li>mxbai-embed-large:latest<\/li>\n\n\n\n<li>nemotron3:33b<\/li>\n\n\n\n<li>qwen2.5:latest<\/li>\n\n\n\n<li>qwen3-coder-next:q8_0<\/li>\n\n\n\n<li>qwen3-coder:latest<\/li>\n\n\n\n<li>qwen3-embedding:latest<\/li>\n\n\n\n<li>qwen3.5:27b<\/li>\n\n\n\n<li>qwen3.5:35b<\/li>\n\n\n\n<li>qwen3.5:9b<\/li>\n\n\n\n<li>qwen3.5:latest<\/li>\n\n\n\n<li>qwen3.6:35b<\/li>\n\n\n\n<li>qwen3:235b<\/li>\n\n\n\n<li>qwen3:32b<\/li>\n\n\n\n<li>qwen3:8b<\/li>\n<\/ul>\n\n\n\n<p>The backend uses vLLM and Ollama to deploy models. You can use the instructions from the\u00a0<a href=\"https:\/\/docs.ollama.com\/api\/openai-compatibility\">Ollama docs<\/a> (give preference to OpenAI endpoints, as all backend servers work with these endpoints) and provide the header \u201cx-api-key\u201d or Bearer Authorization with your RCS API Key, which can be requested\u00a0<a href=\"https:\/\/carleton.ca\/rcs\/llm-access\/\">here<\/a>. If approved, you will receive information about the host and API Key. A complete list of currently deployed models can be found <a href=\"htpps:\/\/rcsllm.carleton.ca\/rcsapi\">here<\/a> (Carleton VPN connection required).<\/p>\n\n\n\n<p><strong class=\"myprefix-text-bold\">The code below is an example of access using the Ollama package in Python:<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import json\nimport requests\nfrom argparse import ArgumentParser, BooleanOptionalAction\n\nDEFAULT_HOST = \"https:\/\/rcsllm.carleton.ca\/rcsapi\"\nparser = ArgumentParser(description=\"RCS API Client Example\")\nparser.add_argument(\"--host\", type=str, help=\"RCS api endpoint\", required=False, default=DEFAULT_HOST)\nparser.add_argument(\"--model\", type=str, help=\"Model to use for requests\", required=False, default=\"gpt-oss:120b\")\nparser.add_argument(\"--prompt\", type=str, help=\"Prompt to send to the model\", required=False, default=\"What is an LLM and how are you compared to other models?\")\nparser.add_argument(\"--stream\", action=BooleanOptionalAction, help=\"Stream the response\", required=False, default=True)\nparser.add_argument(\"--api_key\", type=str, help=\"API Key for authentication\", required=True)\nparser.add_argument(\"--list_models\", action=\"store_true\", help=\"List available models\", required=False, default=False)\n\nif __name__ == \"__main__\":\n    args = parser.parse_args()\n    api_key = args.api_key\n    client = requests.Session()\n    client.headers.update({\"Authorization\": f\"Bearer {api_key}\"})\n    \n    if args.list_models:\n        models = client.get(f\"{args.host}\/v1\/models\").json()\n        print(\"Available models:\")\n        for model in models&#91;'models']:\n            print(f\"- {model&#91;'model']}\")\n        exit(0)\n    stream = args.stream\n    \n    response = client.post(f\"{args.host}\/v1\/chat\/completions\", json={\n        \"model\": args.model,\n        \"messages\": &#91;\n            {\"role\": \"user\", \"content\": args.prompt}\n        ],\n        \"stream\": stream\n    })\n    \n    if stream:\n        for chunk in response.iter_lines():\n            data = chunk.decode('utf-8').strip('data: ')\n            if data:\n                if data == \"&#91;DONE]\":\n                    break\n                data = json.loads(data)\n                if \"reasoning\" in data&#91;\"choices\"]&#91;0]&#91;\"delta\"]:\n                    print(data&#91;\"choices\"]&#91;0]&#91;\"delta\"]&#91;\"reasoning\"], end=\"\", flush=True)\n                if \"content\" in data&#91;\"choices\"]&#91;0]&#91;\"delta\"]:\n                    print(data&#91;\"choices\"]&#91;0]&#91;\"delta\"]&#91;\"content\"], end=\"\", flush=True)\n    else:\n        print(response.json()&#91;'choices']&#91;0]&#91;'message'])<\/code><\/pre>\n\n\n\n<p>If you wish to get access to this resource, fill out the form available&nbsp;<a href=\"https:\/\/itsjira.carleton.ca\/servicedesk\/customer\/portal\/5\/create\/893\">here<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The Research and Computing Services created a Large Language Model server to provide access to the Carleton University community. REST API endpoints can be accessed upon request. There are many models deployed on this server, including: The backend uses vLLM and Ollama to deploy models. You can use the instructions from the\u00a0Ollama docs (give preference [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_acf_changed":false,"_cu_dining_location_slug":"","footnotes":"","_links_to":"","_links_to_target":""},"cu_page_type":[],"class_list":["post-1058","page","type-page","status-publish","hentry"],"acf":{"cu_post_thumbnail":""},"_links":{"self":[{"href":"https:\/\/carleton.ca\/rcs\/wp-json\/wp\/v2\/pages\/1058","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/carleton.ca\/rcs\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/carleton.ca\/rcs\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/carleton.ca\/rcs\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/carleton.ca\/rcs\/wp-json\/wp\/v2\/comments?post=1058"}],"version-history":[{"count":4,"href":"https:\/\/carleton.ca\/rcs\/wp-json\/wp\/v2\/pages\/1058\/revisions"}],"predecessor-version":[{"id":1212,"href":"https:\/\/carleton.ca\/rcs\/wp-json\/wp\/v2\/pages\/1058\/revisions\/1212"}],"wp:attachment":[{"href":"https:\/\/carleton.ca\/rcs\/wp-json\/wp\/v2\/media?parent=1058"}],"wp:term":[{"taxonomy":"cu_page_type","embeddable":true,"href":"https:\/\/carleton.ca\/rcs\/wp-json\/wp\/v2\/cu_page_type?post=1058"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}