rushabh/codex_truenas_helper
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Enable dual-GPU in RAG modelfile and add API key to upload script

Browse Source
This commit is contained in:
Rushabh Gosar
2026-01-07 18:31:14 -08:00
parent 21ee6be98f
commit 559593b301
2 changed files with 43 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
rag_modelfiles/gpt-oss-rag.Modelfile
View File

@@ -3,6 +3,10 @@ FROM gpt-oss:20b
# Set the system prompt
SYSTEM """You are a specialized assistant for the rushg.me knowledge base. Your goal is to answer questions based *only* on the context provided to you. If the information needed to answer the question is not in the context, you must state that you do not have enough information to answer. Do not use any of your prior knowledge or external information."""
# --- GPU Splitting Configuration ---
# This parameter tells Ollama how many layers to put on each GPU.
PARAMETER num_gpu 2
# Set the template for how the prompt will be structured
TEMPLATE """{{- if .System }}
### System: