# Codex TrueNAS Helper

This project is a collection of scripts, configurations, and applications to manage and enhance a TrueNAS SCALE server, with a special focus on running and interacting with large language models (LLMs) like those powered by `llama.cpp` and `Ollama`.

## Features

*   **`llama.cpp` Wrapper:** A sophisticated wrapper for the `llama.cpp` TrueNAS application that provides:
    *   An OpenAI-compatible API for chat completions and embeddings.
    *   A web-based UI for managing models (listing, downloading).
    *   The ability to hot-swap models without restarting the `llama.cpp` container by interacting with the TrueNAS API.
*   **TrueNAS Inventory:** A snapshot of the TrueNAS server's configuration, including hardware, storage, networking, and running applications.
*   **Automation Scripts:** A set of PowerShell and Python scripts for tasks like deploying the wrapper and testing remote endpoints.
*   **LLM Integration:** Tools and configurations for working with various LLMs.

## Directory Structure

*   `AGENTS.md` & `AGENTS.full.md`: These files contain detailed information and a complete inventory of the TrueNAS server's configuration.
*   `llamaCpp.Wrapper.app/`: A Python-based application that wraps the `llama.cpp` TrueNAS app with an OpenAI-compatible API and a model management UI.
*   `scripts/`: Contains various scripts for deployment, testing, and other tasks.
*   `inventory_raw/`: Raw data dumps from the TrueNAS server, used to generate the inventory in `AGENTS.full.md`.
*   `reports/`: Contains generated reports, test results, and other artifacts.
*   `llamacpp_runs_remote/` & `ollama_runs_remote/`: Logs and results from running LLMs.
*   `modelfiles/`: Modelfiles for different language models.
*   `tests/`: Python tests for the `llamaCpp.Wrapper.app`.

## `llamaCpp.Wrapper.app`

This is the core component of the project. It's a Python application that acts as a proxy to the `llama.cpp` server running on TrueNAS, but with added features.

### Running Locally

1.  Install the required Python packages:
    ```bash
    pip install -r llamaCpp.Wrapper.app/requirements.txt
    ```
2.  Run the application:
    ```bash
    python -m llamaCpp.Wrapper.app.run
    ```
    This will start two web servers: one for the API (default port 9093) and one for the UI (default port 9094).

### Docker (TrueNAS)

The wrapper can be run as a Docker container on TrueNAS. See the `llamaCpp.Wrapper.app/README.md` file for a detailed example of the `docker run` command. The wrapper needs to be configured with the appropriate environment variables to connect to the TrueNAS API and the `llama.cpp` container.

### Model Hot-Swapping

The wrapper can switch models in the `llama.cpp` server by updating the application's command via the TrueNAS API. This is a powerful feature that allows for dynamic model management without manual intervention.

## Scripts

*   `deploy_truenas_wrapper.py`: A Python script to deploy the `llamaCpp.Wrapper.app` to TrueNAS.
*   `remote_wrapper_test.py`: A Python script for testing the remote wrapper.
*   `update_llamacpp_flags.ps1`: A PowerShell script to update the `llama.cpp` flags.
*   `llamacpp_remote_test.ps1` & `ollama_remote_test.ps1`: PowerShell scripts for testing `llama.cpp` and `Ollama` remote endpoints.

## Getting Started

1.  **Explore the Inventory:** Start by reading `AGENTS.md` and `AGENTS.full.md` to understand the TrueNAS server's configuration.
2.  **Set up the Wrapper:** If you want to use the `llama.cpp` wrapper, follow the instructions in `llamaCpp.Wrapper.app/README.md` to run it either locally or as a Docker container on TrueNAS.
3.  **Use the Scripts:** The scripts in the `scripts` directory can be used to automate various tasks.

## Development

The `llamaCpp.Wrapper.app` has a suite of tests located in the `tests/` directory. To run the tests, use `pytest`:

```bash
pytest
```