feat(architect/llm.nix): add OLLAMA_KV_CACHE_TYPE environment variable

- Added `OLLAMA_KV_CACHE_TYPE` with value `"q8_0"` to the environment variables in `llm.nix`
2024-12-08 12:01:01 +00:00 · 2024-12-08 12:01:01 +00:00 · 29c2526e6a
commit 29c2526e6a
parent 109c738594
1 changed files with 1 additions and 0 deletions
--- a/hosts/architect/llm.nix
+++ b/hosts/architect/llm.nix
@ -21,6 +21,7 @@ in
    environmentVariables = {
      OLLAMA_FLASH_ATTENTION = "1";
      OLLAMA_NUM_PARALLEL = "2";
+      OLLAMA_KV_CACHE_TYPE = "q8_0";
    };
  };