diff options
| author | Ben Sima <ben@bsima.me> | 2024-05-14 09:35:45 -0400 |
|---|---|---|
| committer | Ben Sima <ben@bsima.me> | 2024-05-20 22:15:49 -0400 |
| commit | 2d33aa547ff6a516c90ca2b47b13e2add200583a (patch) | |
| tree | 8d4941699982c59c6430f4b9a629b8ea91245bb1 /Biz/Bild.nix | |
| parent | cceefa62d147594d43478e398bbaa9c630670935 (diff) | |
Add simonw/llm as cli/library client for running LLMs
This is basically exactly the client library that I would write myself. Some
parts of it are still beta quality, but it's the sort of thing that I would
contribute to anyway.
Unfortunately I couldn't get the llm-llama-cpp plugin to work because it depends
on llama-cpp-python which is not packaged for nix and is hard to package because
the upstream project vendors a patched version of llama.cpp. So I'm stuck with
ollama for now, but that's fine because it actually works.
Diffstat (limited to 'Biz/Bild.nix')
| -rw-r--r-- | Biz/Bild.nix | 3 |
1 files changed, 3 insertions, 0 deletions
diff --git a/Biz/Bild.nix b/Biz/Bild.nix index ec049d8..894ebe6 100644 --- a/Biz/Bild.nix +++ b/Biz/Bild.nix @@ -82,6 +82,8 @@ let inherit bat bc cmark ctags deadnix fd figlet fzf git git-branchless gitlint guile hlint indent jq lolcat mypy nixfmt ormolu pkg-config ripgrep rustc tree wemux; + llm = nixpkgs.nixos-unstable-small.llm.withPlugins + [ nixpkgs.nixos-unstable-small.python3.pkgs.llm-ollama ]; ollama = nixpkgs.nixos-unstable-small.ollama; ruff = nixpkgs.nixos-unstable-small.ruff; shellcheck = nixpkgs.nixos-unstable-small.shellcheck; @@ -186,6 +188,7 @@ let gitlint jq lolcat + llm ormolu ripgrep stable.haskell.packages.${constants.ghcCompiler}.fast-tags |