| Age | Commit message (Collapse) | Author |
|---|
|
Needed this for some LLM projects, basically I need it whenever I want to make
embeddings because it has all of the embedding models.
|
|
I think this is the only "supported" nixos version now. But in any case scipy
seemed to be broken on the older version, and I couldn't build my code that uses
llm. Also, this allows me to get rid of the bild.os-unstable thing for
Beryllium, which was just a sitting timebomb of breaking stuff.
There are a lot of changes here because ruff updated to the very latest, and it
changed some minor lint things. Also with the new nixos I get a proper cgit
module, and some other breaking changes needed fixing.
|
|
Mynion was a prototype and while it was cool and worked well, it is unused and
causing magma to build, which takes forever. I have settled on using ollama for
local inference and a hosted inference API for production.
|
|
Otherwise it might change from underneath me (which it did already once).
|
|
I first tried to make it into a two-level fzf menu, such that selecting a
namespace dropped you into a second fzf interface that let you select the
command you want to run. I did this because, often you want to perform multiple
actions on the same namespace, rather than jump between namespaces. But this
turned out to be unwieldy, so I went back to the single fzf with bindings for
actions, and I just cleaned up the code and tried to make everything easier to
read or use.
|
|
I basically never want to not fix my code from here. Presumably if I'm invoking
lint from ns.sh, then I don't have the file open yet, or I just closed it. So
it's not super helpful to show me a bunch of things I need to fix and then tell
me that I could autofix them but don't actually fix them for me.
|
|
This shouldn't be polluting the main namespace.
|
|
It's just a simple test, but it effectively tests that the `CustomRepl` can be
instantiated for the ns and path. I also copied the unittest recipes I came up
with in the other place I used Python tests so far: Biz/Llamacpp.py.
Also, I'm beginning to see how a Biz/Cli.py module might work. Probably just a
simple abstract base class with move, test, help, and tidy methods, pretty
similar to the Haskell version.
|
|
I think gitlint violations were sneaking through without this.
|
|
This adds a few things I found from [this gist][1], but cleaned up quite a bit I
think, and designed a bit closer to the ghci user experience.
Along the way I figured out what ruff settings will autoformat my imports in one
alphabetized section, rather than splitting it into multiple sections for
builtins and external deps. So I made that change in the whole repo, but there
weren't too many changes.
[1]: https://gist.github.com/aliles/1153926
|
|
This is meant to emulate the `:e` command in ghci.
|
|
The regex would fail if there was text after the `import X`, so `import X as Y`
would fail, and bild wouldn't detect the local import. This adds a simple test
that guards against this regex failure.
|
|
- ns.sh was broken because I never updated the commands to include the .sh
extensions
- changed a few other UI things in ns.sh to support my workflow
- run.sh was broken when the exe name had a hyphen in it, so just use bild
--plan instead of regex
- changed the noqa to include the English name of the ignored rule because
lets be real I'll never remember what A003 means
|
|
Pretty sure I let this expire.
|
|
Default is 3, which is too low, I keep banning myself.
|
|
At some point I guess the activation script changed to no longer include the
domain, so I have to get it form the 'domainname' systemd service.
Also I switched to sshing as $USER instead of root, which just seems like better
security practice. I originally used root because I thought you had to be root
to do nix system stuff, but it turns out that's not the case.
|
|
It's supposedly faster and actually maintained. Also make preview window in
ftags.sh a bit more robust.
|
|
This small Llamacpp.py file is simply intended to test that llama.cpp can build.
This was previously not working, I guess, because the build system doesn't
verify that the final executable has its dependencies set properly in $PATH. Not
sure if it *should* do that verification or not.
Anyway, I rewrote this to actually test if it could call `llama`, and it could
not, because the Python builder needed the rundeps in its propagatedBuildInputs.
That alone makes `llama` available to the final artifact, but the test still
failed. This is because the wrapPythonPrograms function from nixpkgs (which adds
stuff to PATH) is called in postFixup, which happens after installPhase, but
checkPhase happens before installPhase. So I was testing a program that didn't
have PATH set yet.
Moving the test to installCheck fixed this because it runs after the postFixup
phase. I opted to keep the lint/typecheck stuff in the checkPhase because they
don't need any external dependencies, and having those fail earlier is probably
better? Maybe doesn't make a huge difference time-wise but it kinda makes the
intention clearer to be separate, in checkPhase you are checking the code
itself, in installCheck you are including the installation environment as well.
|
|
This is basically exactly the client library that I would write myself. Some
parts of it are still beta quality, but it's the sort of thing that I would
contribute to anyway.
Unfortunately I couldn't get the llm-llama-cpp plugin to work because it depends
on llama-cpp-python which is not packaged for nix and is hard to package because
the upstream project vendors a patched version of llama.cpp. So I'm stuck with
ollama for now, but that's fine because it actually works.
|
|
This superceedes exllama and tabbyAPI which I could never get working fully.
Unfortunately I had to switch to NixOS unstable to get all the Go builder stuff
to work, so this is a cause of yet another version drift, but I guess it's
inevitable and I should just learn to mitigate it with my nixpkgs shenanigans.
|
|
Added docs to core libraries and expanded the Example.hs test to test for all
the things I want to support in a build.
|
|
I actually thought these were turned on, but found that they weren't when I was
investigating Haddock stuff. There aren't many violations, so I just turned them
on and fixed the violations real quick.
|
|
This brings a bunch of improvements. I got rid of some custom packages, I can
now build exllama without using a non-default cuda version. Oh yeah and I get to
use GHC 9.6.2 now, a huge upgrade from 9.4. Unfortunately I also updated ormolu
and some unrelated formatting changed, but that's life I guess.
|
|
Apparently `$COLUMNS` is a POSIX standard, which allows us to set the print
width to however wide the user's terminal is. This is a better UI on both wide
and narrow terminal layouts: on very narrow layouts, the terminal will properly
clear the line instead of doing the wrap-print thing it does when the line
overflows, and on wide layouts you can see more of the log message if you're
curious. This only works if you export `$COLUMNS` though, because bash only sets
the variable in interactive mode, so by default a running program doesn't see it.
<https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html>
|
|
On timeout, bild would leave it's child processes still running, which means
that a long build would continue to go in the background, and I'd have to
manually find and kill it. The fix was simply to use Conduit's
`closeStreamingProcessHandle` function, and make sure that Conduit created a
group for the processes (by default it doesn't).
I tested this by building and running `bild` with a very low timeout (1 minute)
and watching `gotop` to make sure it stopped the active build process. I
couldn't figure out a simple way to test this automatically.
|
|
I guess this was the real cause of the constant reloads, it was getting
annoying.
|
|
I didn't even realize this would happen, but obviously it would. So my CI system
was trying to run tests on all note commits, and throwing weird errors
obviously, and probably recursing into an endless void of git pushes.
|
|
This was silly, maybe I should add a special case to Lint.hs to check these...
|
|
Add ccache support to the OsBase definition, so it's automatically setup for all
machines. Hopefully this can speed up builds and at least prevent the worst case
performance.
Unfortunately, if I modify the stdenv of any upstream targets, the cache is
invalidated and I have to rebuild the whole package. In particular webkitgtk is
a pain to build, and it has crashed my machine a number of times. I was able to
build it once, after enabling zram, which should be enabled anyway like why not?
- https://nixos.wiki/wiki/CCache
- https://github.com/NixOS/nixpkgs/issues/195522#issuecomment-2043999608
- https://github.com/NixOS/nixpkgs/pull/216363#issuecomment-2043998790
|
|
This is mostly helpful when debugging or writing programs, not during usage, but
I shouldn't be suppressing errors anyway.
My first version of this patch used `Biz.Log` instead of `panic`. Unfortunately
this exposed a bug in bild: Control/Concurrent/Go.hs (and others) failed to
build because it imported `Biz.Cli` which imported `Biz.Log`, and bild couldn't
find `Biz/Log.hs`. So apparently my function `detectHaskellImports` doesn't
detect all Haskell imports, somewhere along the lines I cut the graph short
instead of recursing. I guess that will be a project for another day.
|
|
Apparently, docopt only parses short opts if they are separated by a single
space from the long opt. I used two spaces, to make it line up with -h below.
:facepalm: So anyway just flip them, to line up.
|
|
This change was motivated by my testing of tabbyAPI. I kept doing like
`nix-build -A pkgs.tabbyAPI` and I thought, can't bild just do this? So I wrote
a file called TabbyAPI.nix with the following contents::
{ bild }: bild.pkgs.tabbyAPI
and it worked, I just needed this change to Bild.hs to supply the `bild`
argument. The benefit of using bild here is that I can get the logging,
concurrency settings, and linking to _/nix etc all by default. Plus, using a
standalone nix file like TabbyAPI.nix might be a good way to pin some package
in the build system and make sure it continues to build, test, and so on.
Also, thie means I don't sprinkle relative paths to the Bild.nix library
throughout the repo, which is bad practice anyway.
Re: explicitly exposing refernces to stable: This keeps things a bit more tidy
and less confusing when working on the nix library.
|
|
This patch fixes two problems:
1. Apparently the docopt spec wasn't written correctly, I needed to add a
metavar (like N for number) to the spec so that docopt would correctly parse
the command line args. Without this, I had just been using the default 6 and
10 that I hardcoded.
2. I never added the --max-jobs and --cores flags to NixBuild, so anytime I
built a nix target, i.e. any of my NixOS images, it would use all of the
available cores. I think this is part of why I struggled so much to build
webkitgtk: I *thought* I was limiting the resources to prevent an OOM crash,
but I wasn't.
With this fix, I can comfortably limit resources for a build, so I can continue
working while building something in the background.
|
|
Such a simple change but could potentially save seconds on every bild invocation
because I have to go to disk and read the file and so on. I really should have
caught this the first time I wrote it.
|
|
I got annoyed losing my well-crafted commit message one too many times.
|
|
This patch does a few things:
1. Switches from nixpkgs-unstable to nixos-unstable{,-small}, simply because
nixpkgs-unstable is not in cache.nixos.org, but nixos-unstable is, and -small
is the same but requires all tests to pass. So we should prefer
nixos-unstable-small, whenever possible.
2. Reorganizes the nixpkgs import code such that Nixpkgs.nix returns an attrset
of all the nixpkgs that I want to use, rather than putting other nixpkgs
branches into the main one as an overlay. This is much simpler and explicit,
but it meant I had to change a lot of usages throughtout the nix codebase.
3. As a consequence of 2, moves the overlays into separate files so they can be
re-used across nixpkgs branches.
|
|
I tried to add it to Lint.hs but I can't because mypy needs the target's
external libraries in its environment to load and check the types for that
dependency. So instead, I just added a function to the REPL environment that
runs the type checker. I already had mypy in the REPL environment so I must have
started down this path before and just didn't add the REPL tooling.
The automatic typechecking on load feels like Haskell. A previous version of
this patch had just provided a `typecheck()` function to the REPL, but it felt
awkward to type all that out just to check my code after loading it.
I would like to bind a key like ctrl-r or alt-r to the `reload()` function, but
I'm not sure how to do that. I think Python uses GNU readline so there should be
some docs, but a first pass search didn't find what I needed.
|
|
Ruff is faster and if it supports everything that black supports than why not? I
did have to pull in a more recent version from unstable, but that's easy to do
now. And I decided to just go ahead and configure ruff by turning on almost all
checks, which meant I had to fix a whole bunch of things, but I did that and
everything is okay now.
|
|
I always want to push notes on a successful code push, so using the pre-push
hook to do so is perfect. This also expands the pre-push hook to handle
new/deleted branches, configures notes automatically, and finally checks that
commit messages are decent with gitlint, since `git amend` can bypass the git
commit-msg hook.
|
|
Adding some helpful info to the repl startup because I kept forgetting that I
even had this reload function. Also, don't call it 'r' because I often do like
r = some_function()
to store results, and then I can't reload anymore.
|
|
I realized that tidy was doing the same thing as git clean, but git clean can do
it more robust and exactly. So switch to that and add it to the git test config
aliases so I don't accidentally test with old artifacts.
|
|
This started failing recently, I think because my key expired or something.
Anyway I shouldn't be doing network requests in unit tests. So until I figure
out how to design a test suite that can handle network requests in a sensible
way, just skip these tests.
|
|
This seems to be needed for git test to work, specifically because in a new
worktree, git is not configured, and direnv is not loaded to auto-configure
these settings. At least that's as far as I can tell what's going on. In either
case, best not to rely too much on git anyway.
|
|
This seems to be needed for git test to work, in particular the directory inside
the nix sandbox is like '/build/<name-of-checkout-directory>' which, in the case
of git test, is the name of the worktree that git-branchless uses to checkout
the commit under test. This is something like 'test-worktree-2'. So, I can't
hardcode anything there.
|
|
The whole point is to catch errors and write the result to the git notes, if the
process exits then we can't do that.
|
|
Uses git test to run CI on every commit before push.
|
|
It got out of sync with the code even within commit
7597d51ed4c866f596fb690d4d53d70bc01181b4, I dunno how I let that happen.
|
|
Removes my custom llama-cpp build and instead pulls in the upstream build from
nixos-23.11.
|
|
Using the same methodology as in d56f05863d789423ff785cec654155d8495373a9, where
I add additional nixpkgs via an overlay, I added access to nixos-23.11. I also
renamed the nixpkgs-stable dep to be more appropriate, given that it comes from
the nixos-23.05 git branch. I'm nost totally sure about the naming of the attr
in the overlay, maybe it should be 'stable-23_11'? But I feel like for
discoverability purposes it should be the same as the name in deps, so I'll just
stick with that.
|
|
This seems to be a bug in nixos, and disabling it is easy enough and doesn't
seem to break anything.
https://github.com/NixOS/nixpkgs/issues/180175
|
