summaryrefslogtreecommitdiff
path: root/Biz
AgeCommit message (Collapse)Author
6 daysDon't auto-suspend-to-RAMBen Sima
I don't want the machine to suspend after some time period, instead I want to manually turn off the monitors.
6 daysFollowup changes to ctagsBen Sima
These were just hanging around as I was doing other things. Included in this commit are the following: - Configure it in .ctags.d to only index the languages I use - Add support for sources listed in Sources.json - Fix broken link to inspekt3d repo
6 daysSet explicit srcs in hand-written bild buildBen Sima
I'm not sure when `lib.fileset` was introduced, but it allows me to easily create a source that can be unpacked by the `unpackPhase` which also maintains the directory structure. This means I can pass a subset of the whole repo to the builder, and changing an unrelated file (outside of that subset) won't cause a rebuild of bild, which wasn't the case before.
6 daysRewrite mktags.sh to MakeTags.pyBen Sima
This makes tags for all third-party dependencies, and uses universal-ctags instead of fast-tags. It's so nice having tags for /everything/. It's also pretty fast, and caches the result for third-party deps anyway. Oh yeah this also added an explicit list of third-party Python packages. I need to filter the entire pkgset down to just the ones I'm using in order to make tags for them.
6 daysAdd llm-sentence-transformersBen Sima
Needed this for some LLM projects, basically I need it whenever I want to make embeddings because it has all of the embedding models.
6 daysUpgrade to nixos-24.05Ben Sima
I think this is the only "supported" nixos version now. But in any case scipy seemed to be broken on the older version, and I couldn't build my code that uses llm. Also, this allows me to get rid of the bild.os-unstable thing for Beryllium, which was just a sitting timebomb of breaking stuff. There are a lot of changes here because ruff updated to the very latest, and it changed some minor lint things. Also with the new nixos I get a proper cgit module, and some other breaking changes needed fixing.
6 daysDelete Biz/Mynion.py and exllamaBen Sima
Mynion was a prototype and while it was cool and worked well, it is unused and causing magma to build, which takes forever. I have settled on using ollama for local inference and a hosted inference API for production.
2024-05-30Pin this url to a hash instead of masterBen Sima
Otherwise it might change from underneath me (which it did already once).
2024-05-29Rewrite ns.shBen Sima
I first tried to make it into a two-level fzf menu, such that selecting a namespace dropped you into a second fzf interface that let you select the command you want to run. I did this because, often you want to perform multiple actions on the same namespace, rather than jump between namespaces. But this turned out to be unwieldy, so I went back to the single fzf with bindings for actions, and I just cleaned up the code and tried to make everything easier to read or use.
2024-05-23Lint fix command in ns.shBen Sima
I basically never want to not fix my code from here. Presumably if I'm invoking lint from ns.sh, then I don't have the file open yet, or I just closed it. So it's not super helpful to show me a bunch of things I need to fix and then tell me that I could autofix them but don't actually fix them for me.
2024-05-23Put __pycache__ in _/intBen Sima
This shouldn't be polluting the main namespace.
2024-05-23Add test to Biz/Repl.pyBen Sima
It's just a simple test, but it effectively tests that the `CustomRepl` can be instantiated for the ns and path. I also copied the unittest recipes I came up with in the other place I used Python tests so far: Biz/Llamacpp.py. Also, I'm beginning to see how a Biz/Cli.py module might work. Probably just a simple abstract base class with move, test, help, and tidy methods, pretty similar to the Haskell version.
2024-05-21Set pre-push hook to fail if subcommand failsBen Sima
I think gitlint violations were sneaking through without this.
2024-05-20Greatly expand Repl.pyBen Sima
This adds a few things I found from [this gist][1], but cleaned up quite a bit I think, and designed a bit closer to the ghci user experience. Along the way I figured out what ruff settings will autoformat my imports in one alphabetized section, rather than splitting it into multiple sections for builtins and external deps. So I made that change in the whole repo, but there weren't too many changes. [1]: https://gist.github.com/aliles/1153926
2024-05-20Add edit capabilities to Python replBen Sima
This is meant to emulate the `:e` command in ghci.
2024-05-20Detect Python `import as` statementsBen Sima
The regex would fail if there was text after the `import X`, so `import X as Y` would fail, and bild wouldn't detect the local import. This adds a simple test that guards against this regex failure.
2024-05-20Minor Ide changes & fixesBen Sima
- ns.sh was broken because I never updated the commands to include the .sh extensions - changed a few other UI things in ns.sh to support my workflow - run.sh was broken when the exe name had a hyphen in it, so just use bild --plan instead of regex - changed the noqa to include the English name of the ignored rule because lets be real I'll never remember what A003 means
2024-05-20Remove nixpert.chat domainBen Sima
Pretty sure I let this expire.
2024-05-20Increase fail2ban maxretryBen Sima
Default is 3, which is too low, I keep banning myself.
2024-05-20Fix push.sh, act as $USER instead of rootBen Sima
At some point I guess the activation script changed to no longer include the domain, so I have to get it form the 'domainname' systemd service. Also I switched to sshing as $USER instead of root, which just seems like better security practice. I originally used root because I thought you had to be root to do nix system stuff, but it turns out that's not the case.
2024-05-20Switch to universal-ctagsBen Sima
It's supposedly faster and actually maintained. Also make preview window in ftags.sh a bit more robust.
2024-05-20Test that llama-cpp is buildableBen Sima
This small Llamacpp.py file is simply intended to test that llama.cpp can build. This was previously not working, I guess, because the build system doesn't verify that the final executable has its dependencies set properly in $PATH. Not sure if it *should* do that verification or not. Anyway, I rewrote this to actually test if it could call `llama`, and it could not, because the Python builder needed the rundeps in its propagatedBuildInputs. That alone makes `llama` available to the final artifact, but the test still failed. This is because the wrapPythonPrograms function from nixpkgs (which adds stuff to PATH) is called in postFixup, which happens after installPhase, but checkPhase happens before installPhase. So I was testing a program that didn't have PATH set yet. Moving the test to installCheck fixed this because it runs after the postFixup phase. I opted to keep the lint/typecheck stuff in the checkPhase because they don't need any external dependencies, and having those fail earlier is probably better? Maybe doesn't make a huge difference time-wise but it kinda makes the intention clearer to be separate, in checkPhase you are checking the code itself, in installCheck you are including the installation environment as well.
2024-05-20Add simonw/llm as cli/library client for running LLMsBen Sima
This is basically exactly the client library that I would write myself. Some parts of it are still beta quality, but it's the sort of thing that I would contribute to anyway. Unfortunately I couldn't get the llm-llama-cpp plugin to work because it depends on llama-cpp-python which is not packaged for nix and is hard to package because the upstream project vendors a patched version of llama.cpp. So I'm stuck with ollama for now, but that's fine because it actually works.
2024-05-20Setup ollama API serviceBen Sima
This superceedes exllama and tabbyAPI which I could never get working fully. Unfortunately I had to switch to NixOS unstable to get all the Go builder stuff to work, so this is a cause of yet another version drift, but I guess it's inevitable and I should just learn to mitigate it with my nixpkgs shenanigans.
2024-05-09Improve Haskell documentation and bild testBen Sima
Added docs to core libraries and expanded the Example.hs test to test for all the things I want to support in a build.
2024-05-09Turn on more Haskell errorsBen Sima
I actually thought these were turned on, but found that they weren't when I was investigating Haddock stuff. There aren't many violations, so I just turned them on and fixed the violations real quick.
2024-05-09Switch to nixpkgs-23.11, ghc 9.6.3Ben Sima
This brings a bunch of improvements. I got rid of some custom packages, I can now build exllama without using a non-default cuda version. Oh yeah and I get to use GHC 9.6.2 now, a huge upgrade from 9.4. Unfortunately I also updated ormolu and some unrelated formatting changed, but that's life I guess.
2024-04-15Wrap bild log at the terminal-supplied widthBen Sima
Apparently `$COLUMNS` is a POSIX standard, which allows us to set the print width to however wide the user's terminal is. This is a better UI on both wide and narrow terminal layouts: on very narrow layouts, the terminal will properly clear the line instead of doing the wrap-print thing it does when the line overflows, and on wide layouts you can see more of the log message if you're curious. This only works if you export `$COLUMNS` though, because bash only sets the variable in interactive mode, so by default a running program doesn't see it. <https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html>
2024-04-15Cleanup build processes on timeoutBen Sima
On timeout, bild would leave it's child processes still running, which means that a long build would continue to go in the background, and I'd have to manually find and kill it. The fix was simply to use Conduit's `closeStreamingProcessHandle` function, and make sure that Conduit created a group for the processes (by default it doesn't). I tested this by building and running `bild` with a very low timeout (1 minute) and watching `gotop` to make sure it stopped the active build process. I couldn't figure out a simple way to test this automatically.
2024-04-11Don't reload direnv every checkoutBen Sima
I guess this was the real cause of the constant reloads, it was getting annoying.
2024-04-11Don't run the pre-push script when pushing git notesBen Sima
I didn't even realize this would happen, but obviously it would. So my CI system was trying to run tests on all note commits, and throwing weird errors obviously, and probably recursing into an endless void of git pushes.
2024-04-11Fix syntax error in pre-push hookBen Sima
This was silly, maybe I should add a special case to Lint.hs to check these...
2024-04-11Use ccache in buildsBen Sima
Add ccache support to the OsBase definition, so it's automatically setup for all machines. Hopefully this can speed up builds and at least prevent the worst case performance. Unfortunately, if I modify the stdenv of any upstream targets, the cache is invalidated and I have to rebuild the whole package. In particular webkitgtk is a pain to build, and it has crashed my machine a number of times. I was able to build it once, after enabling zram, which should be enabled anyway like why not? - https://nixos.wiki/wiki/CCache - https://github.com/NixOS/nixpkgs/issues/195522#issuecomment-2043999608 - https://github.com/NixOS/nixpkgs/pull/216363#issuecomment-2043998790
2024-04-10Report cli errors on runBen Sima
This is mostly helpful when debugging or writing programs, not during usage, but I shouldn't be suppressing errors anyway. My first version of this patch used `Biz.Log` instead of `panic`. Unfortunately this exposed a bug in bild: Control/Concurrent/Go.hs (and others) failed to build because it imported `Biz.Cli` which imported `Biz.Log`, and bild couldn't find `Biz/Log.hs`. So apparently my function `detectHaskellImports` doesn't detect all Haskell imports, somewhere along the lines I cut the graph short instead of recursing. I guess that will be a project for another day.
2024-04-10Fix --fix/-f in Lint.hsBen Sima
Apparently, docopt only parses short opts if they are separated by a single space from the long opt. I used two spaces, to make it line up with -h below. :facepalm: So anyway just flip them, to line up.
2024-04-10Explicitly pass Bild.nix to nix builds & expose references to stableBen Sima
This change was motivated by my testing of tabbyAPI. I kept doing like `nix-build -A pkgs.tabbyAPI` and I thought, can't bild just do this? So I wrote a file called TabbyAPI.nix with the following contents:: { bild }: bild.pkgs.tabbyAPI and it worked, I just needed this change to Bild.hs to supply the `bild` argument. The benefit of using bild here is that I can get the logging, concurrency settings, and linking to _/nix etc all by default. Plus, using a standalone nix file like TabbyAPI.nix might be a good way to pin some package in the build system and make sure it continues to build, test, and so on. Also, thie means I don't sprinkle relative paths to the Bild.nix library throughout the repo, which is bad practice anyway. Re: explicitly exposing refernces to stable: This keeps things a bit more tidy and less confusing when working on the nix library.
2024-04-10Fix --cpus and --jobsBen Sima
This patch fixes two problems: 1. Apparently the docopt spec wasn't written correctly, I needed to add a metavar (like N for number) to the spec so that docopt would correctly parse the command line args. Without this, I had just been using the default 6 and 10 that I hardcoded. 2. I never added the --max-jobs and --cores flags to NixBuild, so anytime I built a nix target, i.e. any of my NixOS images, it would use all of the available cores. I think this is part of why I struggled so much to build webkitgtk: I *thought* I was limiting the resources to prevent an OOM crash, but I wasn't. With this fix, I can comfortably limit resources for a build, so I can continue working while building something in the background.
2024-04-10Don't analyze namespaces we can't buildBen Sima
Such a simple change but could potentially save seconds on every bild invocation because I have to go to disk and read the file and so on. I really should have caught this the first time I wrote it.
2024-04-10Save commit message if gitlint failsBen Sima
I got annoyed losing my well-crafted commit message one too many times.
2024-04-10Reorganize and update nixpkgs upstreamsBen Sima
This patch does a few things: 1. Switches from nixpkgs-unstable to nixos-unstable{,-small}, simply because nixpkgs-unstable is not in cache.nixos.org, but nixos-unstable is, and -small is the same but requires all tests to pass. So we should prefer nixos-unstable-small, whenever possible. 2. Reorganizes the nixpkgs import code such that Nixpkgs.nix returns an attrset of all the nixpkgs that I want to use, rather than putting other nixpkgs branches into the main one as an overlay. This is much simpler and explicit, but it meant I had to change a lot of usages throughtout the nix codebase. 3. As a consequence of 2, moves the overlays into separate files so they can be re-used across nixpkgs branches.
2024-04-10Add mypy to Python REPLBen Sima
I tried to add it to Lint.hs but I can't because mypy needs the target's external libraries in its environment to load and check the types for that dependency. So instead, I just added a function to the REPL environment that runs the type checker. I already had mypy in the REPL environment so I must have started down this path before and just didn't add the REPL tooling. The automatic typechecking on load feels like Haskell. A previous version of this patch had just provided a `typecheck()` function to the REPL, but it felt awkward to type all that out just to check my code after loading it. I would like to bind a key like ctrl-r or alt-r to the `reload()` function, but I'm not sure how to do that. I think Python uses GNU readline so there should be some docs, but a first pass search didn't find what I needed.
2024-04-10Switch from black to ruff formatBen Sima
Ruff is faster and if it supports everything that black supports than why not? I did have to pull in a more recent version from unstable, but that's easy to do now. And I decided to just go ahead and configure ruff by turning on almost all checks, which meant I had to fix a whole bunch of things, but I did that and everything is okay now.
2024-04-10Push ci notes automaticallyBen Sima
I always want to push notes on a successful code push, so using the pre-push hook to do so is perfect. This also expands the pre-push hook to handle new/deleted branches, configures notes automatically, and finally checks that commit messages are decent with gitlint, since `git amend` can bypass the git commit-msg hook.
2024-04-10Improvements to the Python replBen Sima
Adding some helpful info to the repl startup because I kept forgetting that I even had this reload function. Also, don't call it 'r' because I often do like r = some_function() to store results, and then I can't reload anymore.
2024-04-09Replace tidy.sh with git clean, add it to ciBen Sima
I realized that tidy was doing the same thing as git clean, but git clean can do it more robust and exactly. So switch to that and add it to the git test config aliases so I don't accidentally test with old artifacts.
2024-04-09Skip tests that require networkBen Sima
This started failing recently, I think because my key expired or something. Anyway I shouldn't be doing network requests in unit tests. So until I figure out how to design a test suite that can handle network requests in a sensible way, just skip these tests.
2024-04-03Hardcode hooksPath in bildBen Sima
This seems to be needed for git test to work, specifically because in a new worktree, git is not configured, and direnv is not loaded to auto-configure these settings. At least that's as far as I can tell what's going on. In either case, best not to rely too much on git anyway.
2024-04-03Set CODEROOT dynamically in nix buildsBen Sima
This seems to be needed for git test to work, in particular the directory inside the nix sandbox is like '/build/<name-of-checkout-directory>' which, in the case of git test, is the name of the worktree that git-branchless uses to checkout the commit under test. This is something like 'test-worktree-2'. So, I can't hardcode anything there.
2024-04-03Don't exit CI script on failureBen Sima
The whole point is to catch errors and write the result to the git notes, if the process exits then we can't do that.
2024-04-03Enable CI for every commitBen Sima
Uses git test to run CI on every commit before push.