summaryrefslogtreecommitdiff
path: root/Biz
AgeCommit message (Collapse)Author
2 daysAdd shebangs and x bit to executablesBen Sima
With run.sh, we can build and run the file in one go. This means we can also use it as an interpreter in a shebang line and properly use the Unix executable bit. This is pretty cool and gives a few advantages: running any executable file is just `exec file.hs` or even `./file.hs`, finding all executables is `fd -t x`, you don't need to specify or know an `out` name to run something, execution of a program is standardized. There is a hack to get this to work. In C and Common Lisp, `#!` is illegal syntax, so I had to use shell syntax to invoke run.sh, call it on the current file, and then exit the shell script. Meanwhile, run.sh takes the file and evals the whole thing, building and running it. As long as either `//` or `;` is a comment character in the target language, then this works. Maybe a better thing to do would be to pre-process the file and remove the `#!` before passing it to the C compiler, like [ryanmjacobs/c][1] and [tcc][2]? However this won't work in Lisp because then I can't just load the file directly into the repl, so maybe the comment hack needs to stay. [1]: https://github.com/ryanmjacobs/c/tree/master [2]: https://repo.or.cz/tinycc.git/blob/HEAD:/tccrun.c
3 daysBuild and deploy storybookBen Sima
I put the storybook into a new Biz.nix deploy target. The idea here is that any Biz/* targets should be hosted by this one VM for simplicity. Over time I can grow this as need be, but this should work to host a few services.
3 daysAsync end-to-end Storybook workingBen Sima
I deleted the tests because they were overspecifying the functionality. My mistake was to try and build out the objects and endpoints before the end-to-end sync thing was fully working. And then I misunderstood how to do async with HTMX, I was overcomplicating it trying to create objects and endpoints for everything instead of just focusing on the HTML that I should be generating. This all just led to a clusterfuck of code doing all the wrong things in the wrong places. So far this is much better architected. And it turns out that using image n-1 with OpenAI's create_variation function doesn't work very well anyway, so I scrapped that too; I'll have to look into different image gen services in the future.
3 daysAdd some mock tests of the Image endpointBen Sima
These were contributed in part by gptme, thanks!
3 daysManage Storybook ImagesBen Sima
This adds the Images endpoint and related functions for loading and saving images to the filesystem. In the view layer, it also loads the images asynchronously using HTMX, so the images get lazy-loaded only when they are done generating.
3 daysPrune unused stuffBen Sima
This was all dead weight, just delete it and move on.
3 daysConvert Biz/Storybook.py to LudicBen Sima
This is basically a full rewrite. I ripped out Flask and rearchitected the whole thing to use fully RESTful resources and endpoints using Ludic. The UI was completely redone to use Ludic's components. I added tests for everything that I reasonably could. This is almost ready for an alpha launch. Before shipping it I still need to: 1. generate images using image n-1 applied to `openai.images.create_variation()` 2. write a nix service, get it on a VM somewhere, I'll probably provision a new VM for this 3. replace the `db` thing with a real sqlite database I only need the first one done to show it to Lia and see if she likes it, that should be completed in a day or two. Then the nix service and deployment won't take long at all. Setting up a sqlite database will be annoying, but that I can't see that actually taking more than 2 days. So max 5 days out from launching this to friends and family.
3 daysImplement storybook prototypeBen Sima
This paritally used gptme to create a storybook generator. The problem I ran into is that gptme doesn't do any architecting or considerations for maintainable code, or even readable code, so it just wrote a long script. I couldn't test it. Also, it didn't actually generate a 10-page story, it generated 10 separate stories. So, I ended up writing it myself and using gptme to fixup TODOs that I wrote along the way.
3 daysTest Python code with bild --testBen Sima
I had forgotten to add this feature, apparently, so bild --test just didn't do the test part.
3 daysRe-namespace some stuff to OmniBen Sima
I was getting confused about what is a product and what is internal infrastructure; I think it is good to keep those things separate. So I moved a bunch of stuff to an Omni namespace, actually most stuff went there. Only things that are explicitly external products are still in the Biz namespace.
3 daysAdd my iPhone keyBen Sima
Hopefully I can use this with gpgme to code remotely.
3 daysUse newer ollama in system serviceBen Sima
Ollama releases often, and nixos-24.05 has a very old version, it was preventing me from doing work. Here I'm putting the newer nixos unstable as an overlay into the 24.05 version so I can backpull versions as needed.
3 daysDisable nixpertBen Sima
This domain is also expired... dang I really need to cleanup my dead projects.
3 daysRemove dragons.dev domain nameBen Sima
Yeah I let it expire... I am basically giving up on the idea but I'll keep the code around for now until I have time to comb through it.
3 daysEnable jack and wireplumberBen Sima
Trying to enable reaper and my microphone.
3 daysEnable efnet and zeronode irc networksBen Sima
I just wanted to connect to them and needed to fix these configs.
3 daysEnable the zsa voyager keyboardBen Sima
The voyager needed some special udev rules, and a program to do the flashing. I also made the barrier port explicit by moving to Ports.nix.
3 daysDon't auto-suspend-to-RAMBen Sima
I don't want the machine to suspend after some time period, instead I want to manually turn off the monitors.
3 daysFollowup changes to ctagsBen Sima
These were just hanging around as I was doing other things. Included in this commit are the following: - Configure it in .ctags.d to only index the languages I use - Add support for sources listed in Sources.json - Fix broken link to inspekt3d repo
3 daysSet explicit srcs in hand-written bild buildBen Sima
I'm not sure when `lib.fileset` was introduced, but it allows me to easily create a source that can be unpacked by the `unpackPhase` which also maintains the directory structure. This means I can pass a subset of the whole repo to the builder, and changing an unrelated file (outside of that subset) won't cause a rebuild of bild, which wasn't the case before.
3 daysRewrite mktags.sh to MakeTags.pyBen Sima
This makes tags for all third-party dependencies, and uses universal-ctags instead of fast-tags. It's so nice having tags for /everything/. It's also pretty fast, and caches the result for third-party deps anyway. Oh yeah this also added an explicit list of third-party Python packages. I need to filter the entire pkgset down to just the ones I'm using in order to make tags for them.
3 daysAdd llm-sentence-transformersBen Sima
Needed this for some LLM projects, basically I need it whenever I want to make embeddings because it has all of the embedding models.
3 daysUpgrade to nixos-24.05Ben Sima
I think this is the only "supported" nixos version now. But in any case scipy seemed to be broken on the older version, and I couldn't build my code that uses llm. Also, this allows me to get rid of the bild.os-unstable thing for Beryllium, which was just a sitting timebomb of breaking stuff. There are a lot of changes here because ruff updated to the very latest, and it changed some minor lint things. Also with the new nixos I get a proper cgit module, and some other breaking changes needed fixing.
3 daysDelete Biz/Mynion.py and exllamaBen Sima
Mynion was a prototype and while it was cool and worked well, it is unused and causing magma to build, which takes forever. I have settled on using ollama for local inference and a hosted inference API for production.
2024-05-30Pin this url to a hash instead of masterBen Sima
Otherwise it might change from underneath me (which it did already once).
2024-05-29Rewrite ns.shBen Sima
I first tried to make it into a two-level fzf menu, such that selecting a namespace dropped you into a second fzf interface that let you select the command you want to run. I did this because, often you want to perform multiple actions on the same namespace, rather than jump between namespaces. But this turned out to be unwieldy, so I went back to the single fzf with bindings for actions, and I just cleaned up the code and tried to make everything easier to read or use.
2024-05-23Lint fix command in ns.shBen Sima
I basically never want to not fix my code from here. Presumably if I'm invoking lint from ns.sh, then I don't have the file open yet, or I just closed it. So it's not super helpful to show me a bunch of things I need to fix and then tell me that I could autofix them but don't actually fix them for me.
2024-05-23Put __pycache__ in _/intBen Sima
This shouldn't be polluting the main namespace.
2024-05-23Add test to Biz/Repl.pyBen Sima
It's just a simple test, but it effectively tests that the `CustomRepl` can be instantiated for the ns and path. I also copied the unittest recipes I came up with in the other place I used Python tests so far: Biz/Llamacpp.py. Also, I'm beginning to see how a Biz/Cli.py module might work. Probably just a simple abstract base class with move, test, help, and tidy methods, pretty similar to the Haskell version.
2024-05-21Set pre-push hook to fail if subcommand failsBen Sima
I think gitlint violations were sneaking through without this.
2024-05-20Greatly expand Repl.pyBen Sima
This adds a few things I found from [this gist][1], but cleaned up quite a bit I think, and designed a bit closer to the ghci user experience. Along the way I figured out what ruff settings will autoformat my imports in one alphabetized section, rather than splitting it into multiple sections for builtins and external deps. So I made that change in the whole repo, but there weren't too many changes. [1]: https://gist.github.com/aliles/1153926
2024-05-20Add edit capabilities to Python replBen Sima
This is meant to emulate the `:e` command in ghci.
2024-05-20Detect Python `import as` statementsBen Sima
The regex would fail if there was text after the `import X`, so `import X as Y` would fail, and bild wouldn't detect the local import. This adds a simple test that guards against this regex failure.
2024-05-20Minor Ide changes & fixesBen Sima
- ns.sh was broken because I never updated the commands to include the .sh extensions - changed a few other UI things in ns.sh to support my workflow - run.sh was broken when the exe name had a hyphen in it, so just use bild --plan instead of regex - changed the noqa to include the English name of the ignored rule because lets be real I'll never remember what A003 means
2024-05-20Remove nixpert.chat domainBen Sima
Pretty sure I let this expire.
2024-05-20Increase fail2ban maxretryBen Sima
Default is 3, which is too low, I keep banning myself.
2024-05-20Fix push.sh, act as $USER instead of rootBen Sima
At some point I guess the activation script changed to no longer include the domain, so I have to get it form the 'domainname' systemd service. Also I switched to sshing as $USER instead of root, which just seems like better security practice. I originally used root because I thought you had to be root to do nix system stuff, but it turns out that's not the case.
2024-05-20Switch to universal-ctagsBen Sima
It's supposedly faster and actually maintained. Also make preview window in ftags.sh a bit more robust.
2024-05-20Test that llama-cpp is buildableBen Sima
This small Llamacpp.py file is simply intended to test that llama.cpp can build. This was previously not working, I guess, because the build system doesn't verify that the final executable has its dependencies set properly in $PATH. Not sure if it *should* do that verification or not. Anyway, I rewrote this to actually test if it could call `llama`, and it could not, because the Python builder needed the rundeps in its propagatedBuildInputs. That alone makes `llama` available to the final artifact, but the test still failed. This is because the wrapPythonPrograms function from nixpkgs (which adds stuff to PATH) is called in postFixup, which happens after installPhase, but checkPhase happens before installPhase. So I was testing a program that didn't have PATH set yet. Moving the test to installCheck fixed this because it runs after the postFixup phase. I opted to keep the lint/typecheck stuff in the checkPhase because they don't need any external dependencies, and having those fail earlier is probably better? Maybe doesn't make a huge difference time-wise but it kinda makes the intention clearer to be separate, in checkPhase you are checking the code itself, in installCheck you are including the installation environment as well.
2024-05-20Add simonw/llm as cli/library client for running LLMsBen Sima
This is basically exactly the client library that I would write myself. Some parts of it are still beta quality, but it's the sort of thing that I would contribute to anyway. Unfortunately I couldn't get the llm-llama-cpp plugin to work because it depends on llama-cpp-python which is not packaged for nix and is hard to package because the upstream project vendors a patched version of llama.cpp. So I'm stuck with ollama for now, but that's fine because it actually works.
2024-05-20Setup ollama API serviceBen Sima
This superceedes exllama and tabbyAPI which I could never get working fully. Unfortunately I had to switch to NixOS unstable to get all the Go builder stuff to work, so this is a cause of yet another version drift, but I guess it's inevitable and I should just learn to mitigate it with my nixpkgs shenanigans.
2024-05-09Improve Haskell documentation and bild testBen Sima
Added docs to core libraries and expanded the Example.hs test to test for all the things I want to support in a build.
2024-05-09Turn on more Haskell errorsBen Sima
I actually thought these were turned on, but found that they weren't when I was investigating Haddock stuff. There aren't many violations, so I just turned them on and fixed the violations real quick.
2024-05-09Switch to nixpkgs-23.11, ghc 9.6.3Ben Sima
This brings a bunch of improvements. I got rid of some custom packages, I can now build exllama without using a non-default cuda version. Oh yeah and I get to use GHC 9.6.2 now, a huge upgrade from 9.4. Unfortunately I also updated ormolu and some unrelated formatting changed, but that's life I guess.
2024-04-15Wrap bild log at the terminal-supplied widthBen Sima
Apparently `$COLUMNS` is a POSIX standard, which allows us to set the print width to however wide the user's terminal is. This is a better UI on both wide and narrow terminal layouts: on very narrow layouts, the terminal will properly clear the line instead of doing the wrap-print thing it does when the line overflows, and on wide layouts you can see more of the log message if you're curious. This only works if you export `$COLUMNS` though, because bash only sets the variable in interactive mode, so by default a running program doesn't see it. <https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html>
2024-04-15Cleanup build processes on timeoutBen Sima
On timeout, bild would leave it's child processes still running, which means that a long build would continue to go in the background, and I'd have to manually find and kill it. The fix was simply to use Conduit's `closeStreamingProcessHandle` function, and make sure that Conduit created a group for the processes (by default it doesn't). I tested this by building and running `bild` with a very low timeout (1 minute) and watching `gotop` to make sure it stopped the active build process. I couldn't figure out a simple way to test this automatically.
2024-04-11Don't reload direnv every checkoutBen Sima
I guess this was the real cause of the constant reloads, it was getting annoying.
2024-04-11Don't run the pre-push script when pushing git notesBen Sima
I didn't even realize this would happen, but obviously it would. So my CI system was trying to run tests on all note commits, and throwing weird errors obviously, and probably recursing into an endless void of git pushes.
2024-04-11Fix syntax error in pre-push hookBen Sima
This was silly, maybe I should add a special case to Lint.hs to check these...
2024-04-11Use ccache in buildsBen Sima
Add ccache support to the OsBase definition, so it's automatically setup for all machines. Hopefully this can speed up builds and at least prevent the worst case performance. Unfortunately, if I modify the stdenv of any upstream targets, the cache is invalidated and I have to rebuild the whole package. In particular webkitgtk is a pain to build, and it has crashed my machine a number of times. I was able to build it once, after enabling zram, which should be enabled anyway like why not? - https://nixos.wiki/wiki/CCache - https://github.com/NixOS/nixpkgs/issues/195522#issuecomment-2043999608 - https://github.com/NixOS/nixpkgs/pull/216363#issuecomment-2043998790