| Age | Commit message (Collapse) | Author |
|---|
|
Add ccache support to the OsBase definition, so it's automatically setup for all
machines. Hopefully this can speed up builds and at least prevent the worst case
performance.
Unfortunately, if I modify the stdenv of any upstream targets, the cache is
invalidated and I have to rebuild the whole package. In particular webkitgtk is
a pain to build, and it has crashed my machine a number of times. I was able to
build it once, after enabling zram, which should be enabled anyway like why not?
- https://nixos.wiki/wiki/CCache
- https://github.com/NixOS/nixpkgs/issues/195522#issuecomment-2043999608
- https://github.com/NixOS/nixpkgs/pull/216363#issuecomment-2043998790
|
|
This change was motivated by my testing of tabbyAPI. I kept doing like
`nix-build -A pkgs.tabbyAPI` and I thought, can't bild just do this? So I wrote
a file called TabbyAPI.nix with the following contents::
{ bild }: bild.pkgs.tabbyAPI
and it worked, I just needed this change to Bild.hs to supply the `bild`
argument. The benefit of using bild here is that I can get the logging,
concurrency settings, and linking to _/nix etc all by default. Plus, using a
standalone nix file like TabbyAPI.nix might be a good way to pin some package
in the build system and make sure it continues to build, test, and so on.
Also, thie means I don't sprinkle relative paths to the Bild.nix library
throughout the repo, which is bad practice anyway.
Re: explicitly exposing refernces to stable: This keeps things a bit more tidy
and less confusing when working on the nix library.
|
|
This patch does a few things:
1. Switches from nixpkgs-unstable to nixos-unstable{,-small}, simply because
nixpkgs-unstable is not in cache.nixos.org, but nixos-unstable is, and -small
is the same but requires all tests to pass. So we should prefer
nixos-unstable-small, whenever possible.
2. Reorganizes the nixpkgs import code such that Nixpkgs.nix returns an attrset
of all the nixpkgs that I want to use, rather than putting other nixpkgs
branches into the main one as an overlay. This is much simpler and explicit,
but it meant I had to change a lot of usages throughtout the nix codebase.
3. As a consequence of 2, moves the overlays into separate files so they can be
re-used across nixpkgs branches.
|
|
Ruff is faster and if it supports everything that black supports than why not? I
did have to pull in a more recent version from unstable, but that's easy to do
now. And I decided to just go ahead and configure ruff by turning on almost all
checks, which meant I had to fix a whole bunch of things, but I did that and
everything is okay now.
|
|
This seems to be needed for git test to work, in particular the directory inside
the nix sandbox is like '/build/<name-of-checkout-directory>' which, in the case
of git test, is the name of the worktree that git-branchless uses to checkout
the commit under test. This is something like 'test-worktree-2'. So, I can't
hardcode anything there.
|
|
nixfmt is the soon-to-be official formatter for Nix code, as per the NixOS
GitHub group. So I figure I should just adopt it without worrying too much about
the specifics of the formatting. I just formatted everything in one go, hence
the huge diff, oh well.
|
|
By default, nix will use the maximum amount of cores available to the machine.
On my machine it was maxxing out the CPUs and then actually running out of RAM
when compiling JavaScriptCore and literally shutting down my machine. So, I need
to be able to control the concurrency and parallelism.
The default settings I chose should reserve 4 cores for the user.
I also changed --json to --plan because -j makes more sense for --jobs, as its
used this way in other tools like make and nix-build.
|
|
After using this for a while, I've decided that git-branchless will be the
default tool for doing trunk-based development in the omnirepo.
|
|
Getting beryllium setup with the omnirepo showed that a few things
needed fixed.
|
|
Previously, if there was a problem with the inputs and bild failed to
determine the namespace, 'fromPath' would return 'Nothing' and then
'catMaybes' would drop the error-causing input altogether. In the one
time that I had a bad input, this made debugging incredibly difficult.
It's always a bad idea to swallow errors silently, so instead lets just
kill the program if we have bad inputs.
|
|
This allows me to specify runtime dependencies, not just system or
language deps.
|
|
BIZ_ROOT was too specific. CODEROOT allows for other (non-biz) projects to live
in the root of the repo. I didn't want to call it GIT_ROOT because maybe someday
I won't want to use git. But I'll never not use code.
|
|
Previously I would lint every file individually, in serial. This took
forever. Now I group the files by extension (by first getting the Namespace) and
run each linter on all relevant files at once. This is so much faster its
stupid.
Also I added formatters back into the dev env because my editor needs them to
autoformat.
|
|
Ruff is like a million times faster, and I mostly ignored pylint's suggestions
anyway.
I also took this opportunity to move lint tools into a runtime dependency on
Lint.hs, which meant adding a wrapper to the Haskell builder, which was easy
enough. This paves the way for proper rundeps in bild.
|
|
Turns out that gitlint by default enforces the exact commit-msg format that I
like to use. I'm enabling this because even I write poor commit messages
sometimes, and looking back on my commits from even a few days ago is sometimes
not very helpful.
I also made some minor comment and nix changes that I noticed while reviewing my
work from the last few days.
|
|
Lots of changes here but the code is much improved. The nix code is clearer and
structured better.
The Haskell code improved in response to the nix changes. I needed to use a
qualified path instead of the abspath because the BIZ_ROOT changes based on
whether bild runs in nix or runs in the user environment.
Rather than passing every argument into Builder.nix, now I just pass the json
from bild and deconstruct it in nix. This is obviously a much better design and
it only came to be after sleeping on it the other night.
|
|
This is working with libsodium as an example. Its unfortunate that we need the
extra ':arg -lsodium' but how else can I get the name of the library for
linking? Is that something in the nix attr metadata? Anyway, an optimization for
another day.
|
|
Not getting deps yet but thats okay, I basically need to do a bunch of annoying
nix work to get rustPackages into a thing like pythonPackages.
|
|
I added 'black' to Biz/Lint.hs, but not the others because they rely on
dependencies being in the PYTHONPATH to work, so they are only relevant
in nix builds and repls.
I also made some other tweaks to the python checkPhase and linted all
the files. Everything should be building and linting correctly now.
|
|
The main change here is 'puts' now returns a value, this enables me to
collect the value from the conduit source while also doing stuff with
it, like printing or logging it as I want.
Previously I was running conduit over the source, *and then* kicking off
the concurrent processes to wait for the process and collect the output.
This would (I think) drain the source before it got to the 'puts'
conduit run, and so I wouldn't be able to get the output streamed in
real time.
It took a lot of refactoring and exploratory programming to get to this
relatively-small diff, but now puts works correctly. At least I think it
does... it seems to work more reliably from ghci than from the shell.
Maybe the shell or TERM is causing nix-store to do some buffering? Maybe
I need to use the threaded runtime in GHC? Not sure, but I will look out
for this issue and try to identify and fix.
Update: yep it was the threaded runtime. I enabled that and now it works
in the shell. I squashed that commit into this one.
|
|
|
|
This represents quite a few evenings of hacking. It doesn't build all of my
Python code, because my Python code is not up to snuff, but it builds the
examples and pulls in third party dependencies.
Some design points:
- I'm using buildPythonApplication in Builder.nix because it was getting way too
annoying to wrap the Python script and set PYTHONPATH myself. Easier and more
robust to just use the upstream nix builder
- Because of this, I had to generate a setup.py. Maybe switch to pyproject.toml
in the future, whatever.
- Also because of this, Target.wrapper is becoming redundant. I'll just remove
it when I get Guile built in nix.
- Biz/Bild.nix is getting messy and could use a refactor.
- In Builder.nix, I worked around the empty directories bug by just finding and
deleting empty directories after unpacking. If its stupid but works it ain't
stupid!
- I had to touch __init__.py files in all directories before building. Annoying!
- `repl` just works, which is awesome
- To ensure good Python code, I moved lints and added type checking to the
build. So I can't build anything unless it passes those checks. This seems
restrictive, but if I want to run some non-passing code, I can still use
`repl`, so it's actually not inhibitory.
|
|
|
|
This is prototype quality. For some reason I think it breaks when doing
`build **/*.hs`, which isn't good. But also it's working, and the code feels
good. Next I'd like to get Python builds working, as hopefully that will force
me to improve the existing code to support a second language.
|
|
There are some limitations to this implementation:
1. Using `runCommand` means this will re-run any time anything changes in the
repo. The solution is to use the existing import detection to make a list of
source files, and put that into a `stdenv.mkDerivation`, which I'll do next.
2. The `NeatInterpolation` usage is ugly. The templated nix code should be
extracted into its own file, such as `Biz/Bild/Builder.nix`.
3. I'm not actually calling it yet. The ideal thing would be to call
`nix-instantiate`, get the output drv path, and then call `nix-store --realise`
on that. To do that I need to refactor my `proc` function to return stdout to
the calling function, and I should probably just make helper functions like
`nixInstatiate :: Target -> IO DrvPath` and `nixStoreRealise :: DrvPath ->
NixStorePath`, or something like that.
|
|
|
|
|
|
Without this, Guile libraries like SDL2 will use Guile 2.0 instead.
|
|
At least one namespace (Biz/Dragons/Analysis.nix) needs this.
|
|
|
|
Mostly thid required packaging up some deps, but also had to recompile
stuff with cuda support.
|
|
|
|
|
|
As a byproduct this also (sorta) generalizes how I pass flags to the C compiler
using pkg-config, instead of using the guile-config. Now the 'lib' metadata will
be added with 'pkg-config --libs', and the 'sys' metadata will be added with
'pkg-config --cflags'. I'm not *really* sure what the difference is, but if it
works it works.
|
|
|
|
Unfortunately, until bild can instantiate nix builds, it needs all of its
compilers in the dev environment, and I need to pass this environment into the
dev nix-shell in order for it to work in the repl or after being built to _/bin.
|
|
|
|
In order to write Guile code against C, I need to distiguish between libs and
bins, so I did that, then I got the flags that gcc needs from `guile-config` and
put them in the args for any C lib build. I tested this with Bessel.c and
Bessel.scm (not in this patch, because I don't really want that code in my
tree, I'll come up with another way to test it later).
|
|
|
|
|
|
|
|
Rust seems to not be supported in my ctags version :(
Also rename some Ide scripts because these are commands, not really scripts.
|
|
|
|
I don't care about ghcjs anymore, the most javascript I want to do is jQuery.
|
|
|
|
I had thought that I needed some unmerged commits to nixpkgs that enable Guile
3.0, but turns out I just needed to use a different target name :P
|
|
asdf seems to always be necessary for any other packages to work, so I just
include that in the call to nix-shell, and swank is included because it's just
useful to start a repl server.
|
|
I ended up deleting miso, and consequently all files under Hero/ and Miso/,
because I couldn't get miso to build with GHC 9.2.
Other things:
- Niv has been wrapped by Biz/Bild/Deps.hs, so I can extend it to my liking.
- Apply-refact is gone because I couldn't get it to build.
- Disabled python stuff.
|
|
This way I can still inspect and use them from the nix repl by just doing `:l
Biz/Bild.nix`, but its also clear that they aren't part of the normal build
rules.
|
|
|
