Nix 2.22.0 release with Jacek Galowicz

Welcome to the Full Time Nix podcast. Today, you are listening to the release notes for Nix 2.22 with Jacek Galowicz. This podcast is hosted by Shahar "Dawn" Or and processed by the Full Time Nix team. Consider becoming our patreon. Goto https://fulltimenix.com for more information.

Shahar "Dawn" Or: Nix released 2.22.0, 2024 April 23rd. One significant change and one other change. We’ll start with the significant one. Remove experimental `repl-flake`, that’s a sub command. REPL Flake, I suppose.

Jacek Galowicz: `repl-flake` is the name of the experimental setting. So you have to put `repl-flake` thingy into your Nix config. Or you had to use that. Now you don’t have to, any longer. And then you can run `nix repl` on a flake directly.

Shahar "Dawn" Or: The “REPL Flake” experimental feature has been removed. The `nix repl` command now works and-I should say `nix repl` commands so it’s the REPL sub-command now works like the rest of the new CLI. In that, `nix repl ` now tries to load the Flake at path, or fails if the `flakes` experimental feature isn’t enabled. So what I understand is that the same feature is now covered by the `flakes` experimental feature. Sort of been absorbed into it?

Jacek Galowicz: Of course. With Flakes, you get the Flakes syntax right? You can run `nix build` or `nix develop` or `nix run`. And then you describe the path to a flake, then hashtag and then the attributes that you want to have from there. That’s of-course the nix flakes syntax which is activated with the `flakes` option. Yes. But that’s not how `nix repl` worked in the past, right? So, even before Flakes, `nix repl` was a normal command. When you run `nix repl` with a path to a Nix file, it would simply open that and put all the variables that the Nix expression that you referenced on the command line, everything that it exports will be in your scope of the REPL and then you can play with it. One would typically run `nix repl` and then `nixpkgs` in HTML brackets (``), simply open Nixpkgs and then play in the shell/REPL with whatever the Nixpkgs has to offer, right? So of course, with flakes or with the introduction of Flakes, you basically want to use the same syntax of Flakes as for the other commands. Right? So you want to run `nix repl` and a PATH to a Nix expression as usual, but to a Flake Nix expression that has the attributes. And of course that would be a clash so what they did is they introduced-

Shahar "Dawn" Or: It should be apparent from the syntax whether it’s a Flake or vanilla.

Jacek Galowicz: Yeah. I mean especially when you would have been writing `nix repl .` What does it mean? In the old world, it meant the `default.nix` file but in the new world it’s a `flake.nix` file. And for the Flake, you especially want to also select an attribute. What they did is they introduced this REPL flake option that was experimental and that enables the new syntax so you would try `nix repl` and then `.` would mean a `flake.nix` and then you can optionally add a hashtag and then whatever attributes you want. And if you wanted the old syntax or the old semantics, you would write `nix repl --file` and then the filepath to a normal Nix expression. So you can have both but this kind of breaks the old scripts.

Shahar "Dawn" Or: Right, it’s not backwards compatible.

Jacek Galowicz: Exactly. But, nix- this repl flake option was not enabled by default, so it wouldn’t break stuff for people. They did this for a few releases with the intention to have this also print the warning and then when users have been warned for multiple releases, eventually remove it again. Which is what happened right now. So it should not be a surprise any longer. Of course, only if you regularly update, right? That the old semantics are now gone. If you have flakes enabled and you have to write `nix repl --file` if you want to reference a normal non-flake nix expression, right?

Shahar "Dawn" Or: Right. Since what release do we have `repl-flake` experimental feature?

Jacek Galowicz: I think that was something like 18/19/20. I’m not completely sure but I have been using this `--file` syntax for months and months already.

Shahar "Dawn" Or: Well, so there’s a bit of a caveat there. Our current release is 2.22.0 and Nixpkgs, so if you’re using NixOS for example, you’ll have, I believe still 2.18. Isn’t it?

Jacek Galowicz: Oh, I’m running unstable for a year already myself. So I’m not really on track on what we’re using.

Shahar "Dawn" Or: I am talking about unstable. I’m running nix unstable and it is 2.18.2.

Jacek Galowicz: Oh okay, so you have 2.18.2.

Shahar "Dawn" Or: So the caveat here is that basically we still don’t have, and I never had `repl flake`- you said it was since 2.18/2.19?

Jacek Galowicz: I just looked it up. So release 2.10 introduced the new semantics. So `--file` and `--expr` were flags that were kind of introduced with this `repl-flake` feature. You have been warned since 2.10.

Shahar "Dawn" Or: Oh, 2.10. Okay. Okay. So that was a long long time ago. You have been warned. And that was the release of 2022 July. So that’s a long time. Okay, no caveats.

Jacek Galowicz: No, not really. Although, one thing now that this is gone, one might think that running `nix repl .` on the flake would be the same without any parameters and then in the REPL, run `:lf` for load flake and then `.` but when you do this, you still get different results. So you will notice that when you load the flake from within the already opened REPL manually with `:lf` then you will get more variables into the scope added than if you ran `nix repl .` on the terminal. Although this is something that you wouldn’t have to know forever because the Nix team already decided that they want to fix this. So sometime in the future, both variants of loading a flake would be on par.

Shahar "Dawn" Or: Okay. Well, if it's a temporary caveat. By the way, what is it? What are the extra attributes that you would have?

Jacek Galowicz: I don’t have them all present right now, but, for example, the revision information and everything that Nix expects from the Git repository. That is something that you get extra when you use `:lf` and a few other things around that. Basically, meta information.

Shahar "Dawn" Or: So the difference is not between `:lf` and the extra `.` parameter, it’s between the dot (`.`) and the no dot. Right? The dot and the implicit dot.

Jacek Galowicz: No no, not really. I mean when you run `nix repl` and then dot. It will open the flake that is in the current directory, right? Okay, if you do this, or if you run `nix repl` without parameters and then run `:lf .` Of course, that doesn’t have to be a dot. It could be any path. The trick is that you use `:lf` or the same path on the command line. That’s where the difference comes from. Same flake, different variables.

Shahar "Dawn" Or: Right. Because if you only did a `nix repl` even if you are in a directory that has a flake, the flake would not be loaded. Doesn’t load a flake by default. Just opens the Nix REPl with nothing in it.

Jacek Galowicz: Yeah. With no parameters, it loads nothing and you just get a naked REPL, yes.

Shahar "Dawn" Or: Okay. So the difference between the dot in the parameter and in the `:lf .`

Jacek Galowicz: Exactly. With dots being just the example path, right?. It could be any flake path.

Shahar "Dawn" Or: Right. Dot is just a path. And, so, with `:lf`, we get some of the extra attributes.

Jacek Galowicz: Mhm. But that will vanish. So in the future we will get the same amount of variables added or the same number of variables added regardless of the method.

Shahar "Dawn" Or: Still experimental. Subject to change.

Jacek Galowicz: Yeah true.

Shahar "Dawn" Or: Alright! So that’s all for the removal of the experimental of `repl-flake`?

Jacek Galowicz: Yes.

Shahar "Dawn" Or: And moving on to the other change. `nix eval` prints derivations as `.drv` paths. Just a reminder, `nix eval` can do a couple of different things, right? What does `nix eval` do?

Jacek Galowicz: Basically, eval just evaluates, like the name says. Or like the derivation suggests that it evaluates the Nix expression, right? You just evaluate random Nix expressions and it works basically the same way like on the repl. You do `1 + 1` and it will give you `2`. But, `nix eval` of course could be used for much more complicated Nix expressions and then also give you a RAW or a JSON formatted outputs. So if you use additional tooling to get information out of structured informations out of Nix expressions, you could use `nix eval` on a subshell.

Shahar "Dawn" Or: `nix eval` prints the derivations as `.drv` paths. `nix eval` will now print derivations as their `.drv` paths rather than as attribute sets. This makes command like `nix eval nixpkgs#bash` terminate instead of infinitely looping into recursive, self referential attributes. And there’s an example. I see it executed `nix eval nixpkgs#bash` and I see this- I’m actually not familiar with this unicode character. It’s a single character that has thrdr double angle brackets. Let’s find out what this-once and, hopefully for all-what this is called.

Jacek Galowicz: Left pointing double angle quotation mark.

Shahar "Dawn" Or: Right, left pointing, or the right pointing variant double angle quotation mark. I wonder what it is used [for] but I digress. So there is in these double angle quotation marks there is the derivation, space and then a store path. That is specifically, ends with `.drv`. So what is this about infinitely looping recursive self referential attributes? Why does the bash [expression], when you try to `nix eval` the Bash attribute of Nixpkgs, why do you end up with [an] infinite loop?

Jacek Galowicz: Yeah so, when you go through-

Shahar "Dawn" Or: Before this change.

Jacek Galowicz: For example, in the REPL, if you just opened the REPL on Nixpkgs and pointed to `pkgs.hello`, `pkgs.bash`, the REPL will already tell you that this is a derivation. But, when you run `pkgs.bash` dot then press Tab to expand it will show you that this is already an attribute set with many many attribute names. And `nix eval`-

Shahar "Dawn" Or: Because a derivation is an attribute set?

Jacek Galowicz: Yeah yeah, exactly! I can tell you, it basically is. When you force the derivation to a string, then it would. Either use the `outPath` variable to just give you the out path of this thing when it was built, or you can also evaluate the `.drv` path attribute which kind of tells where the derivation resides, as a file. And the `nix eval` command was not really distinguishing what it prints there. So it sees an attribute set, might that be a normal attribute set with a few keys in there, which is nicely printable, or a derivation; both are the same for it. So it would simply print all the keys from the derivation. But while obtaining all those keys, it might go down and down-of course those attribute sets of derivations are nested and, for example the package `hello` has a pass through the test sub-attribute set which also tests the `hello` package. So the `hello` package references itself. And when you go down and down and down, of course, that’s an infinite recursion when you try to print all of these attributes sets keys and values, right? What the `nix eval` command does now is see that the type of this attribute set is basically a derivation because it has the corresponding keys that a derivation has and then stop printing all the attribute members and simply prints the derivation path. You can get rid of those unicode special characters surrounding it if you use `--raw` or `--json` for example.

Shahar "Dawn" Or: If you do `--raw` or `--json` you will get a string that is the derivation store path?

Jacek Galowicz: Ah yeah- let me try it out. This got my attention from reading the release notes. Yeah. So when you use `--json`, I just tried it out. You get the same thing in quotes. When you use `--raw`, let’s try it out.

Shahar "Dawn" Or: When you say the same thing, it’s with the word derivation?

Jacek Galowicz: Yes so I just ran in my terminal `nix eval nixpkgs#bash` and then `--raw` or `--json`. And both worked. So with `--raw`, you would simply get the naked string. But that doesn’t seem to be the- Oh no it does actually do the derivation path. Yea. No, it’s the output path. I’m sorry. So `--raw` and `--json` would give you the output of the build derivation. Yes. And then you run `nix eval nixpkgs#bash` without anything else, you get the raw derivation and then can decide what to do with it.

Shahar "Dawn" Or: That makes sense because JSON is like stringification so it would do the same as the-

Jacek Galowicz: Yeah it does get mixed up with this a bit because I don’t regularly use it like this so you look at things a new way when you try out what they fixed after reading the changelog. But yeah, that’s how it works right now. No infinite recursion. That’s the most important part.

Shahar "Dawn" Or: Right. And those keys that Bash would infinitely recurse on are not necessarily there, it’s not the case that derivations are inherently infinitely recursive? It just happens to be for some of the Nix packages derivations.

Jacek Galowicz: Yeah. I guess for, like many, we just come up with the simplest derivation possible that would be, like an artificial attribute set that simply contains the key type which is filled with the value derivation as a string and another key `drvPath` equals an another string and that is an attribute set, of course. That can’t be printed like a derivation so you’ll get an error that’s most likely the derivation path that you put in there doesn’t exist but it would try to handle it as a derivation object while at the same time it’s simply a minimal, naked attribute set, that, by accident, contains the right keys, right?

Shahar "Dawn" Or: Well, that’s also not a real derivation, right?

Jacek Galowicz: Yeah, true. But it wouldn’t be by nature recursive, right?

Shahar "Dawn" Or: So, you’re kind of countering my question with, “Well, what is a real derivation?”

Jacek Galowicz: *laughter* Yeah.

Shahar "Dawn" Or: Makes sense.

Jacek Galowicz: I mean many real life derivations, for example it’s- in Nixpkgs, many packages are expected to have like their own tests so then you have to, inside the derivation attribute set, you will have to pass `foo.test` or some attribute set and there any good package will have some kind of test where it tests itself and that’s one good reason to have a lot of recursion- like infinite recursion in most packages, right?

Shahar "Dawn" Or: Okay. Makes sense! So, that is all for Nix release notes 2.22. Thank you very much Jacek!

Jacek Galowicz: Thank you for inviting me!

Shahar "Dawn" Or: Looking forward to next one.

Jacek Galowicz: Yes!