#+title: SwarselSystems: NixOS + Emacs Configuration #+EXPORT_FILE_NAME: site/index.html #+HTML_HEAD: #+OPTIONS: toc:6 author:nil creator:nil timestamp:nil validate:nil html-postamble:nil html-preamble:nil broken-links:mark #+PROPERTY: header-args:config :eval never-export #+PROPERTY: header-args:css :eval never-export #+PROPERTY: header-args:emacs-lisp :tangle files/emacs/init.el :mkdirp yes :eval never-export #+PROPERTY: header-args:markdown :eval never-export #+PROPERTY: header-args:nix-ts :mkdirp yes :eval never-export #+PROPERTY: header-args:shell :mkdirp yes :eval never-export #+MACRO: revision-date (eval (format-time-string "%F %T %z")) #+MACRO: count-words (eval (count-words (point-min) (point-max))) #+MACRO: count-lines (eval (count-lines (point-min) (point-max))) #+MACRO: days-since (eval (number-to-string (- (org-today) (org-time-string-to-absolute (format "%s-%s-%s" $1 $2 $3))))) #+MACRO: NOTE (eval "This file has {{{count-words}}} words spanning {{{count-lines}}} lines and was last revised on {{{revision-date}}}.") *{{{NOTE(If you can see this, you might want to switch to [[https://swarsel.github.io/.dotfiles/][the hmtl version of this document]] in order to have working links and other QoL functions while reading this file.)}}}* * Introduction (no config code) :PROPERTIES: :CUSTOM_ID: h:a86fe971-f169-4052-aacf-15e0f267c6cd :END: This literate configuration file holds the entirety of all configuration for both NixOS PCs and servers as well as home-manager only systems across all machines that I currently use. It also holds an extensive Emacs configuration. I use this project to manage my entire home + cloud infrastructure This configuration is part of a NixOS system that is for the most part fully declarative (execpt for the steps outlined in [[#h:ed34ee4d-31f9-4d27-bc6e-ba37ee502d5a][Manual steps when setting up a new machine]]) and can be found here: - [[https://github.com/Swarsel/.dotfiles][Swarsel/.dotfiles on GitHub]] The literate configuration approach lets me explain my choices to my future self as well as you, the reader. I go to great lengths to explain the choices for all design steps that I take in order for me to pay due diligence in crafting my setup, and not simply copying big chunks of other peoples code. Also, this is very convenient to me as I only need to keep of (ideally) a single file to manage all of my configuration. I hope that this documentation will make it easier for beginners to get into NixOS (and, to some extent, Emacs) as I know it can be a struggle in the beginning. ** What I achieve with this project :PROPERTIES: :CUSTOM_ID: h:150ce3b3-20c6-4dc1-afcd-381cb9101719 :END: [[https://github.com/Swarsel/.dotfiles/tree/main/files/topology/topology.png][file:./files/topology/topology_small.png]] /(click to enlarge)/ This project manages my entire IT infrastructure. In particular: - A mailserver ([[#h:81bc8746-b46b-4d29-87de-ddbd77788b43][Eagleland (Hetzner)]]) - My home router ([[#h:58c7563e-6954-42e6-a622-9d06523e8e24][Hintbooth (Router: HUNSN RM02)]]) and its MicroVMs - Two homeservers ([[#h:82bf7fb1-631b-4acd-966b-d0c71a9eb463][Summers (Server: ASUS Z10PA-D8)]], [[#h:932ef6b0-4c14-4200-8e3f-2e208e748746][Winters (Server: ASRock J4105-ITX)]] and their respective MicroVMs) and one cloud server ([[#h:f547ed16-5e6e-4744-9e33-af090e0a175b][Moonside (OCI)]]) that are using other services defined in [[#h:79f3258f-ed9d-434d-b50a-e58d57ade2a7][Services]] (a list can be found in [[#h:191e82b6-6ae5-4ec8-ae6d-dc683ce325d9][Services]]) - Two servers (the cloud host [[#h:19300583-322b-4e0b-b657-857fbf23dfa1][Twothreetunnel (OCI)]] and one microvm [[#h:90dc7f71-f9da-49ef-b273-edfab7daaa05][Nginx]] hosted on [[#h:58c7563e-6954-42e6-a622-9d06523e8e24][Hintbooth (Router: HUNSN RM02)]]) that proxy requests to those services - A NixOS hydra buildfarm ([[#h:90457194-6b97-4cd6-90bc-4f42d0d69f51][Belchsfactory (OCI)]]) with binary caching that helps me build derivations faster and cache them for reuse - An authoritative DNS server ([[#h:1888ded8-69dc-431f-bb39-5089a8e8b1f4][Stoicclub (OCI)]]) that pushes records to both Hetzner and Hurricane Electric DNS - An SSH bastion ([[#h:a6baab45-b608-4289-bc92-4454bb0856c6][Liliputsteps (OCI)]]) that gatekeeps access to all cloud hosts - My work laptop ([[#h:6c6e9261-dfa1-42d8-ab2a-8b7c227be6d9][pyramid (Framework Laptop 16)]]) and my personal laptop ([[#h:a320569e-7bf0-4552-9039-b2a8e0939a12][Bakery (Lenovo ThinkPad)]]) - My work workstation ([[#h:ced1795a-9884-4277-bcde-6f7b9b1cc2f0][Treehouse (DGX Spark)]]) - My phone ([[#h:729af373-37e7-4379-9a3d-b09792219415][Magicant (Phone)]]) This is a system that grew organically over {{{days-since(2021,11,27)}}} days and has reached considerable complexity at this point. This documents exists to try and make it understandable to other people as well. ** How to use this document :PROPERTIES: :CUSTOM_ID: h:6f4b190c-fe69-47a3-9df2-ee429bd9b48b :END: When I started out with nix, it was a painful time. For a beginner, the available resources tend to be too detailed or assume too much prior knowledge. Also, it seems that using nix requires the user to understand it pretty well before many things start to make sense. That is the reason why I keep this configuration as a literate one: so that I am able to explain how everything works as best as I can. In the start, it was my goal to keep this project simple, so that it would be easy to understand when seen by a beginner. However, over time I have implemented more and more complicated solutions. Still, I try to keep the prosaic descriptions sufficient. For a beginner, I recommend to read this file like a book, from start to finish. I will try to explain concepts whenever they first come up, and will regularly link to [[#h:8ea35dcc-ef94-4c10-9112-8be8efd6f424][Appendix C: Explanations to nix functions and operators]] when more context is needed. For the first few times that I am using a new function, I will place such a link again. However, to keep the writing of this file manageable, I will generally only do this no more than three times. This page offers some utilities to you: - you can pin specific headings to the right "pinned" bar by hovering over the heading and clicking =[pin]= - If a section seems uninteresting to you, you can press the =↓= button to skip to the next one. - If a sections upcoming subsections seem uninteresting to you, you can press the =⇣= button to skip to the next heading of at least same or higher level. - If you are currently in a level of subsections that all seem uninteresting to you, you can press the =⇑= button to skip to the next heading that is at least one level higher. - And if you want to send a section to somebody else, you can click the =#= in order to copy its link to the clipboard. Your pinned headings will be saved locally, so you can continue reading in case you take a break. ** Structure of this file :PROPERTIES: :CUSTOM_ID: h:bcc3ebbe-df8a-46bd-b42d-73aad6fc66e5 :END: Now, I will outline how this document is structured. I have segmented this file into the following sections: - [[#h:a86fe971-f169-4052-aacf-15e0f267c6cd][Introduction (no code)]] This is the block you are currently in. It holds no code that actually builds the system, it just outlines the general approach and explains the rough design mentality. For understanding the nix (or Emacs) code in here, reading this should not be necessary if you already know some nix (feel free to skip to [[#h:c7588c0d-2528-485d-b2df-04d6336428d7][flake.nix]]). Otherwise, I will also give a brief introduction to some nix terms here. - [[#h:10a6f80d-7e00-4db1-953a-811993c22cb0][An introduction to nix]] Here I will give a coarse overview over some important concepts in the nix landscape. This is aimed at beginners in the field; others might want to skip this section as they are likely to not find much that will be worth their time. - [[#h:c7588c0d-2528-485d-b2df-04d6336428d7][flake.nix]] This block holds everything related to the heart of the nix side of the configuration - the =flake.nix= file. I am using [[https://github.com/hercules-ci/flake-parts][flake-parts]] to manage this flake, so different aspects of the configuration are handled by flake-part modules in different files. - [[#h:02cd20be-1ffa-4904-9d5a-da5a89ba1421][System]] This section holds all configuration options that apply to NixOS or home-manager. In other words, here we are doing system and user level configuration. In a way, I consider this the most important part of this file, as (nearly) all of the nix magic is going to happen here. - [[#h:ed4cd05c-0879-41c6-bc39-3f1246a96f04][Emacs]] This section defines my Emacs configuration. For a while, I considered to use rycee's =emacs-init= module ([[https://github.com/nix-community/nur-combined/blob/master/repos/rycee/hm-modules/emacs-init.nix]]) to manage my Emacs configuration; I have since come to the conclusion that this would be a bad idea: at the moment, even though it might seem as I am very bound to the configuration file that you are currently reading, if I ever decide to change how I run my system, I can simply take the generated =.nix= and =.el= files and put them wherever I need them. This file only simplifies that generation without putting further restrictions on my. If I were however to switch to =emacs-init= then I would be indeed to some level confined to the nix ecosystem with my Emacs configuration, as I would no longer have a valid =.org= file to manage it with, instead generating an =init.el= directly from nix code. I like to keep that level of freedom for potential future use. Also, you will notice there is no package system setup in this configuration. This is because packages are automatically handled on the NixOS side by parsing the generated =init.el= file for package installs. [[#h:fa377c74-36e5-4aea-bc39-4d3def4b67d1][nix build]] My emacs is built using the emacs-overlay nix flake, which builds a bleeding edge emacs on wayland (pgtk) with utilities like treesitter support. By executing the below source block, the current build setting can be updated at any time, and you can see my most up-to-date build options (last updated: {{{revision-date}}}) #+begin_src emacs-lisp :tangle no :exports both system-configuration-options #+end_src #+RESULTS: : --prefix=/nix/store/lymgpfqr5dp1wc0khbcbhhjnxq8ccsy9-emacs-pgtk-20240521.0 --disable-build-details --with-modules --with-pgtk --with-compress-install --with-toolkit-scroll-bars --with-native-compilation --without-imagemagick --with-mailutils --without-small-ja-dic --with-tree-sitter --without-xinput2 --with-xwidgets --with-dbus --with-selinux This file is not loaded by Emacs directly as the configuration (even though this would be possible - actually I did that in the past!) - instead, it generates two more files: - =early-init.el= This file handle startup optimization and sets up the basic frame that I will be working in. - =init.el= This file handles the rest of the Emacs configuration. By using the configuration offered by this file, the file you are reading right now (=SwarselSystems.org=) will not be freshly tangled on every file save, as this slows down emacs over time. However, when you clone this configuration yourself and have not yet activated it, you need to tangle the file yourself. This can be done using the general keybind =C-c C-v t= or my personal chord =C-SPC o t=. Alternatively, execute the following block: #+begin_src emacs-lisp :tangle no :export code :results silent (org-babel-tangle) #+end_src - [[#h:dae0c5bb-edb7-4fe4-ae31-9f8f064cc53c][Appendix A: Noweb-Ref blocks]] This section holds code that can be templated by other parts of the configuration. These blocks were used in several places throughout the configurations, but not on all machines necessarily. For example, the theming section used need to be in a NixOS block on NixOS machines but in a home-manager block on non-NixOS. Originally, I used this method a lot throughout my configuration. However, as my knowledge of NixOS grew, I have been weeding these snippets out more and more as I find more efficient native solutions. Now, only the theming block remains. For example, the above problem can be solved by defining a =theme= attribute set and using =lib.recursiveUpdate= as shown in [[#h:79f7150f-b162-4f57-abdf-07f40dffd932][Shared Configuration Options (holds firefox & stylix config parts)]] and [[#h:a92318cd-413e-4e78-a478-e63b09df019c][Theme (stylix)]]. Nowadays I use this mostly for some meta information in this file, e.g. for definining the [[#h:3bb92528-c61c-4b8d-8214-bf2a40baaa32][Programs]], [[#h:191e82b6-6ae5-4ec8-ae6d-dc683ce325d9][Services]] and [[#h:48e0cb2c-e412-4ae3-a244-80a8c09dbb02][Hosts]] blocks only once. I also use it to merge the [[#h:ed34ee4d-31f9-4d27-bc6e-ba37ee502d5a][Manual steps when setting up a new machine]]. As such, this served to reduce code duplication in this file. The tangled files experienced no size reduction, since noweb-ref only substitutes these blocks in. An example of using a noweb-ref block: First we define the block: #+begin_src nix-ts :tangle no :noweb-ref blockName enable = true; #+end_src which can then be used in a block like: #+begin_src markdown :tangle no :noweb no :results silent <> #+end_src and is finally parsed as: #+begin_src markdown :tangle no :noweb yes :results silent <> #+end_src Note that noweb-reffed blocks will not always be indented correctly. You will want to account for that when checking your nix flake with the formatter of your choice. Personally, I have solved this issue using the functions defined in [[#h:59d4306e-9b73-4b2c-b039-6a6518c357fc][org-mode: Upon-save actions (Auto-tangle, export to html, formatting)]]. Originally, I also automatically exported to html there, but it incurred a too high memory penalty which made Emacs become sluggish over time - instead I now build the website version whenever I push to GitHub. - [[#h:8fc9f66a-7412-4091-8dee-a06f897baf67][Appendix B: Supplementary Files]] This section holds files that are not written in nix but are still referenced in the configuration in some way. This is mostly used for configuration of programs that have no native nix support, like tridactyl, as well as "meta" files like the [[#h:bf3e6fc0-a95a-46d0-9305-0d1068b2f1ec][GitHub Readme]] or the [[#h:abed312c-5bf7-43f5-b0d0-43cce513ccb9][GitHub Workflow: Build-and-deploy]]. Note that shell scripts are still defined under their respective entry in [[#h:64a5cc16-6b16-4802-b421-c67ccef853e1][Packages]]. Over time, the goal is to reduce this section to a minimum, but things like the aforementioned tridactyl might stay for a long time, until we have a stable nix interface to configure browser plugins. - [[#h:8ea35dcc-ef94-4c10-9112-8be8efd6f424][Appendix C: Explanations to nix functions and operators]] When I started to learn about nix, I found that journey quite arduous; while I disagree with the general public in that the documentation is too sparse, I will say that, while it is very good, reading (and understanding!) it requires a certain level of existing nix knowledge that one will problably not have when starging out. Hence, the goal of this document is to explain common nix functions as they come up in this document (I thing I wrote this before :sweat:), in hopes that you will be able to understand most of the code. When a new function appears for the first time, I will try to link to an entry in the appendix. *** HTML version of this document :PROPERTIES: :CUSTOM_ID: h:12880c64-229c-4063-9eea-387a97490676 :END: I will also quickly talk about the webpage that (I hope) you are currently viewing works: The =.html= version of this page is built by a GitHub workflow; it can also be generated manually by calling the chord =C-SPC o e=, or by executing the below block #+begin_src emacs-lisp :tangle no :export code :results silent (org-html-export-to-html) #+end_src The web version is useful because it allows navigation using org-mode links, which makes the configuration easier to follow in a number of ways: - mobile-friendly documentation - Functionality to save links to certain headers in a "Pinned" menu that can be toggled - These links are locally saved I add a javascript bit to the file in order to have a darkmode toggle when exporting to html (defined in [[#h:cf6a12cf-929b-4080-9945-e6f02afc7990][HTML Export: Darkmode toggle]]). NOTE: What I am doing is defining an elisp multiline string which gives me heredoc capabilities for defining blocks that should be both exported to the final =.html= but should also be shown in the document itself (only putting a =#+[begin/end]_export= block would not show the block in the content of the page) which is then output as html and exported to both. #+begin_src elisp :noweb yes :exports both :results html " <> " #+end_src I also add this javascript to add header pinning functionality to the site, using the same trick as above (this is defined in [[#h:e5f900a0-9d68-4269-a663-53c52434c342][HTML Export: Docs QoL]]): #+begin_src elisp :tangle no :noweb yes :exports both :results html " " #+end_src ** TODO Structure of this flake :PROPERTIES: :CUSTOM_ID: h:2c5529ed-e6d9-44b6-b0d3-5bf96a6bed64 :END: The structure of this flake as seen many revisions, however lately I have settled on a system that I have grown to like: - =hosts=: This folder is used to house all configurations that are used across the infrastructure. At the top level, it splits into the subfolders =nixos=, =home=, =darwin=, and =android=. These folders specify the mode that the configuration is running in: - nixos: Full NixOS host (may or may not also use home-manager) - darwin: Host that uses NixOS on MacOS (may or may not use home-manager) - home: Host that uses only home-manager (no full NixOS) - android: Phone using nix-on-droid (may or may not use home-manager) The corresponding configurations are automatically generated by =mkFullHostConfigs= and =mkHalfHostConfigs=. A "full" host either in the nixos or darwin folder, while a "half" host is in either of home or android. This has to do with the scheme in which these configurations are generated. These folders hold on the first level a folder describing the machine archetype (=x86_64-linux= or =aarch64-linux= for linux, =x86_64-darwin= or =aarch64-darwin= for macs). Those folders then hold a number of folders, the actual configurations. At this time, the files stored in this folder are: - default.nix: This file holds the abstracted configuration of the host. This should mostly be enabling [[#h:f0f1c961-3e7a-47b8-99ab-1654bb45dffc][Profiles]] as well as setting some [[#h:79f7150f-b162-4f57-abdf-07f40dffd932][Configuration options]]. - hardware-config.nix: It is not clearly defined what I hold in this file. Mostly it is just the attributes that nix originally sets when setting up the system for the first time (although at this time modified by me!), bar any filesystem configuration. This makes my deployment in [[#h:74db57ae-0bb9-4257-84be-eddbc85130dd][swarsel-bootstrap]] a little bit simpler. - disk-config.nix Holds the aforementioned filesystem configuration and is applied using [[https://github.com/nix-community/disko][disko]]. - The hosts// folders may also have a =secrets= folder, under which files of the ending =.nix.enc= may be stored. As the name suggests, these files should be encrypted. Specifically, they need to be [[https://github.com/getsops/sops][sops]]-encrypted files (sops does not seem to suggest a file ending other than .yml or others, which is not verbose enough for me, so I went with =.enc=). This should have the structure of a nix expression, e.g.: #+begin_src nix-ts :tangle no { my_value = 2; my_attrSet = { enable = true; }; } #+end_src It is also possible to pass it as a function: #+begin_src nix-ts :tangle no { config }: { my_value = 2; my_attrSet = { enable = config.myconfig.enable; }; } #+end_src Using the mechanisms in [[#h:82b8ede2-02d8-4c43-8952-7200ebd4dc23][PII management]] (which in turn uses [[#h:87c7893e-e946-4fc0-8973-1ca27d15cf0e][extra-builtins]] and [[#h:315e6ef6-27d5-4cd8-85ff-053eabe60ddb][sops-decrypt-and-cache]]), these files are decrypted during evaluation time and stored under a persistent directory. As the name suggests, I am using these files to store personally identifiable information - these "secrets" are stored world-readable in the nix store. As such, this should not be used to store important secrets, but rather information that you would not like everyone on the internet to easily find in your git repo. Other than that, the =secrets= folder will also be used to store conventional (decryted at activation-time) sops-encrypted secrets in the standard =.yaml= / =.toml= / =.ini= formats. - =modules= This folder holds the most part of the actual system configuration done in this repository. At some point I thought it was cool to have my whole configuration exposed under the flakes =nixosModules=, which is indeed achieved (its usefulness is however debatable). In any way, this folder splits up as: - nixos: Holds true NixOS configuration - home: Holds configuration to be used by home-manager (either as a NixOS submodule or not) - shared: This is for configuraion bits that are to be used by both types. The nixos and home folders further split up: - common: Configuration that can be used by all hosts (TODO: this currently includes configuration used by my user devices, which will mostly not be used by servers) - server: Configuration to be used on servers - darwin: Holds configuration for nix-darwin. - optional: Configuration that will be used rather rarely This structure is very optionated and highly subjective. I will possibly change this in the future. By themselves, most of the files in the modules folder will not do anything. In order for them to do something, their corresponding =config.swarselmodules= attribute needs to be enabled. This is partly done using... - =profiles=: This folder splits up into =home= and =nixos= subfolders, where groupings of module enablers are stored for the respective home and nix setups. Note that =home= profiles are also used in NixOS setups (extensively even)! This is used to quickly enable common configuration for a machine use, e.g. the [[#h:dfc076fd-ee74-4663-b164-653370c52b75][Server]] profile. - =nix=: This special folder holds mostly =.nix= files that are not automatically loaded, but rather setup specific things that affect most of the flake. For example, here lies the aforementioned [[#h:87c7893e-e946-4fc0-8973-1ca27d15cf0e][extra-builtins]] as well as the setup for the [[*Globals][Globals]] system. Also in here are the flake-parts files that you read about earlier. This gives the following functionality: - =lib=: I define some utility functions that I add to the nixpkgs library under the =swarselsystems= attribute set. An example would be the =mkIfElse= function. - =checks=: As part of a [[#h:4d0548db-99b2-4e07-b762-6d86fbb26d4c][Devshell (checks)]], I declare pre-commit hooks that should run before I push changes to my repo. - =overlays=: Here we also define the main (default) overlay I am using in my configuration. It is responsible for adding my defined packages and modifications to the final nixpkgs. Also I add some other conveniences like all past stable nixpkgs and some other package sets. - =apps=: I also define [[#h:52e1fae8-0e8c-4be6-a6ce-758ada652dd3][Apps]], which is an output of derivations that can be called by =nix run= without having the flake locally - this is mostly used for my =swarsel-*= utilities. - =topology=: I also created a diagram of my infrastructure using [[https://github.com/oddlama/nix-topology][nix-topology]]. While I do not update this too often, this (I think) can quickly give a good overview of the scope of this flake as well as its services. - =pkgs=: This folder holds derivations (mostly packages) that I define myself. This is mostly used to grab versions that are not (yet) in nixpkgs, or modified versions of another package. Each derivation in this folder is in turn in its own folder which holds a defautlt.nix. Using the mechanism in [[#h:64a5cc16-6b16-4802-b421-c67ccef853e1][Packages]], these are automatically built and available to all configurations (packages still need to be installed e.g. in =environment.systemPackages=). Note that the folder at the top level splits up in =config= and =flake= subdirectories: - The =config= dir is used for packages that need the actual config of the machine where they run in order to be built. These packages cannot simply be released as a flake output (or better, it would not make a lot of sense). Instead, these are added within the configuration as an overlay - The =flake= dir is used for the conventional packages that I described above. - =files=: This is kind of a catchall folder that holds (nearly) all non-nix files. It mostly holds blocks created in [[#h:8fc9f66a-7412-4091-8dee-a06f897baf67][Appendix B: Supplementary Files]], but also some more specific directories: - =scripts=: This folder holds a bunch of shell scripts that I use for various tasks. Nearly all of these are made into a derivation using =pkgs.writeShellApplication=. In the future (TODO?), I might convert these to native nix, but in the past I kept the as true shellfiles in case I ever wanted to move away from nix. This is becoming less and less likely, however. And even in case that this would happen, I could retrieve these files from the nix store and would simply have to remove the nix store paths. - =wallpaper=: Holds my wallpapers and profile pictures :) - =topology-images=: Holds pictures used by [[#h:391e7712-fef3-4f13-a3ed-d36e228166fd][Topology]] :) - =secrets=: Unlike the similar folder under =hosts=, this folder holds sops-encrypted secrets and PIIs that are used by a number of hosts that is greater than one. - =install=: This folder holds another [[#h:1d4514b4-e952-4faf-b30e-d89e73a526c6][Installer flake]]. That flake pulls in the =nixosConfigurationsMinimal= that are defined in [[#h:5c5bf78a-9a66-436f-bd85-85871d9d402b][Hosts]] of the main flake, which enables me to build an extemely reduced configuration when I deploy a new host for the first time - this is used by [[#h:74db57ae-0bb9-4257-84be-eddbc85130dd][swarsel-bootstrap]] in the first installation step. It also holds the configuration of the two installer images that I use to deploy this flake: - [[#h:8583371d-5d47-468b-84ba-210aad7e2c90][Drugstore (ISO installer config)]]: This is the general installer ISO that I use whenever I can when I want to deploy a new host. It has a few conveniences like some of my utility programs for figuring out some dependencies or network quirks, as well as my public ssh keys so that I can immediately login to them. - [[#h:e9fe580c-f1b2-4d7b-aaff-bbdf89a8c9f9][Brick Road (kexec image)]]: This is a kexec tarball that can be used by [[#h:74db57ae-0bb9-4257-84be-eddbc85130dd][swarsel-bootstrap]] in case that I need to deploy to a machine that has less than 1GB of RAM. It is basically just an even more stripped down version of the detault one used by nixos-anywhere, but notably I added cryptsetup so that it can be used when setting up an encrypted device using disko. - =.github=: Canonically, this holds github related files like the [[#h:bf3e6fc0-a95a-46d0-9305-0d1068b2f1ec][GitHub Readme]] and some workflows. ** Hosts :PROPERTIES: :CUSTOM_ID: h:48e0cb2c-e412-4ae3-a244-80a8c09dbb02 :END: Here I give a brief overview over the host machines that I am using. This is held in markdown so that I can render it into my [[#h:bf3e6fc0-a95a-46d0-9305-0d1068b2f1ec][GitHub Readme]] without further effort. #+begin_src markdown :tangle no :noweb-ref hosts | Name | Hardware | Use | |---------------------|-----------------------------------------------------|-----------------------------------------------------------------| |💻 **pyramid** | Framework Laptop 16, AMD 7940HS, RX 7700S, 64GB RAM | Work laptop | |💻 **bakery** | Lenovo Ideapad 720S-13IKB | Personal laptop | |💻 **machpizza** | MacBook Pro 2016 | MacOS reference and build sandbox | |🏠 **treehouse** | NVIDIA DGX Spark | AI Workstation, remote builder, hm-only-reference | |🖥️ **summers** | ASUS Z10PA-D8, 2* Intel Xeon E5-2650 v4, 128GB RAM | Homeserver (microvms), remote builder, data storage | |🖥️ **winters** | ASRock J4105-ITX, 32GB RAM | Homeserver (IoT server in spe) | |🖥️ **hintbooth** | HUNSN RM02, 8GB RAM | Router, DNS Resolver, home NGINX endpoint | |☁️ **stoicclub** | Cloud Server: 1 vCPUs, 8GB RAM | Authoritative DNS server | |☁️ **liliputsteps** | Cloud Server: 1 vCPUs, 8GB RAM | SSH bastion | |☁️ **twothreetunnel**| Cloud Server: 2 vCPUs, 8GB RAM | Service proxy | |☁️ **eagleland** | Cloud Server: 2 vCPUs, 8GB RAM | Mailserver | |☁️ **moonside** | Cloud Server: 4 vCPUs, 24GB RAM | Game servers, syncthing + other lightweight services | |☁️ **belchsfactory** | Cloud Server: 4 vCPUs, 24GB RAM | Hydra builder and nix binary cache | |🪟 **chaostheater** | Asus Z97-A, i7-4790k, GTX970, 32GB RAM | Home Game Streaming Server (Windows/AtlasOS, not nix-managed) | |📱 **magicant** | Samsung Galaxy Z Flip 6 | Phone | |💿 **drugstore** | - | NixOS-installer ISO for bootstrapping new hosts | |💿 **policestation** | - | NixOS live ISO for generating cryptographic keys | |💿 **brickroad** | - | Kexec tarball for bootstrapping low-memory machines | |❔ **hotel** | - | Demo config for checking out this configuration | |❔ **toto** | - | Helper configuration for testing purposes | #+end_src ** Programs :PROPERTIES: :CUSTOM_ID: h:3bb92528-c61c-4b8d-8214-bf2a40baaa32 :END: This is meant to give a brief overview over the main programs/components that I use on a daily basis on my client machines. This should be mostly useful for people wanting to rice their config, or people who believed this repos title and are looking for =.dotfiles= :p On [[#h:02df9dfc-d1af-4a37-a7a0-d8da0af96a20][Sway]], I run a combination of different services that are all replaced by [[#h:385cc6c7-416c-4570-a5d3-bf8fb7c841e7][Noctalia Shell]] when I am running on [[#h:06e77ca4-28ff-4cfd-bc60-b7fd848bfedb][Niri]], which has grown to become my default recently. #+begin_src markdown :tangle no :noweb-ref programs | Topic | Program | |---------------|-----------------------------------------------------------------------------------------------------------------------------| |🐚 **Shell** | [zsh](https://github.com/Swarsel/.dotfiles/tree/main/modules/home/common/zsh.nix) | |🚪 **DM** | [greetd](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/common/login.nix) | |🪟 **WM** | [SwayFX](https://github.com/Swarsel/.dotfiles/tree/main/modules/home/common/sway.nix) or [Niri](https://github.com/Swarsel/.dotfiles/tree/main/modules/home/optional/niri.nix) | |⛩️ **Bar** | [Waybar](https://github.com/Swarsel/.dotfiles/tree/main/modules/home/common/waybar.nix) or [Noctalia Shell](https://github.com/Swarsel/.dotfiles/tree/main/modules/home/optional/noctalia.nix) | |✒️ **Editor** | [Emacs](https://github.com/Swarsel/.dotfiles/tree/main/files/emacs/init.el) | |🖥️ **Terminal**| [Kitty](https://github.com/Swarsel/.dotfiles/tree/main/modules/home/common/kitty.nix) | |🚀 **Launcher**| [Fuzzel](https://github.com/Swarsel/.dotfiles/tree/main/modules/home/common/fuzzel.nix) or [Noctalia Shell](https://github.com/Swarsel/.dotfiles/tree/main/modules/home/optional/noctalia.nix) | |🚨 **Alerts** | [Mako](https://github.com/Swarsel/.dotfiles/tree/main/modules/home/common/mako.nix) or [Noctalia Shell](https://github.com/Swarsel/.dotfiles/tree/main/modules/home/optional/noctalia.nix) | |🌐 **Browser** | [Firefox](https://github.com/Swarsel/.dotfiles/tree/main/modules/home/common/zsh.nix) | |🎨 **Theme** | [City-Lights (managed by stylix)](https://github.com/Swarsel/.dotfiles/tree/main/modules/home/common/sharedsetup.nix) | #+end_src ** Services :PROPERTIES: :CUSTOM_ID: h:191e82b6-6ae5-4ec8-ae6d-dc683ce325d9 :END: This is a comprehensive list of the services/components ran by my server machines. #+begin_src markdown :tangle no :noweb-ref services | Topic | Program | |------------------------------|----------------------------------------------------------------------------------------------------------------| |📖 **Books** | [Kavita](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/kavita.nix) | |📼 **Videos** | [Jellyfin](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/jellyfin.nix) | |🎵 **Music** | [Navidrome](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/navidrome.nix) + [Spotifyd](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/spotifyd.nix) + [MPD](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/mpd.nix) | |🗨️ **Messaging** | [Matrix](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/matrix.nix) | |📁 **Filesharing** | [Nextcloud](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/nextcloud.nix) | |🎞️ **Photos** | [Immich](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/immich.nix) | |📄 **Documents** | [Paperless](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/paperless.nix) | |🔄 **File Sync** | [Syncthing](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/syncthing.nix) | |💾 **Backups** | [Restic](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/restic.nix) | |👁️ **Monitoring** | [Grafana](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/monitoring.nix) | |🍴 **RSS** | [FreshRss](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/freshrss.nix) | |🌳 **Git** | [Forgejo](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/forgejo.nix) | |⚓ **Anki Sync** | [Anki Sync Server](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/ankisync.nix) | |🪪 **SSO** | [Kanidm](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/kanidm.nix) + [oauth2-proxy](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/oauth2-proxy.nix) | |💸 **Finance** | [Firefly-III](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/firefly-iii.nix) | |🃏 **Collections** | [Koillection](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/koillection.nix) | |🗃️ **Shell History** | [Atuin](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/atuin.nix) | |📅 **CalDav/CardDav** | [Radicale](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/radicale.nix) | |↔️ **P2P Filesharing** | [Croc](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/croc.nix) | |✂️ **Paste Tool** | [Microbin](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/microbin.nix) | |📸 **Image Sharing** | [Slink](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/slink.nix) | |🔗 **Link Shortener** | [Shlink](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/shlink.nix) | |⛏️ **Minecraft** | [Minecraft](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/minecraft.nix) | |☁️ **S3** | [Garage](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/garage.nix) | |🕸️ **Nix Binary Cache** | [Attic](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/attic.nix) | |🐙 **Nix Build farm** | [Hydra](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/hydra.nix) | |🔑 **Cert-based SSH** | [OPKSSH](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/opkssh.nix) | |🔨 **Home Asset Management**| [Homebox](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/homebox.nix) | |👀 **DNS Records** | [NSD](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/nsd.nix) | |✉️ **Mail** | [simple-nixos-mailserver](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/mailserver.nix) | |🚇 **VPN Access** | [Firezone](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/firezone.nix) | |🛡️ **Local DNS Resolver** | [AdGuard Home](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/adguardhome.nix) | |🛎️ **DHCP** | [Kea](https://github.com/Swarsel/.dotfiles/tree/main/modules/nixos/server/kea.nix) | #+end_src * An introduction to nix :PROPERTIES: :CUSTOM_ID: h:10a6f80d-7e00-4db1-953a-811993c22cb0 :END: This is where it gets interesting. In this section, I want to give an overview over some important concepts needed when working in the nix ecosystem. ** Nix, NixOS, Nixpkgs, and Nix :PROPERTIES: :CUSTOM_ID: h:6d0a91d4-e478-45ae-82de-0997fa289e7a :END: First off, when talking about nix, we need to differentiate between several things: - The [[#h:b2222c7c-0bd5-44f8-aa69-17869d6913c0][nix language]] - The [[#h:f4fa6a46-e016-48f8-9ead-608b739c706a][nix package manager]] - The linux distribution [[#h:d62af55c-a7f3-47ba-b2a5-78cc60b03aef][NixOS]] - the =nix-community= input [[#h:f4e89635-e006-4c41-a545-d4b56c7ac293][nixpkgs]] While these terms are all connected with each other, it is important to keep in mind that fundamentally those are separate entities. Let us briefly talk about each one: *** nix language :PROPERTIES: :CUSTOM_ID: h:b2222c7c-0bd5-44f8-aa69-17869d6913c0 :END: The base of all the other concepts named above is the =nix language=, a pure functional programming language that is declarative, dynamically typed, and evaluated lazily. That means: - Functions are just values, that can be assigned to variables, passed as arguments to other functions, or returned by functions - All variables are immutable (their values cannot change during computation) - Values will only be computed when their result is needed - There is no sequential order of operations; instead, operations get ordered they need to evaluate another expression that they depend on - We do not need to specify a values type when declaring it Now, I will give a brief overview over some important concepts of the nix language. This aims to only build the essential understanding needed for dealing with the configuration I am using (this should be sufficient to then understand most other configurations online). For deepening your knowledge, you might want to check out the [[https://nix.dev/reference/nix-manual.html][nix reference manual]]. **** Derivations and the nix store :PROPERTIES: :CUSTOM_ID: h:45135360-96d4-4348-b6fb-e8bc4685611e :END: The main purpose of the nix language is to build packages. These packages are built by functions that can be roughly described by something like this: ~f: { source, compiler, dependencies, etc. } -> package~ This function along with all its inputs we call a =derivation=. What is important to realize is that such a derivation can itself depend on other derivations, and the resulting package does not necessarily need to be a callable program (a more correct name for the function result is =outputs=). When we as nix users define a package, we will usually use a function called =pkgs.stdenv.mkDerivation= (ore one of its derived wrappers like =buildRustPackage=; for now, it is not important what these do exactly), which will run things like install phases and setup hooks for us. It is a wrapper around nix function ~derivation(name, system, builder, args=[],outputs=["out"])~, where =name= is the derivation name, =system= is the system architecture to build for, =builder= is the executable responsible for building the result, =args= are arbitrary arguments passed to =builder=, and =outputs=, which can be thought of a list of locations where we want to put different build artifacts that can be referenced separately when using the package. There are [[https://nix.dev/manual/nix/2.28/language/advanced-attributes][more attributes]], but these five are the most important and commonly used ones. Building a package (in nix terms: "realising") from a derivation causes the package to be created in the =nix store=. This is a read-only filesystem that is usually located at =/nix/store=. When building a package, nix will place its content of each output at =/nix/store/---/=, where == is a unique identifier that changes whenever one of the inputs to the above function changes. That means that it is no problem to store different versions of the same file on one system, and each program can use the correct dependencies that it needs. A key observation that follows from this is that packages are declarative and reproducible; when not changing the inputs, a derivation will always yield the same package with the same store path. We call the set of derivations (and, recursively, the derivations that those depend on) that a derivation depends on its =closure=. Usually when installing a package, we do not need all the outputs it provides: For example, the [[https://search.nixos.org/packages?channel=25.11&query=pcsclite][pcscliteWithPolkit package on nixpkgs]] that is used for configuring smart cards provides 5 outputs: =out=, =dev=, =doc=, =man=, and =lib=. By default, a NixOS system will install =out=, =man=, =info= and =doc= outputs (the latter three have respective =documentation..enable= NixOS options, where == would be either of =man=, =info=, or =doc=) as well as any outputs listed in =environment.extraOutputsToInstall= (by default an empty list). However, if we need to debug something, we might need the =dev= output of the package, which can then be installed by referencing =pcscliteWithPolkit.dev=. Also, a note that may be useful when reading package source code (for example in =nixpkgs=); you will often come across =$out= and =$src=. The =$out= represents the root of that build output (=$src= are the build sources). You will often see that in build phases (it is there set as an environment variable). In other places, you will instead see =placeholder =, which will be replaced by the outputs future location in the nix store. **** Types in the nix language :PROPERTIES: :CUSTOM_ID: h:7bbb4e50-f5e9-4496-a0cd-d8eb34aa2514 :END: The nix language supports the following types and how they look in the wild: - null: =null= - bool: =true= - strings: ="text"= - strings can also be defined as =multiline strings= - Those will look for the lowest level of indent shared over all lines and strip it: #+begin_src bash :tangle no :exports both swarsel-instantiate " '' indent0 indent2 indent1 '' " #+end_src #+RESULTS: : indent0\n indent2\n indent1\n /(you can ignore the [[#h:95ebfd13-1f6b-427f-950d-e30c1ed6f9fa][swarsel-instantiate]]; it is just a wrapper around =nix-instantiate= that preloads nixpkgs. I will use this to show you the evaluation results of nix calls)/ - note that tab characters will /not/ be stripped: #+begin_src bash :tangle no :exports both swarsel-instantiate " '' indentTab indent1 indent2 '' " #+end_src #+RESULTS: : \tindentTab\nindent1\n indent2\n - an URI string can assume the type of string without need for quoting (e.g. you can write =http://about.org= instead of ="http://about.org"=) - Strings can be interpolated by using =${expression}= (read on for an example) - =expression= can be any valid nix expression that is compatible with the enclosing type (here: string) - As in many languages, you have =\n=, =\r=, and =\t= available - in normal strings, you can escape stuff using =\= - the following characters need to be escaped in normal strings: - Backslash: =\= (escape using =\\=) - Double quote: ="= (using =\"=) - Opening of nix expression: =${= (using =\${=) - in multiline strings, you escape stuff using one or two apostrophes ='= - the following characters need to be escaped in multline strings: - Two apostrophes are escaped with a single apostrophe: =''= (escape with ='''=) - All other things are escaped using =''=: - Dollar sign: =$= (escape with =''$=) - =\n=, =\r=, and =\t= (=''\n= and so on) - =''\= is a catchall for all other escaping - somewhat interestingly, double dollars =$$= (as used in Makefiles) never need to be escaped - ints: 1 - floats: 1.1 - paths: =/home= - a path can also be given as a relative path (e.g. =./.config=) - attribute sets: ={ }= - these hold name value pairs, e.g. ={ a = 3; }= - a "chain" of attributes, separated by dots, is called an =attribute path=, e.g. =config.environment.systemPackages= - in such a chain, all attributes but the last will be =attribute sets= - =config.environment.systemPackages = ;= is equivalent to =config = { environment = { systemPackages = ; }; };= - lists: [ ] - these hold values, e.g. =[ { a = 3; b = 2; } ]=. In this example, the list holds a single value, that is, the attribute set ={ a = 3; b = 2; }=. - functions: =arg: body= - =arg= can be any of these data types, including functions - when =arg= is an attribute set, some special things apply: - default values can be specified using =arg ? defaultValue=, e.g. ={ name ? "Default", age }: "${name} is ${age} years old"= will yield ="X is 20 years old"= when called using the attribute set ={ name = "X"; age = "20"; }=. Otherwise it will yield "Default is 20 years old" when called using ={ age = "20"; }=. When not passing =age= in this example, it will throw an error (also not that we had to pass =age= as a string as the value is not cast to a string automatically. Alternatively, we could have passed =age = 20;= and updated the function body to ="${name} is ${builtins.toString age} years old"=. But that is just a sidenote) Let's see this in action: Calling with explicit values: #+begin_src bash :tangle no :exports both swarsel-instantiate 'let f = {name ? "Default", age }: "${name} is ${age} years old"; in f { name = "X"; age = "20"; }' #+end_src #+RESULTS: : X is 20 years old Calling by using a default value with =?=: #+begin_src bash :tangle no :exports both swarsel-instantiate 'let f = {name ? "Default", age }: "${name} is ${age} years old"; in f { age = "20"; }' #+end_src #+RESULTS: : Default is 20 years old Not passing =age= errors out: #+begin_src bash :tangle no :exports both :results discard :eval never-export swarsel-instantiate 'let f = {name ? "Default", age }: "${name} is ${age} years old"; in f { }' #+end_src #+RESULTS: #+begin_example error: … from call site at «string»:1:104: 1| let lib = import ; in let f = {name ? "Default", age }: "${name} is ${age} years old"; in f { } | ^ error: function 'f' called without required argument 'age' at «string»:1:44: 1| let lib = import ; in let f = {name ? "Default", age }: "${name} is ${age} years old"; in f { } | ^ #+end_example - the evaluator will error out if it is called with an argument that is not explicitely mentioned in the function definition. Passing a superfluous =another= errors out: #+begin_src bash :tangle no :exports both :results discard :eval never-export swarsel-instantiate 'let f = {name ? "Default", age }: "${name} is ${age} years old"; in f { age = "2"; another = "0"; }' #+end_src #+RESULTS: #+begin_example error: … from call site at «string»:1:104: 1| let lib = import ; in let f = {name ? "Default", age }: "${name} is ${age} years old"; in f { age = "2"; another = "0"; } | ^ error: function 'f' called with unexpected argument 'another' at «string»:1:44: 1| let lib = import ; in let f = {name ? "Default", age }: "${name} is ${age} years old"; in f { age = "2"; another = "0"; } | ^ #+end_example - this behaviour can be suppressed by adding an ellipsis =...= to the function arguments (={ name ? "Default", age, ... }:=). Adding this ellipsis =...= to the function definition, we can now pass =another=: #+begin_src bash :tangle no :exports both swarsel-instantiate 'let f = {name ? "Default", age, ... }: "${name} is ${age} years old"; in f { age = "2"; another = "0"; }' #+end_src #+RESULTS: : Default is 2 years old The latter is useful mainly in the NixOS module system, where a lot of things will be passed by default. More on that later. Finally, we can make the values in =...= available by using =@= as in =extra @ { name, ... }: "${name} likes ${extra.object}"=: #+begin_src bash :tangle no :exports both swarsel-instantiate 'let f = extra @ { name, ... }: "${name} likes ${extra.object}"; in f { name = "Nyx"; object = "nix"; }' #+end_src #+RESULTS: : Nyx likes nix Note that it is equivalent to write ={} @ extra=: #+begin_src bash :tangle no :exports both swarsel-instantiate 'let f = { name, ... } @ extra: "${name} likes ${extra.object}"; in f { name = "Nyx"; object = "nix"; }' #+end_src #+RESULTS: : Nyx likes nix This looks cumberesome on first sight, but is for example useful when referencing flake inputs in the flake outputs, as it allows you to forego listing them all in the arguments (which can be a very long list!). More on that later. **** Language features :PROPERTIES: :CUSTOM_ID: h:5b660aa4-6dbb-4ca7-a006-c97319ae6b7e :END: - The functions available to the nix language can be found in [[https://hydra.nixos.org/build/318491462/download/1/manual/language/builtins.html][the nix reference manual]]. Some explanations to common functions are provided in [[#h:ebe86cf4-fe65-464f-8f64-04b57870c5e9][Builtin functions]]. - Scopes are static in nix; scoped variables are not inherited by the context of the evaluation - Variables can be created in an enclosing scope using =let in =. You actually saw me use this several times in the above function call. Note that "usually", you will not be able to declare arbitrary variables; this is because usually we will be working within [[#h:8067f8fc-92d7-46af-b1be-e8a337d7a953][The module system]] - in that scope, you either need to resort to the =let ... in= construct or you need to declare things as module options (more on that later). In general nix however, this would be no problem. - another way to add an expression to the scope is using the =rec= keyword: - this adds all attributes of the enclosing attribute - the following will error out: #+begin_src bash :tangle no :exports both :results discard :eval never-export swarsel-instantiate ' { a = true; b = a; } ' #+end_src #+RESULTS: : error: undefined variable 'a' : at «string»:4:9: : 3| a = true; : 4| b = a; : | ^ : 5| } - however, =rec= makes =a= available to the scope (also remember that the order of expressions does /not/ matter): #+begin_src bash :tangle no :exports both swarsel-instantiate ' rec { b = a; a = true; } ' #+end_src #+RESULTS: : { a = true; b = true; } - the last way to something to the scope is by using =with=: - this adds the passed set to the lexical scope of the enclosing expression: - the following will error out: #+begin_src bash :tangle no :exports both :results discard :eval never-export swarsel-instantiate ' let functions = { print = v: v; }; in print "ok" ' #+end_src #+RESULTS: : error: undefined variable 'print' : at «string»:7:5: : 6| in : 7| print "ok" : | ^ : 8| - using =with= will make it work: #+begin_src bash :tangle no :exports both swarsel-instantiate ' let functions = { print = v: v; }; in with functions; print "ok" ' #+end_src #+RESULTS: : ok - generally, the values of the variable in the innermmost scope is given precedence in evaluation: #+begin_src bash :tangle no :exports both swarsel-instantiate ' let scope = "outer"; in let scope = "inner"; in scope ' #+end_src #+RESULTS: : inner - the same holds true for =with=: #+begin_src bash :tangle no :exports both swarsel-instantiate ' with { scope = "outer"; }; with { scope = "inner"; }; scope ' #+end_src #+RESULTS: : inner - you can copy variables from an outer scope by using =inherit () ;= - This is syntactic sugar to achieve the same as = = .;= - when not in a string, we can place parentheses around a nix expression to provide its return value to other expressions (remember the =builtins.toString (-1)= example from before. =-1= is indeed a nix expression.) - you can write inline comments using =#= and multiline comments using =/* ... */= - The following words are reserved keywords and cannot be used for variable names: =assert=, =else=, =if=, =in= =inherit=, =let=, =or=, =rec=, =then=, =with= **** Operators :PROPERTIES: :CUSTOM_ID: h:f995d46f-ff07-4a5e-9164-ed7aef369c42 :END: The following operators work as you know them from most other programming languages: - Any operator involving numbers (=+=, =-=, =*=, =/=) /Sidenote:/ When negating a number in actual nix code you will need to wrap it in parentheses nearly every time due to function evaluation: #+begin_src bash :tangle no :exports both swarsel-instantiate ' builtins.toString (-1) ' #+end_src #+RESULTS: : -1 but: #+begin_src bash :tangle no :exports both :results discard :eval never-export swarsel-instantiate ' builtins.toString -1 ' #+end_src #+RESULTS: #+begin_example error: … while calling the 'sub' builtin at «string»:2:21: 1| let lib = import ; in 2| builtins.toString -1 | ^ 3| … while evaluating the first argument of the subtraction error: expected an integer but found the built-in function 'toString': «primop toString» #+end_example - Logical operators (=!=, ====, =!==, =<=, =>=, =>==, =<==, =&&=, =||=) - additionally, logical implications are available as =->= (defined as =!v1 || v2= a.k.a. =if v1 then v2 else true=) The following operators are nix specific: - Concatenations can be done between strings and paths (in any order) using =+= - List Concatenations however use the =++= operator - you can use the =?= operator on an attribue set to check whether it has an attribute: #+begin_src bash :tangle no :exports both swarsel-instantiate ' { exists = true; } ? exists ' swarsel-instantiate ' { exists = true; } ? noExists ' #+end_src #+RESULTS: | true | | false | - Attribute selection (on attribute sets) is done using =.=: #+begin_src bash :tangle no :exports both swarsel-instantiate ' { v = 1; }.v ' #+end_src #+RESULTS: : 1 - Updates to attribute sets are done using [[#h:b1fe7a9a-661b-4446-aefa-98373108f8fd][The '//' operator]] **** Flakes :PROPERTIES: :CUSTOM_ID: h:b44026d5-bf7e-4ab3-a31b-68b72292e908 :END: Next, we shall talk about some concepts regarding nix flakes. What are flakes? The answer is surprisingly simple: Flakes can be thought of packages that contain nix expressions. They can include other flakes as inputs (acting as libraries if you want) and build on their functionality (in fact, this is exactly what happens when you are using [[#h:d62af55c-a7f3-47ba-b2a5-78cc60b03aef][NixOS]] with [[#h:f4e89635-e006-4c41-a545-d4b56c7ac293][nixpkgs]]). What exactly is provided by a flake is up to the owner entirely, although there are some "norms" that we will get to later. A directory becomes a flake as soon as it contains a file called =flake.nix=, which contains an attribute set that has at least an attribute =outputs=, which must be a function returning an attribute set. Hence, the simplest possible flake is the following: #+begin_src nix-ts :tangle no { outputs = _: { }; } #+end_src What is very nice about flakes is that each flake generates a corresponding =flake.lock= file the first time a nix command is called targetting it. This file exactly specifies at which revision every flake input should be pulled in. This makes flakes very nice for declaratively setting up development environments that can then be committed to a project; this enables collaborators to do [[#h:c22f65c4-4c0d-4aea-9b70-d34c295764f0][nix develop]] which will setup a reproducible environment, which is the same for every developer. That was a very dry introduction. Now, what we are usually interested in when pulling in a flake is any of the following: - modules to extend our configuration with - packages that are not in =nixpkgs= and, to a somewhat lesser degree: - flake templates - devshells Where these things come from we will learn soon. ***** Why flakes :PROPERTIES: :CUSTOM_ID: h:96d28671-8b9d-4fbc-8620-1a7d5fcec480 :END: Flakes are, as of writing, an experimental feature in nix, although their interface has been quite stable for a while now - in fact, stable enough that most bigger nix projects are exposing a flake nowadays. People like to (mistakenly) claim that it is because of flakes that we have version pinning in nix. In fact, people were doing this long before flakes were a thing using a construct that looks roughly like this: #+begin_src nix-ts :tangle no let let pkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/0b20bf89e0035b6d62ad58f9db8fdbc99c2b01e8.tar.gz") {}; in pkgs.mkShell { buildInputs = [ pkgs.cowsay ]; } #+end_src This approach simply imports a chunk of nix code that is fetched using =fetchTarball= - and the version is pinned by the git revision. However, imagine a bigger project; even if you were to import every input like this and pass the expressions to the modules that need them, updating the inputs manually would be a chore. And even though projects like [[https://github.com/nmattia/niv][niv]] and [[https://github.com/andir/npins][npins]] exist(/ed) to help with these problems, flakes provide a uniform interface and are reasonably easy to work with, and provide a lot more other stuff on the side. That being said, some people decide to work without flakes. Depending on the reasoning, I think this is totally fair (as really for the code itself they dont automatically provide a lot of extra functionality that cannot be achieved with other tools). However, some people dismiss flakes for the only reason being "they are too complex" with really is not at all true (which is what I hope to prove to you soon!). A simple project devshell for example could very reasonably be built only from the =fetchTarball= construct I showed above. ****** Channels :PROPERTIES: :CUSTOM_ID: h:2cb443a5-9de2-4ae5-8a7e-d8e96748d599 :END: Finally and for completeness sake, a historical note on nix channels. Nix channels are one of the official mechanisms to provide package sources. They are managed manually through the cli. A channel would be given a name (like =nixpkgs=) which could then in nix code be referenced for example like: #+begin_src nix-ts :tangle no { pkgs ? import {} }: ... #+end_src Really, this leaves it up to the caller to decide what =nixpkgs= really is, along with a few other considerations that I do not want to get into. This chapter is intentionally left short; I just want you to know how the pattern above (the ==) looks so that you know what you are working with if you encounter it in the wild. Channels are not a great tool for reproducability and I would advise against using them if possible. ***** Flake arguments :PROPERTIES: :CUSTOM_ID: h:2dec2fcf-5923-4bae-9407-027dd9d917e1 :END: Let us now talk about arguments that a flake (kind of) expects. ****** Inputs :PROPERTIES: :CUSTOM_ID: h:e965faf8-5782-4720-aa44-d20d700a339c :END: As we already learned, a flake must provide an attribute called =outputs= (whatever that might mean at this point!). However, unless we want to build everything that our flake provides from the ground up, we probably also want to pass some libraries or build tools to it. This is done using the attribute called =inputs=. An =inputs= definition might look something like this: #+begin_src nix-ts :tangle no { inputs = { nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable"; hydra = { url = "github:nixos/hydra/nix-2.30"; inputs.nix-eval-jobs.follows = "nix-eval-jobs"; }; nix-eval-jobs = { url = "github:nix-community/nix-eval-jobs/v2.30.0"; flake = false; }; toplevel.url = "./.."; }; } #+end_src This is a surprisingly real-world example from my [[#h:aee5ec75-7ca6-40d8-b6ac-a3e7e33a474b][flake.nix]] that we can use so explain most of the important =inputs= options: - the most important attribute is =url=. It describes where the source should be fetched from. As you can see from the =toplevel= input, you can put as an input a path on the local filesystem. Note that by default, it is expected that every input is itself a flake - meaning, a =flake.nix= exists there which will be loaded into our flake along with its dependencies. You can also see that we set this to =false= for the =nix-eval-jobs= input. This is usually done when the target project is simply not a flake. Nix will then load in the contents of the repository as a raw filetree. However, if you go and check out [[https://github.com/nix-community/nix-eval-jobs][nix-eval-jobs]] you will see that this project /does/ have a =flake.nix=, and it is also very active; so what is going on here? In this particular case, this is done to avoid pulling in the dependencies that the =nix-eval-jobs= would bring along. Instead, the files provided by it must be then made to work using what is provided in the parent flake. This is not a common pattern, but something that is good to know. Another interesting option is the =hydra.inputs.nix-eval-jobs.follows=. Let us first talk about the general interface =.inputs.=: This structure allows to customize options for the *dependency* of a flake input. For example, we could have customized the source of the =nix-eval-jobs= dependency of =hydra= had we written =hydra.inputs.nix-eval-jobs.url = ...=. The =follows= option takes a similar approach by setting an input dependency to the same version as some other input of the parent flake (in our case, it is pinned to the =nix-eval-jobs= v2.30.0 version). When we are done defining our flake inputs, we can then decide what our flake shoud actually do. ****** Outputs :PROPERTIES: :CUSTOM_ID: h:66a04339-dbc7-4d98-a763-5a9be9185f9b :END: As we already learned earlier, =outputs= is a function returning an attribute set. In fact, it is usually a function that takes as an argument an attribute set containing our inputs, looking roughly like this: #+begin_src nix-ts :tangle no { inputs.nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable"; outputs = inputs @ { self, nixpkgs, ... }: let inherit (self) outputs; in { nixosConfigurations = { main = nixpkgs.lib.nixosSystem { modules = [ "${self}/configuration.nix" ]; specialArgs = { inherit inputs outputs; }; }; }; }; } #+end_src With what we learned so far, you are already able to understand most of what is going on here: - outputs is a function that expects =self= and =nixpkgs= but /tolerates/ other attributes in the set, and the attribute set =inputs= can be referenced. - an attribute =outputs= is inherited from the outer scope =self=, meaning =outputs = self.outputs;= (what =self= is and why we can reference the =outputs= in the same place where we are defining them I will explain shortly) - the resulting attribute set holds an attribute called =nixosConfigurations= that holds an attribute called =main=, which is set by calling a function that we now do not want to get into too much. You might (rightfully!) wonder where the =self= argument comes from. It is actually not a flake input. Instead, =self= is a special flake construct that refers to the root of the current flake, meaning all flake attributes can be accessed using =self.=. Moreover, when using =self= as a path, it refers to the path os the current flake in the nix store. This can be used to reference files within the repository without needing relative paths (this is exactly what happenss in =self.outputs.nixosConfigurations.main.modules=: it loads in =configuration.nix=, which resides in the same directory as the =flake.nix=. We could have also written =./configuration.nix= here instead). Another question might be what happens when we load in =self.outputs= while defining outputs. Normally that should not work? Remember that nix is a lazy language. When =outputs= is first created in the =let ... in= block, its value will be something along the lines of =outputs = ;=. That means, not all values are loaded upon instanciation, but only whenever they are needed. And if a value is needed, nix will then move on to compute that value. Of course that does mean that we need to be careful to not introduce circular chains of dependency. Doing that would result in what is called an =infinite recursion error=, a type of error that can be very hard to wrap one's head around. Luckily, when writing a simple config, you are not very likely to encounter it in a shape that is not rather easily solved. Like we saw here with =nixosConfigurations= and some other outputs that we heard about earlier, there are some "standard" outputs that nix recognizes and is able to apply some defaults on. Let us quickly go over them: ****** Standard flake outputs :PROPERTIES: :CUSTOM_ID: h:680b38c1-14de-4892-8840-2c82accd6827 :END: I will now list output names and explain what they to; some outputs are what I call "system-scoped". That means they have toplevel attribute in the name of system architectures, e.g. =x86_64-linux=. I will mention when an output is system-scoped. This list is roughly ordered by importance to a NixOS beginner: - =nixosConfigurations=: A set of NixOS host configurations - =devShells=: *system-scoped*. A set of devshells that can be used by calling [[#h:c22f65c4-4c0d-4aea-9b70-d34c295764f0][nix develop]]. Defaults to the devshell called =default=. - =nixosModules=: A set of nixos [[#h:de7bc464-e04f-45d4-9d29-e46592535419][Modules]]. By default, when consuming one of these modules but not specifying which one, the overlay =default= will be chosen. - =overlays=: A set of nixpkgs [[#h:7a059bd9-13f8-4005-b270-b41eeb6a4af2][Overlays]]. By default, when consuming one of these overlays but not specifying which one, the overlay =default= will be chosen. - =checks=: *system-scoped*. Flake checks to perform when calling [[#h:b07b1776-75cd-4a40-9e71-fb00eab65c6f][nix flake check]] - =formatter=: *system-scoped*. Formatting package/config to use when calling [[#h:cf6cc5fa-cbaa-415a-b1d8-c6c5b7016700][nix fmt]] - =packages=: *system-scoped*. Packages that can be directly built using [[#h:fa377c74-36e5-4aea-bc39-4d3def4b67d1][nix build]] - =apps=: *system-scoped*. Packages that can be directly run using [[#h:9f55f619-892e-488c-936f-43962f90cde8][nix run]] - =templates=: A set of templates that can be initialized using =nix flake init -t