Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
1b1b6d6232 |
123
SingleFile.org
123
SingleFile.org
@ -1,123 +0,0 @@
|
||||
#+TITLE: SingleFile
|
||||
#+DATE: 2024-03-20
|
||||
* Prologue
|
||||
I am using the [[https://en.wikipedia.org/wiki/Zettelkasten#Use_in_personal_knowledge_management][Zettelkasten]] methodology for personal knowledge
|
||||
management, with [[https://www.orgroam.com/][Org-roam]] to implement it.
|
||||
|
||||
While the Internet does not forget things in general, it might be
|
||||
difficult to find things again. Maybe someone published some
|
||||
information in a GitHub gist or has their whole blog there.
|
||||
Suddenly GitHub falls out of favour because it got bought by an
|
||||
evilcorp and information disappears.
|
||||
|
||||
For those reasons I store a link to the information directly, a link
|
||||
to the [[https://web.archive.org/][Wayback Machine]] and a local copy. My Zettelkasten and all
|
||||
references, including a local copy, are stored in git.
|
||||
|
||||
* SingleFile
|
||||
[[https://github.com/gildas-lormeau/SingleFile][SingleFile]] is a Firefox[fn::Other browsers are supported as well, I
|
||||
just think Firefox is the least-evil one.] extension to save a website,
|
||||
including images and css, into a single HTML file. It's perfect for
|
||||
personal archival purposes. When run as a plugin it will save the page
|
||||
as it is currently rendered. For example dismissed cookie banners will
|
||||
not be part of the safes page.
|
||||
|
||||
The downside is that it's some semi-manual process to get the backup
|
||||
into the Zettelkasten note.
|
||||
|
||||
* SingleFile-CLI
|
||||
There is also a [[https://github.com/gildas-lormeau/single-file-cli][CLI]] version, like all modern stuff it runs in
|
||||
docker. Of course I do not have docker on OpenBSD, nor the alternative
|
||||
podman.
|
||||
|
||||
So I setup a Fedora VM, installed podman and run singlefile-cli via
|
||||
ssh. On the Fedora VM I use this wrapper:
|
||||
#+begin_src sh
|
||||
#! /bin/sh
|
||||
|
||||
set -e
|
||||
|
||||
content=$(podman run --privileged -u 0:0 singlefile "$@")
|
||||
|
||||
now=$(date --iso-8601=seconds -u)
|
||||
title=$(echo ${content} | /home/florian/.cargo/bin/htmlq --text title | \
|
||||
awk '{$1=$1};1' | \
|
||||
head -1 | tr -d '\n' | tr -c '[:alnum:]' '_' | cut -c -128)
|
||||
|
||||
fn="/tmp/${title}_${now}.html"
|
||||
|
||||
echo ${content} > ${fn}
|
||||
echo $fn
|
||||
#+end_src
|
||||
|
||||
It fetches a copy of the website, uses [[https://github.com/mgdm/htmlq][htmlq]] to extract the title,
|
||||
removes special characters and saves the page in ~/tmp~. It then spits
|
||||
out the file-name of the backup.
|
||||
|
||||
On my laptop I have this script:
|
||||
#+begin_src sh
|
||||
#! /bin/sh
|
||||
|
||||
set -e
|
||||
|
||||
fn=$(ssh fedora bin/singlefile "$@")
|
||||
bn=$(basename ${fn})
|
||||
scp fedora:${fn} .
|
||||
open ${bn}
|
||||
#+end_src
|
||||
This causes the Fedora VM to fetch a backup of the website and then
|
||||
copies it over and stores it in the current working directory. Finally
|
||||
it opens it in the browser for inspection.
|
||||
|
||||
* org-cliplink
|
||||
We still do not have links stored in the Zettelkasten note.
|
||||
|
||||
For that I use [[https://github.com/rexim/org-cliplink][org-cliplink]] to insert org mode links from the
|
||||
clipboard.
|
||||
|
||||
A bit of elisp code massages the link text to add "Archive: " or
|
||||
"Local: " in front of the title. It also changes the URL for the local
|
||||
backup to have it relative so that it works correctly no matter where
|
||||
the git repo with the Zettelkasten is checked out.
|
||||
|
||||
#+begin_src elisp
|
||||
(use-package org-cliplink
|
||||
:config
|
||||
(defun custom-org-cliplink ()
|
||||
(interactive)
|
||||
(org-cliplink-insert-transformed-title
|
||||
(org-cliplink-clipboard-content) ;take the URL from the CLIPBOARD
|
||||
(lambda (url title)
|
||||
(let* ((parsed-url (url-generic-parse-url url)) ;parse the url
|
||||
(turl
|
||||
(cond
|
||||
;; if type is file make the url relative to zettelkasten
|
||||
((string= (url-type parsed-url) "file")
|
||||
(url-unhex-string (replace-regexp-in-string "\\(.+\\)\/zettelkasten\/\\(.+\\)" "file:\\2" url)))
|
||||
;; otherwise keep the original url
|
||||
(t url)))
|
||||
(ttitle
|
||||
(cond
|
||||
;; if type is file, add Local: to title
|
||||
((string= (url-type parsed-url) "file")
|
||||
(replace-regexp-in-string "\\(.+\\)" "Local: \\1" title))
|
||||
;; otherwise keep the original title
|
||||
(t title))))
|
||||
;; forward to the default org-cliplink transformer
|
||||
(org-cliplink-org-mode-link-transformer turl ttitle)))))
|
||||
:custom
|
||||
(org-cliplink-title-replacements
|
||||
'(("https://github.com/.+/?"
|
||||
("\\(.*\\) · \\(?:Issue\\|Pull Request\\) #\\([0-9]+\\) · \\(.*\\) · GitHub" "\\3#\\2 \\1"))
|
||||
("https://twitter.com/.+/status/[[:digit:]]+/?"
|
||||
(".+ on Twitter: \\(.+\\)" "\\1"))
|
||||
("https://web.archive.org/.+/?"
|
||||
("\\(.+\\)" "Archive: \\1"))))
|
||||
:bind (:map org-mode-map
|
||||
("C-c C-S-L" . custom-org-cliplink)))
|
||||
#+end_src
|
||||
|
||||
* Epilogue
|
||||
I only recently added singlefile-cli to my work-flow, so far it is a
|
||||
big improvement. Time will tell if the backups are cluttered with
|
||||
banners and I have to go back to semi-manual mode.
|
||||
File diff suppressed because it is too large
Load Diff
Binary file not shown.
@ -1,686 +0,0 @@
|
||||
% Intended LaTeX compiler: pdflatex
|
||||
\documentclass[conference]{IEEEtran}
|
||||
%\documentclass[11pt]{article}
|
||||
\usepackage[utf8]{inputenc}
|
||||
\usepackage[T1]{fontenc}
|
||||
\usepackage{graphicx}
|
||||
\usepackage{longtable}
|
||||
\usepackage{wrapfig}
|
||||
\usepackage{rotating}
|
||||
\usepackage[normalem]{ulem}
|
||||
\usepackage{amsmath}
|
||||
\usepackage{amssymb}
|
||||
%\usepackage{capt-of}
|
||||
\usepackage{hyperref}
|
||||
\date{2023-03-07}
|
||||
\title{Dynamic host configuration, please}
|
||||
\hypersetup{
|
||||
pdfauthor={Florian Obser},
|
||||
pdftitle={Dynamic host configuration, please},
|
||||
pdfkeywords={},
|
||||
pdfsubject={},
|
||||
pdfcreator={Emacs 28.2 (Org mode 9.5.5)},
|
||||
pdflang={English}}
|
||||
\author{\IEEEauthorblockN{Florian Obser}
|
||||
\IEEEauthorblockA{florian@openbsd.org}
|
||||
}
|
||||
\begin{document}
|
||||
|
||||
\maketitle
|
||||
\begin{abstract}
|
||||
\label{sec:orgdbc4e3b}
|
||||
Smartphones are always online devices in urban areas. They are even
|
||||
mostly online in rural areas. They deal with many different kinds of
|
||||
networks with only minimal configuration from the user. This paper
|
||||
will cover how we achieved a similar user experience on OpenBSD
|
||||
laptops. We will cover how we remember past visited Wi-Fi networks,
|
||||
automatically configuring IPv4 and IPv6 addresses and dealing with DNS
|
||||
in challenging network environments. We will also point out security
|
||||
measurements we put in place while dealing with untrusted networks.
|
||||
\end{abstract}
|
||||
|
||||
\section{Join the Wi-Fi}
|
||||
\label{sec:orgff06dc2}
|
||||
When we bring a device to a never-before-visited network location we
|
||||
need the network name and password for the Wi-Fi or select an open
|
||||
Wi-Fi network.
|
||||
|
||||
Smartphones provide a user interface (UI) for this where we select
|
||||
the Wi-Fi from a drop-down list and then we are prompted for a
|
||||
password.
|
||||
|
||||
On OpenBSD, network interfaces are configured by \href{https://man.openbsd.org/ifconfig.8}{ifconfig(8)}, or
|
||||
persistently in \href{https://man.openbsd.org/hostname.if.5}{/etc/hostname.IF}\footnote{IF denotes a specific network
|
||||
interface. For example for iwm0 the file is \texttt{/etc/hostname.iwm0}},
|
||||
which is read by \href{https://man.openbsd.org/netstart.8}{netstart(8)} during boot. netstart(8) calls ifconfig(8)
|
||||
internally to handle the network configuration. \texttt{\# ifconfig iwm0 scan}
|
||||
will list the available Wi-Fi networks for the \texttt{iwm0} interface.
|
||||
|
||||
For a long time, we could only configure one Wi-Fi network:
|
||||
\begin{verbatim}
|
||||
$ cat /etc/hostname.iwm0
|
||||
nwid home wpakey "trivial password"
|
||||
inet autoconf
|
||||
inet6 autoconf
|
||||
up
|
||||
\end{verbatim}
|
||||
|
||||
This configures a Wi-Fi network named "home" and a password "trivial
|
||||
password". IPv4 and IPv6 auto-configuration are enabled. Whenever the
|
||||
network is in range the kernel automatically connects to it.
|
||||
|
||||
That is not a good user experience (UX). We typically take our laptops
|
||||
with us and connect to different Wi-Fi networks, like our smartphones. We
|
||||
have a Wi-Fi at home, at work, there are open Wi-Fis at hotels, and so
|
||||
on.
|
||||
|
||||
People came up with all kinds of unusual shell scripts that would run in
|
||||
the background or were triggered by \href{https://man.openbsd.org/cron.8}{cron(8)} to notice when the laptop moved
|
||||
to a different Wi-Fi. The script would then call ifconfig(8) to
|
||||
reconfigure Wi-Fi from a list of networks it knew about. This was all fragile and unlike how OpenBSD works.
|
||||
|
||||
Peter Hessler (phessler@), with the help of Stefan Sperling (stsp@)
|
||||
went ahead and tackled this problem: what if we could pass multiple
|
||||
\texttt{(name, password)} tuples to the kernel and the kernel would choose the
|
||||
right one?
|
||||
|
||||
\begin{verbatim}
|
||||
$ cat /etc/hostname.iwm0
|
||||
join home wpakey "trivial password"
|
||||
join work wpakey zUDciIezevfySqam
|
||||
join "Airport Wi-Fi"
|
||||
join ""
|
||||
inet autoconf
|
||||
inet6 autoconf
|
||||
up
|
||||
\end{verbatim}
|
||||
\texttt{join} implements exactly this. The argument to \texttt{join} is the name of
|
||||
the network and the following \texttt{wpakey} is the password for that
|
||||
network. If we leave out the \texttt{wpakey}, the Wi-Fi is open and does not
|
||||
require a password. Using \texttt{join} with the empty string (\texttt{join ""})
|
||||
means the kernel will try to connect to any open Wi-Fi if no Wi-Fi
|
||||
from the join-list is found first.
|
||||
|
||||
We still need to configure the name and password by editing a file
|
||||
in \texttt{/etc/} and run netstart(8) when we encounter a new Wi-Fi. This is
|
||||
probably not the best UI\footnote{ed(1) is the
|
||||
pinnacle of UI design, as far as the author is concerned.} but the UX is pretty good and on par
|
||||
with a smartphone. Once the Wi-Fi has been configured by adding a \texttt{join}
|
||||
line, the kernel will automatically reconnect to a known Wi-Fi
|
||||
whenever it comes within range.
|
||||
\section{Stop slacking}
|
||||
\label{sec:orgcc75305}
|
||||
Now that we are connected to the Wi-Fi, we need to configure IP
|
||||
addresses.
|
||||
|
||||
We started our efforts to improve the network configuration user
|
||||
experience with IPv6 for two reasons. One is that IPv6 is still a technology for early adopters
|
||||
who are used to difficulties when using new technologies and are eager to help debug the problems that might arise.
|
||||
|
||||
Another reason for our work was the fact that OpenBSD got IPv6 support from the KAME project
|
||||
in the late 1990s and early 2000s after which it quieted down again.
|
||||
The network configuration was handled mostly in the
|
||||
kernel, so there was no isolation from malicious input. For the most
|
||||
part it assumed a stationary workstation that tried to acquire an
|
||||
IPv6 prefix for stateless address auto-configuration during boot by
|
||||
sending three router solicitations and then listened for router
|
||||
advertisements to create auto-configuration addresses and renewed
|
||||
their lifetimes when a new advertisement flew by. There was some
|
||||
rudimentary code in rtsold(8) to handle movement between networks, but
|
||||
it was rarely used since it was optional. rtsold(8) was used in
|
||||
one-shot mode where it would send at most three router solicitations
|
||||
when an interface connected to the network and then it would exit.
|
||||
|
||||
We started to write \href{https://man.openbsd.org/slaacd.8}{slaacd(8)}
|
||||
and once that was working we could delete rtsold(8)
|
||||
and remove considerable pieces of code from the kernel.
|
||||
|
||||
slaacd(8) is a privilege-separated network daemon that build previous
|
||||
experience with privilege separation in OpenBSD. It uses three
|
||||
processes, (1) the \emph{parent} process to configure the system, (2) the
|
||||
\emph{frontend} process to talk to the outside world and (3) the \emph{engine}
|
||||
process to handle untrusted data and run a state machine for the
|
||||
stateless address auto-configuration protocol.
|
||||
|
||||
pledge(2) restricts what a process is allowed to do and this is
|
||||
enforced by the kernel. Enforcement means that the kernel will
|
||||
terminate processes that violate what they have pledged to do. The
|
||||
pledges themselves are in broad strokes, we do not concern ourselves
|
||||
with single system calls but with groups of system calls. For example,
|
||||
the process is allowed to interact with open file descriptors
|
||||
(\texttt{"stdio"}), it is allowed to open connections to hosts on the
|
||||
Internet (\texttt{"inet"}), and it is allowed to open files for reading
|
||||
(\texttt{"rpath"}).
|
||||
|
||||
The \emph{parent} process pledges that it will only open new network
|
||||
sockets, send those to other processes and reconfigure the routing
|
||||
table (\texttt{"stdio inet sendfd wroute"}). The \emph{frontend} process pledges
|
||||
to only receive file descriptors, open unix domain sockets and check
|
||||
the state of the routing table (\texttt{"stdio unix recvfd route"}). Checking
|
||||
the routing table includes seeing which flags are configured per
|
||||
interface. The \emph{engine} process pledges to only read and write to
|
||||
already open file-descriptors (\texttt{"stdio"}). The \emph{engine} process is
|
||||
very restricted in what it is allowed to do. This is important because it
|
||||
handles untrusted data coming from the network. While the \emph{frontend}
|
||||
process talks to the network, it never looks at the data. An attacker
|
||||
will not be able to confuse the \emph{frontend} process with data they
|
||||
send. They can and have \href{https://ftp.openbsd.org/pub/OpenBSD/patches/7.0/common/014\_slaacd.patch.sig}{confused} the \emph{engine} process.
|
||||
|
||||
For more details see \href{privsep.org}{"Privilege drop, privilege separation, and
|
||||
restricted-service operating mode in OpenBSD"}.
|
||||
|
||||
slaacd(8) is enabled per default on all OpenBSD installations.
|
||||
|
||||
IPv6 stateless address auto-configuration is enabled on an interface
|
||||
by setting the \texttt{AUTCONF6} flag using
|
||||
\href{https:///man.openbsd.org/ifconfig.8}{ifconfig(8)}: \texttt{ifconfig iwm0 inet6
|
||||
autoconf}. The kernel announces this changed interface flag to the
|
||||
whole system using a broadcast route message. slaacd(8) reads those
|
||||
messages using a \href{https://man.openbsd.org/route.4}{route(4)} socket.
|
||||
|
||||
slaacd(8) handles all aspects of stateless address
|
||||
auto-configuration. It sends router solicitations when needed, either
|
||||
multi-cast or uni-cast, depending on which is appropriate. It waits
|
||||
for router advertisements, parses them, and configures default routes,
|
||||
global and temporary IPv6 addresses, and passes name server
|
||||
information via a route message to the rest of the system. It takes
|
||||
care of the lifetimes of addresses, default routes, and name server
|
||||
information expiring and removes those from the system when no router
|
||||
advertisements are received to extend the lifetime.
|
||||
|
||||
slaacd(8) also monitors when network interfaces regain their
|
||||
connection to a network. For example because the laptop woke up from
|
||||
suspension or it got moved out of range of a Wi-Fi network and moved back
|
||||
within range. It then needs to find out whether it connected to the same
|
||||
network as before or whether it is now in a new network. If it is a new
|
||||
network we need to replace the old addresses, default route and name
|
||||
servers. If there is no IPv6 available it needs to remove the old
|
||||
information.
|
||||
|
||||
The stateless address auto-configuration specification allows multiple
|
||||
default routers being present on the same layer-two network,
|
||||
announcing the same or different network information. slaacd(8) tries
|
||||
to handle this, but this has not been extensively tested in all
|
||||
possible cases. There are still open questions being discussed at the
|
||||
IETF on how to run networks with different network prefixes in the
|
||||
same layer-two network.
|
||||
|
||||
slaacd(8) is able to handle multiple interfaces and we will show
|
||||
later how we pick the right source address when multiple addresses are available
|
||||
to choose from.
|
||||
|
||||
\section{Dynamic host configuration, please}
|
||||
\label{sec:org76b66c3}
|
||||
With IPv6 address configuration mostly solved, it was time to look at
|
||||
IPv4 again. We used a fork of ISC's dhclient(8). Henning Brauer
|
||||
(henning@) added privilege-separation to it and Kenneth Westerback (krw@)
|
||||
has been maintaining it in the past few years. However, the privilege-separation was never quite right which
|
||||
became more visible with the integration of pledge(2) and it turned out to be
|
||||
difficult to integrate some of the features we developed in slaacd(8).
|
||||
|
||||
It was time to write a new daemon and Otto Moerbeek (otto@) came up with a name for it: dhcpleased(8).
|
||||
It is pronounced as "dynamic host configuration, please" with the "d" silent.
|
||||
|
||||
On a very high level, IPv4 DHCP and IPv6 stateless address
|
||||
auto-configuration are very similar. We request some information from
|
||||
the router\footnote{In IPv6 we might not need to request the information,
|
||||
it might just show up unannounced.}, we use it to configure the system
|
||||
and we make sure that information does not expire. When we move
|
||||
networks we need to probe whether our information is still up to date and
|
||||
if not, reconfigure the system.
|
||||
|
||||
We opted for the obvious solution, which is to copy \texttt{sbin/slaacd} to \texttt{sbin/dhcpleased} and
|
||||
replace the IPv6 specific bits with IPv4 specific bits.
|
||||
|
||||
On paper DHCP looks more complicated than IPv6 stateless address
|
||||
auto-configuration because it negotiates with the server and there is
|
||||
a complicated state machine to implement.
|
||||
|
||||
In practice it is the other way around. The "stateless" part in IPv6
|
||||
does not apply to the client. The client must keep state and implement
|
||||
a state machine to keep track of which routers are available and when
|
||||
various information expires. In IPv4 we talk to one server and all
|
||||
information expires at the same time.
|
||||
|
||||
We will talk about a few differences between slaacd(8) and
|
||||
dhcpleased(8) in a moment, but from the user perspective both behave
|
||||
in the same way. They make sure that the address configuration and default
|
||||
gateway are always up to date and they pay attention when the machine
|
||||
moves between networks, either while awake or while sleeping.
|
||||
|
||||
Because dhcpleased(8) has to use \href{https://man.openbsd.org/bpf.4}{bpf(4)} instead of regular sockets for
|
||||
some of the network packets it needs to send, the \emph{parent} process
|
||||
cannot use pledge(2). Currently, there is nothing it could pledge that would
|
||||
allow the usage of bpf(4). To protect the system and
|
||||
prevent exfiltration of sensitive data we use \href{https://man.openbsd.org/unveil.2}{unveil(2)} to restrict
|
||||
the \emph{parent} process' view of the file system. dhcpleased(8) can only
|
||||
read its configuration file, read and write \texttt{/dev/bpf}, and read,
|
||||
write and create files underneath \texttt{/var/db/dhcpleased/} to store
|
||||
information about received leases.
|
||||
|
||||
While we could get away with not implementing a config file for
|
||||
slaacd(8), this did not work for dhcpleased(8). Some systems out
|
||||
there will only give us a DHCP lease if we send the correct \emph{client
|
||||
id}, for example.
|
||||
|
||||
There are many DHCP options specified in RFC 2132. We have only
|
||||
implemented the bare minimum, only the options we need and can
|
||||
handle. We do not need a swap server or a cookie server, to name a few.
|
||||
|
||||
Like slaacd(8), dhcpleased(8) is enabled on all OpenBSD
|
||||
installations.
|
||||
\section{Route priorities}
|
||||
\label{sec:orgefb99be}
|
||||
dhcpleased(8) and slaacd(8) can handle multiple interfaces at the same
|
||||
time. The routing table might look like this:
|
||||
|
||||
\begin{verbatim}
|
||||
$ netstat -nrf inet \
|
||||
| awk "{print $1,$2,$7,$8}"
|
||||
Routing tables
|
||||
|
||||
Internet:
|
||||
Destination Gateway Prio Iface
|
||||
default 192.168.1.1 8 em0
|
||||
default 192.168.178.1 12 iwm0
|
||||
[...]
|
||||
\end{verbatim}
|
||||
|
||||
We end up with two default routes, one gateway is reachable via the
|
||||
\emph{em0} interface with priority value 8 and the other gateway is
|
||||
reachable via the \emph{iwm0} interface with priority value 12. A route has
|
||||
higher priority when its priority value is lower. \emph{em0} is an Ethernet
|
||||
interface and it gets higher priority over the Wi-Fi interface
|
||||
\emph{iwm0}. All things being equal, the kernel will pick the address from
|
||||
\emph{em0} as source address when making a new connection to the internet
|
||||
and route traffic over the Ethernet interface, which is presumably
|
||||
faster.
|
||||
|
||||
If we pick up the laptop and unplug the Ethernet interface, the route over \emph{em0} is no longer usable and
|
||||
existing connections using it will stall and time out. New connections
|
||||
will instead use \emph{iwm0}.
|
||||
|
||||
If we plug the Ethernet interface \emph{em0} back in, the session might come alive again and new
|
||||
connections will use \emph{em0}. Connections that are running over \emph{iwm0}
|
||||
will continue working, because the interface is still connected to
|
||||
the Wi-Fi.
|
||||
|
||||
Applications like web browsers, email clients or even video
|
||||
conferencing systems will automatically establish a new connection
|
||||
when they notice that the old one is dead.
|
||||
|
||||
Unfortunately \href{https://man.openbsd.org/ssh.1}{ssh(1)} is not one of them. If switching between wired
|
||||
and wireless happens rarely, \href{https://man.openbsd.org/tmux.1}{tmux(1)} on the remote system might help
|
||||
with ssh(1) disconnects, or a \href{https://man.openbsd.org/wg.4}{wg(4)} tunnel can be used so that
|
||||
the source address does not change when switching between wired and
|
||||
wireless.
|
||||
\section{Cellular networks}
|
||||
\label{sec:org7d7219f}
|
||||
In addition to Ethernet and Wi-Fi networks, OpenBSD supports "Mobile
|
||||
Broadband Interface Model" devices using the \href{https://man.openbsd.org/umb.4}{umb(4)} driver. These can
|
||||
be used to connect to UMTS or LTE networks. They require a SIM card
|
||||
and after being configured using a PIN they will connect to cellular
|
||||
networks and automatically configure an IP address and default
|
||||
route. The default route has an even lower route priority than Wi-Fi
|
||||
so it will only be used when Ethernet and Wi-Fi are not connected.
|
||||
\section{It is always DNS}
|
||||
\label{sec:orgd2b8f30}
|
||||
Humans are not particularly good at remembering
|
||||
addresses like \texttt{2606:2800:220:1:248:1893:25c8:1946} and are much better
|
||||
with names like \emph{example.com}. When we run \texttt{ping6 example.com} we
|
||||
will end up in \href{https://man.openbsd.org/asr\_run.3}{libc's stub resolver}. It will open
|
||||
\texttt{/etc/resolv.conf} and look for \emph{nameserver} lines to use for DNS
|
||||
resolution.
|
||||
|
||||
We can learn name servers from dhcpleased(8), slaacd(8), umb(4),
|
||||
and \href{https://man.openbsd.org/iked.8}{iked(8)}. Historically dhclient(8) owned \texttt{/etc/resolv.conf}, which
|
||||
means that no other process could add name servers to it. dhclient(8)
|
||||
would just overwrite whatever was in there whenever it renewed its
|
||||
lease. This made it impossible to sometimes move to an IPv6-only
|
||||
network. slaacd(8) could not configure name servers and the left-over
|
||||
IPv4 name servers were not reachable.
|
||||
|
||||
We can either teach all name server sources to somehow cooperate and share responsibility of
|
||||
\texttt{/etc/resolv.conf} or we can run an arbitrator that collects name
|
||||
servers from diverse sources and handles the contents of
|
||||
\texttt{/etc/resolv.conf}.
|
||||
|
||||
\href{https://man.openbsd.org/resolvd.8}{resolvd(8)} is such an arbitrator. It is another always enabled
|
||||
daemon. It collects name servers from all the mentioned sources and
|
||||
adds them to \texttt{/etc/resolv.conf}.
|
||||
|
||||
It also monitors if \texttt{/etc/resolv.conf} gets edited in which case it
|
||||
rereads the file and makes sure that the learned name servers are at
|
||||
the beginning of the file. This is useful when the administrator of
|
||||
the machine decides to add options to \texttt{/etc/resolv.conf}. For example,
|
||||
we can edit the file and add \texttt{family inet6 inet} to prefer IPv6 over
|
||||
IPv4 and resolvd(8) will cope. There is no need for an extra
|
||||
configuration file, \texttt{/etc/resolv.conf} is the configuration file.
|
||||
|
||||
Name servers are announced using route messages and resolvd(8) listens
|
||||
for them using a route(4) socket. They can also be observed using the
|
||||
\href{https://man.openbsd.org/route.8}{route(8)} tool: \texttt{\$ route monitor}.
|
||||
|
||||
resolvd(8) can also request that name servers are re-announced by their
|
||||
sources. This is useful when resolvd(8) gets restarted.
|
||||
\section{Let us unwind a bit}
|
||||
\label{sec:orgab25916}
|
||||
Plain DNS is not a secure protocol. It exchanges
|
||||
unauthenticated UDP packets without any integrity protection. This
|
||||
makes it easy for an attacker to spoof answer packets.
|
||||
|
||||
DNS answer packets are untrusted data, they come from the
|
||||
network. However, the process that sends DNS queries and parses the
|
||||
answer using the libc functions is almost always the single main
|
||||
process of the tool. When we run \texttt{ping example.com}, DNS packets are
|
||||
parsed using our user. An attacker who can spoof a DNS answer might be
|
||||
able to trigger a bug in libc and gain code execution that way.
|
||||
|
||||
On OpenBSD, ping(8) pledges \texttt{"stdio DNS"} so the attacker will not get
|
||||
very far, but there are many more programs in ports that are not
|
||||
pledged that might want to resolve names.
|
||||
|
||||
It would be worthwhile to have some sort of proxy running on localhost
|
||||
such that DNS packets from the outside need to traverse a well locked
|
||||
down process running in a different address-space and as a different
|
||||
user than the program that needs to resolve a name.
|
||||
|
||||
An early experiment was rebound(8), written by Ted Unangst (tedu@). It
|
||||
was simplistic and did not understand DNS at all, it would just
|
||||
forward packets, but it would sit between the Internet and the
|
||||
program.
|
||||
|
||||
An alternative is to run a full recursive resolver like \href{https://man.openbsd.org/unbound.8}{unbound(8)} on
|
||||
the laptop, but this leads to problems, too. unbound(8) expects a well
|
||||
working network where nothing interferes with DNS, this is true in data
|
||||
centres and can be achieved in well maintained home networks, but it
|
||||
is not something we find when moving laptops to arbitrary networks
|
||||
like free Wi-Fi in a hotel or at an airport.
|
||||
|
||||
It turns out that often the quality of the network changes over
|
||||
time. When we first connect to a hotel Wi-Fi we may find ourselves in
|
||||
what is referred to as a \emph{captive portal}. Everything is blocked, DNS
|
||||
gets intercepted, and we are redirected to a website where we need to
|
||||
agree to the terms and conditions and maybe provide our name and room
|
||||
number. Once we are past that, network quality improves considerably
|
||||
and we are mostly free to talk to the outside world.
|
||||
|
||||
This is where \href{https://man.openbsd.org/unwind.8}{unwind(8)} comes in.
|
||||
It is another privilege-separated
|
||||
network daemon that provides a recursive name server for the local
|
||||
machine. resolvd(8) detects when it is running and automatically
|
||||
rewrites \texttt{/etc/resolv.conf} to have only \texttt{nameserver 127.0.0.1} listed
|
||||
as name server.
|
||||
|
||||
This solves or improves upon the first problem. Programs that need DNS resolution are insulated from
|
||||
the Internet. An attacker needs to get past unwind(8) first before
|
||||
they can try to attack the libc stub resolver.
|
||||
|
||||
unwind(8) understands and speaks DNS and it actively observes the
|
||||
network quality.
|
||||
|
||||
We did not write our own recursive name server. That would be
|
||||
difficult and since DNS is constantly evolving, it would also require extra work to keep up. Instead we
|
||||
decided to use libunbound, which is
|
||||
part of \href{https://man.openbsd.org/unbound.8}{unbound(8)}. It is developed under a BSD license by \href{https://www.nlnetlabs.nl/}{NLnet Labs}.
|
||||
|
||||
The resolver process pledges \texttt{"stdio inet dns rpath"} and
|
||||
restricts access to the file system using unveil(2) to
|
||||
\texttt{/etc/ssl/cert.pem}. This is the process that is exposed to the
|
||||
Internet and handles untrusted data. It would be preferable to have
|
||||
one process exposed to the Internet and another to parse untrusted
|
||||
data but that is not possible to do with libunbound.
|
||||
|
||||
Since we are using a real recursive name server, that gives us many
|
||||
options on how we can resolve names:
|
||||
\begin{itemize}
|
||||
\item We can do our own recursion, walk down from the root zone using
|
||||
qname minimization to improve privacy.
|
||||
\item We can use the name server we learned from dhcpleased(8) and
|
||||
slaacd(8) as forwarders, so we do not need to do our own recursion,
|
||||
which might be faster.
|
||||
\item We can try to opportunistically speak DNS over TLS (DoT) to the
|
||||
learned name servers to prevent eavesdroppers from listening in.
|
||||
\item We can configure forwarders manually to not depend on the network
|
||||
provided name servers. Those might be more trustworthy. They can
|
||||
also be DoT forwarders to prevent eavesdropping.
|
||||
\item As a last resort, unwind(8) can behave exactly like the libc stub
|
||||
resolver\footnote{Call this the "Dutch train problem": the free Wi-Fi on
|
||||
Dutch trains do not like DNS queries with an \emph{EDNS0} option, they
|
||||
intercept them, do not understand them, and answer \emph{NXDOMAIN}. There
|
||||
are other free Wi-Fi networks that are similarly broken.}.
|
||||
\end{itemize}
|
||||
We call these resolving strategies and unwind(8) actively probes if
|
||||
they are usable by sending test queries when it notices that the
|
||||
network changed, for example because the laptop moved to a different Wi-Fi
|
||||
network or woke up from suspension. It then orders them by quality and
|
||||
picks the best one.
|
||||
|
||||
There is an implicit skew in the strategies for finding the best one:
|
||||
a manually configured DoT name server is always considered better than
|
||||
a name server provided by the local network, as long as it is available
|
||||
and not too slow.
|
||||
|
||||
unwind(8) is not too concerned about preserving privacy, it is
|
||||
pragmatic and tries to resolve names the best way it can, and it will use
|
||||
the local name servers provided by the network if those are the only ones available.
|
||||
|
||||
Since unwind(8) uses libunbound it also supports DNSSEC. DNSSEC
|
||||
provides data integrity and cryptographic authenticity, it does not
|
||||
provide confidentiality.
|
||||
|
||||
unwind(8) is pragmatic about DNSSEC. When it tests the quality of a
|
||||
resolving strategy it also tries to find out if DNSSEC is
|
||||
available. There are many reasons why DNSSEC might not be available: the
|
||||
network is misconfigured, DNSSEC is blocked or the laptop
|
||||
does not (yet) have the correct time. If DNSSEC does not work, unwind(8) does not insist on using it.
|
||||
|
||||
Of course this makes it susceptible to a downgrade attack. To mitigate
|
||||
this, unwind(8) will insist on DNSSEC working after it has discovered once
|
||||
that DNSSEC is working in the local network. This means that an
|
||||
attacker needs to be able to block DNSSEC from the moment we connect
|
||||
to a network. They cannot show up later and try to downgrade
|
||||
us. unwind(8) will only become lenient again when we connect to a new
|
||||
network.
|
||||
|
||||
This is not a strong mitigation of course, but DNSSEC is not a fix for
|
||||
everything at the resolver. Applications also need
|
||||
to do their part and decide how much they are willing to trust
|
||||
DNS. For example ssh(1)'s \emph{VerifyHostKeyDNS} feature will only trust
|
||||
host key fingerprints it obtained from DNS if they were validated
|
||||
using DNSSEC and the validator runs on the local
|
||||
machine\footnote{Technically not entirely true, ssh(1) trusts what libc
|
||||
indicates and libc automatically trusts localhost. See \emph{trust-ad} in
|
||||
\href{https://man.openbsd.org/resolv.conf.5}{resolv.conf(5)}.}. Otherwise it will ask the user what to do.
|
||||
|
||||
A worst-case scenario when joining a partially broken Wi-Fi network
|
||||
with captive portal and a manually configured DoT name server might
|
||||
look like this:
|
||||
\begin{enumerate}
|
||||
\item We connect to the network, we cannot reach the DoT name server and
|
||||
cannot do our own recursion.
|
||||
\item unwind(8) will choose the name server provided by the
|
||||
network. It also notes that we just connected to a new network so
|
||||
it is lenient with respect to DNSSEC validation. In effect it will
|
||||
ignore validation errors.
|
||||
\item We try to access a website and the captive portal
|
||||
detection in the browser triggers. We click the buttons and fill in
|
||||
the forms until we are allowed on the internet.
|
||||
\item unwind(8) notices that it can do its own recursion.
|
||||
\item At the same time, unwind(8) notices that the DoT name server is
|
||||
also reachable now and starts using it.
|
||||
\end{enumerate}
|
||||
|
||||
unwind(8) does not natively support DNS over HTTPS (DoH) and we
|
||||
sometimes find ourselves in networks that block everything except for
|
||||
TCP port 443. One way around this is to use dnscrypt-proxy from ports
|
||||
which does support DoH. We can point unwind(8) at it by manually
|
||||
configuring a plain DNS forwarder in addition to a DoT forwarder:
|
||||
\begin{verbatim}
|
||||
$ cat /etc/unwind.conf
|
||||
forwarder "9.9.9.9" port 853 \
|
||||
authentication name "dns.quad9.net" DoT
|
||||
forwarder "2620:fe::9" port 853 \
|
||||
authentication name "dns.quad9.net" DoT
|
||||
# dnscrypt-proxy for DoH
|
||||
forwarder "127.0.0.1" port 5353
|
||||
\end{verbatim}
|
||||
|
||||
\section{Time for gelato}
|
||||
\label{sec:orgbb01a19}
|
||||
There are various transition technologies that get us from an IPv4-only
|
||||
Internet to an IPv6-only Internet. We will only look at \emph{NAT64},
|
||||
\emph{DNS64}, and \emph{464XLAT}.
|
||||
|
||||
\emph{NAT64} allows us to reach IPv4 hosts from an IPv6-only network by
|
||||
pretending that the hosts are IPv6 enabled. IPv6 addresses are so large
|
||||
that we can easily encode all of IPv4 in an IPv6 /64 prefix, which is
|
||||
the usual size of on IPv6 prefix we see per layer-two network. In fact
|
||||
we don't need the whole /64, a /96 is enough to encode the whole IPv4
|
||||
Internet.
|
||||
|
||||
Let us pretend we know the /96 prefix used for \emph{NAT64} and the IPv4
|
||||
address we want to reach. Forming an IPv6 address for the host is then
|
||||
simply a bitwise-or operation of the IPv4 address with the /96 prefix,
|
||||
the IPv4 address fills in the lower bits of the IPv6 prefix. This is
|
||||
called address synthesis.
|
||||
|
||||
We can then use this address to connect to the IPv4-only
|
||||
host. Somewhere on the network path is the \emph{NAT64} gateway that is
|
||||
dual stacked. It knows that our packets are using \emph{NAT64} because it
|
||||
is configured with the /96 prefix. It intercepts the packets and forms
|
||||
IPv4 packets and sends them on their way. The gateway needs to be
|
||||
stateful to be able to \emph{NAT} the return traffic back to us.
|
||||
|
||||
We use DNS to find out the IPv4 address that we want to connect to. The
|
||||
local name servers that slaacd(8) learned about would know
|
||||
about the \emph{NAT64} prefix used in the network and do the address
|
||||
synthesis for us. This is called \emph{DNS64}. The problem with this is that
|
||||
the name servers spoof DNS answers, something that DNSSEC tries to prevent.
|
||||
unwind(8) will detect this and generate an error, or
|
||||
unwind(8) might not even talk to the designated name servers at all.
|
||||
|
||||
To get around this, unwind(8) itself can detect the presence of \emph{DNS64}
|
||||
on a network by asking the local name servers for the \emph{AAAA} record,
|
||||
i.e. the IPv6 address, for something that is guaranteed to never have
|
||||
one: \emph{ipv4only.arpa}. If it gets an answer, it can reverse the address
|
||||
synthesis and learn the \emph{NAT64} prefix. With that information it can
|
||||
do \emph{DNS64} itself and there is no longer a problem with DNSSEC.
|
||||
|
||||
The downsides of this mechanism are that it is quite complicated, it
|
||||
messes around with DNS, and it does not work with IPv4 address
|
||||
literals. It also does not work with programs that are fundamentally
|
||||
IPv4-only: \texttt{ping example.com} will never work in an IPv6-only network
|
||||
with only \emph{NAT64 / DNS64}.
|
||||
|
||||
Instead of pretending that the IPv4 host we want to reach has IPv6, we can
|
||||
pretend to have working IPv4 if a \emph{NAT64} gateway is present. We ask
|
||||
the kernel via the \href{https://man.openbsd.org/pf.4}{pf(4)} firewall to do the IPv4-to-IPv6 translation
|
||||
for us. The \emph{NAT64} gateway will then do the reverse translation and
|
||||
send an IPv4 packet on its way. This is called \emph{464XLAT}.
|
||||
|
||||
We first need an IPv4 address, RFC 7335 reserved \texttt{192.0.0.0/29} for
|
||||
this purpose:
|
||||
\begin{verbatim}
|
||||
ifconfig pair1 inet 192.0.0.4/29
|
||||
\end{verbatim}
|
||||
We then need a default gateway:
|
||||
\begin{verbatim}
|
||||
ifconfig pair2 rdomain 1
|
||||
ifconfig pair2 inet 192.0.0.1/29
|
||||
\end{verbatim}
|
||||
Because pf(4) will only do address family translation on inbound rules
|
||||
we need a different \emph{rdomain} and use \href{https://man.openbsd.org/pair.4}{pair(4)} interfaces. We need to
|
||||
connect them:
|
||||
\begin{verbatim}
|
||||
ifconfig pair1 patch pair2
|
||||
\end{verbatim}
|
||||
And then we can configure our default route:
|
||||
\begin{verbatim}
|
||||
route add -host -inet default 192.0.0.1 \
|
||||
-priority 48
|
||||
\end{verbatim}
|
||||
We set it to a very low priority\footnote{Remember that a high priority
|
||||
\textbf{value} means low priority.} so that it does not interfere with routes
|
||||
that dhcpleased(8) configures when we move to an IPv4 enabled network.
|
||||
|
||||
We then need to configure address family translation in pf(4) when we
|
||||
detect \emph{NAT64} being present. This is were \href{https://github.com/fobser/gelatod/}{gelatod(8)} comes in. It is
|
||||
a Customer-side transLATor (\emph{CLAT}) configuration daemon. \emph{CLAT}
|
||||
is what \emph{464XLAT} calls the address translation happening on the laptop.
|
||||
|
||||
gelatod(8) is yet another privilege-separated daemon\footnote{At this point
|
||||
we will not go into pledge details.} that checks for the presence of a \emph{NAT64}
|
||||
gateway whenever we change networks. It does so either via the
|
||||
\emph{ipv4only.arpa} trick or explicitly via router advertisements. RFC
|
||||
8781 specifies how a network can signal the presence of a \emph{NAT64}
|
||||
gateway.
|
||||
|
||||
gelatod(8) needs a pf(4) anchor into which it adds rules that are
|
||||
similar to this example:
|
||||
\begin{verbatim}
|
||||
pass in log quick on pair2 inet \
|
||||
af-to inet6 \
|
||||
from 2001:db8::da68:f613:4573:4ed0 \
|
||||
to 64:ff9b::/96 \
|
||||
rtable 0
|
||||
\end{verbatim}
|
||||
The rule is doing address family translation to IPv6 on incoming
|
||||
packets on \texttt{pair2}. In this example it uses
|
||||
\texttt{2001:db8::da68:f613:4573:4ed0} as the IPv6 source address, gelatod(8)
|
||||
learned this from the system when slaacd(8) configured
|
||||
it. \texttt{64:ff9b::/96} is the learned \emph{NAT64} prefix and we are moving
|
||||
traffic back to \texttt{rtable 0}. Remember \texttt{pair2} is in rdomain 1.
|
||||
|
||||
While this works rather well, it is also complicated to set up,
|
||||
which is why gelatod(8) is not in
|
||||
OpenBSD base but lives in ports. We believe in good defaults in
|
||||
OpenBSD and try to make it easy for the user.
|
||||
|
||||
\section{Future work}
|
||||
\label{sec:org3bacedc}
|
||||
We would like to have the functionality of gelatod(8) in OpenBSD base.
|
||||
gelatod(8) was mostly a proof of concept and we imagine that a new network device
|
||||
like clat(4) would take over the role of client side address family
|
||||
translation. It could be always present and gelatod(8) would just enable
|
||||
and disable it. At that point we could move the functionality into
|
||||
slaacd(8) and delete gelatod(8). \emph{CLAT} is defined as a stateless
|
||||
mechanism so it does not need the full pf(4) machinery for address
|
||||
family translation.
|
||||
|
||||
It would be valuable to have DNS over HTTPS (DoH) and DNS over Quic (DoQ)
|
||||
natively in unwind(8). We are mostly waiting on upstream to implement
|
||||
support in unbound(8).
|
||||
|
||||
There are also minor issues that could be improved:
|
||||
\begin{itemize}
|
||||
\item The captive portal detection in unwind(8) is not perfect and could be improved upon.
|
||||
\item dhcpleased(8) and slaacd(8) should remember IP addresses from
|
||||
networks they have been connected to previously, to be able to quickly
|
||||
re-establish connectivity by probing whether we are connecting to a
|
||||
previous network while the lifetime of our addresses have not expired
|
||||
yet. RFC 4436 "Detecting Network Attachment in IPv4 (DNAv4)" and RFC
|
||||
6059 "Simple Procedures for Detecting Network Attachment in IPv6"
|
||||
discuss the details.
|
||||
\item It would be helpful if the dhcpleased(8) parent process could be
|
||||
pledged. This is currently not possible because of bpf(4). Things to
|
||||
investigate here are changes to the network stack that would allow
|
||||
us to use raw sockets instead of bpf(4) sockets or the ability to
|
||||
\href{https://man.openbsd.org/dup.2}{dup(2)} an existing bpf(4) socket and reprogram the interface it is
|
||||
using.
|
||||
\end{itemize}
|
||||
\section{Conclusion}
|
||||
\label{sec:org29b4bac}
|
||||
In this paper we have described how OpenBSD improved the user
|
||||
experience and security of laptop users when visiting diverse network
|
||||
locations. The system remembers Wi-Fi networks and automatically
|
||||
connects to them. It automatically discovers when the network changes
|
||||
and acquires new IPv4 and IPv6 addresses or renews existing
|
||||
configurations. OpenBSD also actively probes available DNS resolving
|
||||
strategies and picks the best one available. Privilege-separation and
|
||||
restricted service operating mode ensure that untrusted data is parsed
|
||||
with the least privileges necessary, protecting the rest of the
|
||||
system.
|
||||
\section*{Acknowledgment}
|
||||
The author would like to thank Mine Temuerhan for copyediting the paper.
|
||||
\end{document}
|
||||
@ -1,249 +0,0 @@
|
||||
#+TITLE: DHCPv6-PD - First steps
|
||||
#+DATE: 2024-05-26
|
||||
* Prologue
|
||||
The single most requested feature missing in OpenBSD base directed at
|
||||
me is DHCPv6-PD. Recently I got a working setup at home using [[https://roy.marples.name/projects/dhcpcd][dhcpcd]]
|
||||
from ports and a donated Fritz!Box 6660 Cable[fn::Thanks again
|
||||
Mischa & Ibsen!][fn::The CPE from my ISP was just too broken to work with, let
|
||||
alone develop against. It only ever hands out a single prefix and
|
||||
would need a factory reset afterwards. It also does not route the
|
||||
delegated prefix but depends on [[https://www.rfc-editor.org/rfc/rfc4389][ND Proxy.]]]. Time to hack on this.
|
||||
* DHCPv6-PD
|
||||
[[https://www.rfc-editor.org/rfc/rfc8415][DHCPv6]] is not wildly deployed outside of enterprise
|
||||
networks[fn::Because of android refusing to implement it.]. DHCP for
|
||||
Prefix Delegation (DHCPv6-PD) on the other hand is *the standard* to
|
||||
get IPv6 prefixes into home networks. [[https://www.rfc-editor.org/rfc/rfc8415#section-6.3][RFC 8415]] has this:
|
||||
#+begin_quote
|
||||
It is appropriate for situations in which the delegating router (1)
|
||||
does not have knowledge about the topology of the networks to which
|
||||
the requesting router is attached and (2) does not require other
|
||||
information aside from the identity of the requesting router to choose
|
||||
a prefix for delegation. This mechanism is appropriate for use by an
|
||||
ISP to delegate a prefix to a subscriber, where the delegated prefix
|
||||
would possibly be subnetted and assigned to the links within the
|
||||
subscriber's network.
|
||||
#+end_quote
|
||||
* Transmogrifying dhcpleased(8)
|
||||
This not being my first rodeo, it took me about 4 hours over a weekend
|
||||
to transmogrify [[https://man.openbsd.org/dhcpleased.8][=dhcpleased(8)=]] into =dhcp6leased(8)= and have it talk
|
||||
to my Fritz!Box. I have also setup [[https://www.isc.org/kea/][ISC's Kea DHCP server]] for easier
|
||||
development and to not risk my production network at home. Of course
|
||||
it is not yet able to configure the system, but it can request a
|
||||
prefix delegation from the server and parse the response. This is
|
||||
enough to play with the protocol and work on the grammar for the
|
||||
configuration file.
|
||||
* Describing network topology
|
||||
=dhcp6leased(8)= will not just request an IPv6 prefix delegation but
|
||||
also use the delegated prefix to assign prefixes to downstream network
|
||||
interfaces. [[https://man.openbsd.org/rad.8][rad(8)]] can then be used to send router advertisements for
|
||||
clients to get IPv6 connectivity on different subnets in the home
|
||||
network.
|
||||
|
||||
The typical use case is probably to have a few networks connected to
|
||||
the OpenBSD router using vlans[fn::Maybe one vlan for WiFi, one for IOT
|
||||
and one for guest WiFi.] and assign =/64= prefixes to each one of
|
||||
them.
|
||||
|
||||
A more advanced use case would be to assign prefixes of different
|
||||
lengths to the vlan interfaces. For example I have a whole (virtual)
|
||||
network lab hanging off of an OpenBSD router which is not a single
|
||||
flat network. I need to assign a =/60= to that interface[fn::The IETF
|
||||
never managed to fully standardize this. I hear this is where homenet
|
||||
failed. A less ambitious working group is working in this problem
|
||||
space now: [[https://datatracker.ietf.org/wg/snac/about/][Stub Network Auto Configuration for IPv6 (snac)]]. But they
|
||||
only want to deal with flat networks.] to have enough space to subnet
|
||||
further.
|
||||
|
||||
Now, DHCPv6-PD allows us to request multiple prefixes. We could just
|
||||
punt the problem of splitting a bigger prefix into smaller prefixes to
|
||||
the DHCPv6 server. However, the [[https://www.rfc-editor.org/rfc/rfc8415#section-6.6][RFC has this]]:
|
||||
|
||||
#+begin_quote
|
||||
In principle, DHCP allows a client to request new prefixes to be
|
||||
delegated by sending additional IA_PD options (see Section 21.21).
|
||||
However, a typical operator usually prefers to delegate a single,
|
||||
larger prefix. In most deployments, it is recommended that the client
|
||||
request a larger prefix in its initial transmissions rather than
|
||||
request additional prefixes later on.
|
||||
#+end_quote
|
||||
|
||||
And indeed, the Fritz!Box only gives us one prefix. We can hand the
|
||||
prefix back and request a larger one, but it will only honour a single
|
||||
=IA_PD= option in a solicit message.
|
||||
|
||||
This means we have to split up the prefix ourselves. This is perfectly
|
||||
simple if we are only dealing with =/64= networks. Just count the
|
||||
networks, round up to the nearest power of two and calculate the
|
||||
required prefix size from that.
|
||||
|
||||
This gets more complicated if the prefix lengths for our sub-networks
|
||||
are non-uniform, like in the more advanced use case.
|
||||
|
||||
I went a bit on a tangent and tried to solve this for the general
|
||||
case. That means arbitrary subnet sizes and an optimal packing in the
|
||||
delegated prefix. I think that would come down to the [[https://en.wikipedia.org/wiki/Bin_packing_problem][Bin packing
|
||||
problem]] which is... annoying[fn::Otherwise known as NP-hard.].
|
||||
|
||||
I then noticed that we want a stable assignment, meaning when we add
|
||||
or remove an interface we do not want to renumber all the existing and
|
||||
remaining interfaces. Which would happen if try to come up with an
|
||||
optimal solution because prefix assignments would most likely shift
|
||||
around every time we change something.
|
||||
* dhcpcd's solution
|
||||
At this point I was somewhat stuck and I had a look at how dhcpcd
|
||||
deals with this. While I was already using dhcpcd in my network, I had
|
||||
not yet setup the more advanced use case with a =/60= and multiple
|
||||
=/64=. I was pretty sure that dhcpcd can handle this, but I did not
|
||||
yet know how.
|
||||
|
||||
Disclaimer: What follows are my notes on how I got it to work. It is
|
||||
likely that I am doing things wrong and misunderstand some
|
||||
parts. Unfortunately I no longer have access to GitHub[fn::An
|
||||
alternative reading is: I refuse to use it because they decided I am a
|
||||
suplier, which I am not. And they locked me out of my account.], so I
|
||||
cannot open an issue with the project to ask for help with this. I am
|
||||
very sorry.
|
||||
|
||||
Here is the relevant part from the [[https://man.freebsd.org/cgi/man.cgi?query=dhcpcd.conf&apropos=0&sektion=0&manpath=FreeBSD+14.0-RELEASE+and+Ports&arch=default&format=html][dhcpcd.conf man page:]]
|
||||
#+begin_example
|
||||
ia_pd [iaid [/ prefix / prefix_len] [interface [/ sla_id [/ prefix_len
|
||||
[/ suffix]]]]]
|
||||
Request a DHCPv6 Delegated Prefix for iaid. This option must
|
||||
be used in an interface block. Unless a sla_id of 0 is as-
|
||||
signed with the same resultant prefix length as the delegation,
|
||||
a reject route is installed for the Delegated Prefix to stop
|
||||
unallocated addresses being resolved upstream. If no interface
|
||||
is given then we will assign a prefix to every other interface
|
||||
with a sla_id equivalent to the interface index assigned by the
|
||||
OS. Otherwise addresses are only assigned for each interface
|
||||
and sla_id. To avoid delegating to any interface, use - as the
|
||||
invalid interface name. Each assigned address will have a
|
||||
suffix, defaulting to 1. If the suffix is 0 then a SLAAC ad-
|
||||
dress is assigned. You cannot assign a prefix to the request-
|
||||
ing interface unless the DHCPv6 server supports the RFC 6603
|
||||
Prefix Exclude Option. dhcpcd has to be running for all the
|
||||
interfaces it is delegating to. A default prefix_len of 64 is
|
||||
assumed, unless the maximum sla_id does not fit. In this case
|
||||
prefix_len is increased to the highest multiple of 8 that can
|
||||
accommodate the sla_id. sla_id is an integer which must be
|
||||
unique inside the iaid and is added to the prefix which must
|
||||
fit inside prefix_len less the length of the delegated prefix.
|
||||
You can specify multiple interface / sla_id / prefix_len per
|
||||
ia_pd, space separated. IPv6RS should be disabled globally
|
||||
when requesting a Prefix Delegation.
|
||||
#+end_example
|
||||
I kinda do not know what all of this means.
|
||||
|
||||
After much experimentation I ended up with this working-ish
|
||||
configuration:
|
||||
#+begin_example
|
||||
ia_pd 2/::/59 vether0/0/60 vether1/1/64
|
||||
#+end_example
|
||||
which put this in =daemon.log=:
|
||||
#+begin_example
|
||||
vio1: delegated prefix 2001:db8:3::/56
|
||||
vether0: adding address 2001:db8:3::1/60
|
||||
vether1: adding address 2001:db8:3:1::1/64
|
||||
#+end_example
|
||||
|
||||
A closer look shows that the two prefixes overlap though:
|
||||
#+begin_src python
|
||||
>>> import ipaddress
|
||||
>>> a = ipaddress.ip_network('2001:db8:3::/60')
|
||||
>>> b = ipaddress.ip_network('2001:db8:3:1::/64')
|
||||
>>> a.overlaps(b)
|
||||
True
|
||||
#+end_src
|
||||
|
||||
This configuration produces non-overlapping prefix assignments:
|
||||
#+begin_example
|
||||
ia_pd 2/::/59 vether0/0/60 vether1/16/64
|
||||
#+end_example
|
||||
|
||||
#+begin_example
|
||||
vio1: delegated prefix 2001:db8:3::/56
|
||||
vether0: adding address 2001:db8:3::1/60
|
||||
vether1: adding address 2001:db8:3:10::1/64
|
||||
#+end_example
|
||||
|
||||
Taking this apart, token by token:
|
||||
+ =ia_pd= :: This is just the keyword to request a prefix delegation.
|
||||
+ =2/::/59= :: 2 is a unique request ID needed by the DHCPv6
|
||||
protocol. =::= is the unspecified prefix and 59 is the requested
|
||||
prefix length. Since the DHCPv6 server does not have an address pool
|
||||
for =/59= it hands out a prefix for the next larger prefix for which
|
||||
it does have a pool, =/56= in this case.
|
||||
+ =vether0/0/60= :: This assigns the 1st (index 0) =/60= prefix to
|
||||
=vether0=.
|
||||
+ =vether1/16/64= :: This assigns the 17th[fn::We are starting to
|
||||
count at 0.] (index 16) =/64= prefix to =vether1=.
|
||||
|
||||
What I misunderstood when I used =vether1/1/64= was that =sla_id=
|
||||
(the 1 in the middle) does not mean use the next free =/64= but use
|
||||
the 2nd =/64= in the delegated prefix.
|
||||
|
||||
I find this confusing because the way I think about subnetting is that
|
||||
the different prefixes do not stand alone. =2001:db8:3:10::/64= is not
|
||||
the 17th =/64= prefix in =2001:db8:3::/56= but the first =/64= in the
|
||||
2nd =/60=. It's a hierarchy.
|
||||
|
||||
* Next steps
|
||||
dhcpcd puts a lot of work on the administrator to get the subnet
|
||||
assignments just right. It neatly avoids[fn::You could say it punts
|
||||
them to the administrator.] the problems I had identified. The
|
||||
assignments are stable and the algorithm is not massively expensive.
|
||||
|
||||
This got me unstuck and I have an idea how =dhcp6leased(8)= should be
|
||||
configured.
|
||||
1. It should work out automatically the size of the prefix it
|
||||
requests. I was under the impression that dhcpcd would also do
|
||||
that, but it did not work. Probably my mistake somewhere.
|
||||
2. Assignments are listed in order and =dhcpleased(8)= will work out
|
||||
the boundaries.
|
||||
|
||||
I am not sure about the exact syntax, but as an example, consider
|
||||
this:
|
||||
|
||||
#+begin_example
|
||||
request prefix delegation on vio0 for {
|
||||
vether0/60
|
||||
reserve/60
|
||||
vether1/64
|
||||
vether2/64
|
||||
vether3/60
|
||||
}
|
||||
#+end_example
|
||||
|
||||
It would request a =/58= which fits 4 =/60=. We assign the first =/60=
|
||||
to =vether0=, keep the next =/60= in reserve in case we want to add
|
||||
interfaces between =vether0= and =vether1= in the future without
|
||||
triggering a renumber. We then pick the first =/64= out of the third
|
||||
=/60= and assign it to =vether1=. We still have space in the third
|
||||
=/60= to assign a =/64= to =vether2=. We pick the fourth and last
|
||||
=/60= and assign it to =vether3=:
|
||||
|
||||
#+begin_example
|
||||
vether0 2001:db8:3::/60
|
||||
reserve 2001:db8:3:10::/60
|
||||
vether1 2001:db8:3:20::/64
|
||||
vether2 2001:db8:3:21::/64
|
||||
vether3 2001:db8:3:30::/60
|
||||
#+end_example
|
||||
|
||||
I think I have code that can do this and it is not overly
|
||||
complicated. It can currently only handle the upper 64 bits of an IPv6
|
||||
address because it does math on =uint64_t=. I will try to extend it to
|
||||
the lower half so that we can assign something like =/96= to a link,
|
||||
even if that means that half the [[https://datatracker.ietf.org/wg/6man/about/][IPv6 Maintenance (6man)]] IETF working
|
||||
group will hunt me down.
|
||||
|
||||
* Epilogue
|
||||
This strikes a slightly better balance between work that needs to be
|
||||
done by the administrator and help the tool provides compared to what
|
||||
dhcpcd implements. But I would not have come up with this without
|
||||
prior work by dhcpcd, kudos to Roy.
|
||||
|
||||
Coming up with an addressing plan is still hard work, so I will
|
||||
implement a feature in =dhcp6leased= to have it output the addressing
|
||||
plan it worked out as a configuration check before going to
|
||||
work. Because renumbering is hard.
|
||||
@ -591,7 +591,7 @@ We set it to a very low priority[fn:: Remember, a high priority
|
||||
dhcpleased(8) configures when we move to an IPv4 enabled network.
|
||||
|
||||
We then need to configure address family translation in pf(4) when we
|
||||
detect /NAT64/ being present. This is were [[https://codeberg.org/fobser/gelatod][gelatod(8)]] comes in. It is
|
||||
detect /NAT64/ being present. This is were [[https://github.com/fobser/gelatod/][gelatod(8)]] comes in. It is
|
||||
a Customer-side transLATor (/CLAT/) configuration daemon[fn::If you
|
||||
squint just right, gelato kinda sounds like clat[fn::Again, I really
|
||||
really should be prohibited from naming things.].]. /CLAT/ is what
|
||||
|
||||
@ -50,8 +50,8 @@ comes down to opening the correct file. So we can copy the code of
|
||||
(if (listp value)
|
||||
(mapcar 'pop-to-buffer-same-window (nreverse value))
|
||||
(pop-to-buffer-same-window value))
|
||||
(text-mode)
|
||||
(read-only-mode)))
|
||||
(read-only-mode)
|
||||
(text-mode)))
|
||||
|
||||
;; RFCs are stored in 'in-notes/'
|
||||
(defun ietf-rfc (filename &optional wildcards)
|
||||
@ -66,15 +66,10 @@ comes down to opening the correct file. So we can copy the code of
|
||||
(if (listp value)
|
||||
(mapcar 'pop-to-buffer-same-window (nreverse value))
|
||||
(pop-to-buffer-same-window value))
|
||||
(text-mode)
|
||||
(read-only-mode)))
|
||||
(read-only-mode)
|
||||
(text-mode)))
|
||||
#+end_src
|
||||
|
||||
*Update 2024-03-25*: Flipping the order of ~(text-mode)~ and
|
||||
~(read-only-mode)~ so that ~(read-only-mode)~ gets activated last
|
||||
makes it work correctly with ~view-mode~ if that mode is automatically
|
||||
activated by ~(setq view-read-only t)~ on read-only buffers.
|
||||
|
||||
Setting a global key binding lets us open RFCs and drafts from
|
||||
anywhere within Emacs:
|
||||
#+begin_src emacs-lisp
|
||||
|
||||
10
index.org
10
index.org
@ -1,5 +1,5 @@
|
||||
#+TITLE: Florian Obser
|
||||
#+DATE: 2024-09-25
|
||||
#+DATE: 2022-12-28
|
||||
+ [[https://openbsd.org][OpenBSD developer]]
|
||||
+ Senior DNS wrangler: [[https://www.ripe.net/analyse/dns/k-root/][k-root]] and [[https://www.ripe.net/analyse/dns/authdns][AuthDNS]]
|
||||
+ Long distance cyclist and short distance hiker.
|
||||
@ -7,11 +7,6 @@
|
||||
+ [[https://www.linkedin.com/in/florian-obser-75900383][Linkedin]]
|
||||
|
||||
* Meditations
|
||||
- [[file:new-sshagent-work.org][2024-07-16: new-sshagent-work]]
|
||||
- [[file:dhcpv6-pd-first-steps.org][2024-05-29: DHCPv6-PD - First steps]]
|
||||
- [[file:SingleFile.org][2024-03-20: SingleFile]]
|
||||
- [[file:openttd-srnw.org][2024-01-13: OpenTTD Self Regulating Networks]]
|
||||
- [[file:mastodon-backup.org][2023-07-26: Mastodon Backup]]
|
||||
- [[file:emacs-ietf.org][2023-04-05: Emacs IETF]]
|
||||
- [[file:dynamic_host_configuration_please.org][2023-03-07: Dynamic host configuration, please]]
|
||||
- [[file:privsep.org][2023-02-19: Privilege drop, privilege separation, and restricted-service operating mode in OpenBSD]]
|
||||
@ -37,8 +32,7 @@
|
||||
- [[http://www.openbsd.org/papers/bsdcan2019_unwind.pdf][2019-05-17, BSDCan: unwind(8) a privilege-separated, validating DNS recursive nameserver for every laptop]], [[https://www.youtube.com/watch?v=88SoI49nO4o][video]]
|
||||
- [[https://archive.fosdem.org/2020/schedule/event/dns_unwind/][2020-02-01, FOSDEM: unwind(8) a privilege-separated, validating DNS recursive nameserver for every laptop]]
|
||||
- [[https://undeadly.org/cgi?action=article;sid=20200922090542][2020-09-21, undeadly: k2k20 hackathon report: Florian Obser on DNS]]
|
||||
- [[https://2023.asiabsdcon.org/program.html.en][2023-04-02, AsiaBSDCon 2023: P09B: Dynamic Host Configuration, please]] ([[https://www.openbsd.org/papers/asiabsdcon2023-dynamic_host_configuration_please-slides.pdf][slides]]), ([[file:asiabsdcon2023/asiabsdcon_2023_final.pdf][paper]])
|
||||
- [[https://events.eurobsdcon.org/2024/talk/AV78U9/][2024-09-21, EuroBSDCon 2024: OpenBSD vs. IPv6]] ([[https://www.openbsd.org/papers/eurobsdcon2024-openbsd_vs_ipv6-slides.pdf][slides]])
|
||||
|
||||
* meta
|
||||
This page is made using [[https://orgmode.org/][Org Mode]]'s [[https://orgmode.org/manual/Publishing.html][publishing]] facility. The
|
||||
[[https://git.tlakh.xyz/florian/tlakh][repository]] is public. The style is [[https://simplecss.org/][Simple.css]] with a few local
|
||||
|
||||
@ -1,103 +0,0 @@
|
||||
#+TITLE: Mastodon Backup
|
||||
#+DATE: 2023-07-26
|
||||
* Prologue
|
||||
People on the Fediverse like to point out how you have to backup your
|
||||
data regularly in case of a catastrophic failure effecting your
|
||||
instance, the host of your instance, the data centre of your instance
|
||||
or maybe your admin. Heck, even a catastrophic failure effecting an
|
||||
admin of /another/ instance might make you wish you had a backup of
|
||||
your follows list[fn::If a remote admin decides to de-federate your
|
||||
instance it will sever all your connections with people on that
|
||||
instance.].
|
||||
|
||||
Remembering to make a backup manually is already difficult
|
||||
enough[fn::My last manual backup was over a year old.], but then you
|
||||
also need to remember *how* to make that backup...
|
||||
* Manual Backups
|
||||
Got to "App Settings". You do not know where "App Settings" are? I
|
||||
know the feeling... They are behind the three cogs on the top
|
||||
left. Navigate to "Import and export". Click "Data export". Click the
|
||||
"CSV" link. Click the other "CSV" link. Click the other other "CSV"
|
||||
link. You have now downloaded your "follows", "muted" and "blocked"
|
||||
accounts lists.
|
||||
* Automatic Backups
|
||||
I have recently discovered [[https://toot.bezdomni.net/introduction.html][toot - Mastodon CLI client]] which can output
|
||||
a list of your follows and people following you. I have [[https://github.com/ihabunek/toot/pull/390][contributed
|
||||
code for "muted" and "blocked" commands]] which got accepted and
|
||||
released in version 0.38.
|
||||
|
||||
We first need to authenticate to our instance by running ~toot
|
||||
login~. This is only needed once. For more information [[https://toot.bezdomni.net/usage.html][see the
|
||||
official documentation]].
|
||||
|
||||
I am running the following script from cron once a day:
|
||||
#+begin_src shell
|
||||
#! /bin/ksh
|
||||
ACCOUNT=@florian@bsd.network
|
||||
INSTANCE=bsd.network
|
||||
BACKUPDIR=/home/mastodonbackup/backup
|
||||
|
||||
set -e
|
||||
set -o pipefail
|
||||
|
||||
exit_if_nonzero_or_stderr() {
|
||||
(
|
||||
set -o pipefail
|
||||
{ "$@" 1>&3 ;} 2>&1 | {
|
||||
if IFS= read -r line; then
|
||||
printf "%s\n" "$line"
|
||||
cat
|
||||
exit 1
|
||||
fi
|
||||
} >&2
|
||||
) 3>&1
|
||||
}
|
||||
|
||||
exit_if_nonzero_or_stderr /usr/local/bin/toot muted | \
|
||||
awk '$2 !~ /^@.*@/ {print $2 "@'${INSTANCE}'"; next} {print $2}' \
|
||||
> ${BACKUPDIR}/muted.new
|
||||
|
||||
mv ${BACKUPDIR}/muted{.new,}
|
||||
|
||||
exit_if_nonzero_or_stderr /usr/local/bin/toot blocked | \
|
||||
awk '$2 !~ /^@.*@/ {print $2 "@'${INSTANCE}'"; next} {print $2}' \
|
||||
> ${BACKUPDIR}/blocked.new
|
||||
mv ${BACKUPDIR}/blocked{.new,}
|
||||
|
||||
exit_if_nonzero_or_stderr /usr/local/bin/toot following ${ACCOUNT} | \
|
||||
awk '$2 !~ /^@.*@/ {print $2 "@'${INSTANCE}'"; next} {print $2}' \
|
||||
> ${BACKUPDIR}/following.new
|
||||
mv ${BACKUPDIR}/following{.new,}
|
||||
|
||||
exit_if_nonzero_or_stderr /usr/local/bin/toot followers ${ACCOUNT} | \
|
||||
awk '$2 !~ /^@.*@/ {print $2 "@'${INSTANCE}'"; next} {print $2}' \
|
||||
> ${BACKUPDIR}/followers.new
|
||||
mv ${BACKUPDIR}/followers{.new,}
|
||||
#+end_src
|
||||
|
||||
~toot~ outputs accounts from the local instance without
|
||||
/@instance/. ~awk(1)~ checks for the existence of two /@/ characters
|
||||
and if they are not present outputs /@instance@/ after the account
|
||||
name.
|
||||
|
||||
Unfortunately ~toot~ always exits with 0, even if there is an error so
|
||||
we need to check if there is some output on =stderr= so that we do not
|
||||
replace our backup with empty files. For that I copy-pasted
|
||||
=exit_if_nonzero_or_stderr= from [[https://stackoverflow.com/a/61468182][stackoverflow]] like a pro.
|
||||
|
||||
My crontab entry looks like this:
|
||||
#+begin_src crontab
|
||||
~ 9 * * * -sn su -s /bin/sh mastodonbackup -c /home/mastodonbackup/backup.sh
|
||||
#+end_src
|
||||
=-sn= ensures that the script is only run once and that no mail is
|
||||
sent after a successful run. See [[https://man.openbsd.org/crontab.5][crontab(5)]] for details.
|
||||
|
||||
* Epilogue
|
||||
I have automated backing up the most important lists from my mastodon
|
||||
account. My mastodon account is running on somebody else's
|
||||
computer. The backup data is written as text files on a system that I
|
||||
control. That system is hooked up to my standard backup solution to
|
||||
have daily, multiple-redundant backups.
|
||||
|
||||
I expire my toots after 90 days using [[https://ephemetoot.hugh.run/][ephemetoot]]. I have configured it
|
||||
to save the toots and media before deleting them from my account.
|
||||
@ -28,12 +28,9 @@ I'll just leave this here...
|
||||
|
||||
- name: reboot and wait for host to return
|
||||
block:
|
||||
- name: reboot
|
||||
- name: schedule reboot in 1 minute
|
||||
ansible.builtin.command:
|
||||
cmd: reboot
|
||||
ignore_errors: yes
|
||||
async: 3600
|
||||
poll: 0
|
||||
cmd: 'shutdown -r +1'
|
||||
- name: wait for ssh to go away
|
||||
ansible.builtin.wait_for:
|
||||
host: '{{ (ansible_ssh_host|default(ansible_host))|default(inventory_hostname) }}'
|
||||
|
||||
@ -1,64 +0,0 @@
|
||||
#+TITLE: new-sshagent-work
|
||||
#+DATE: 2024-07-16
|
||||
* Prologue
|
||||
So I got a YubiKey 5C Nano handed to me.
|
||||
Things kinda got out of hand.
|
||||
|
||||
* Setup
|
||||
The key is so small that it will just stay in one of my laptop's USB-C
|
||||
ports.
|
||||
I want to use the key for =ssh= authentication.
|
||||
Step one is to disable OTP because I do not want to spill random
|
||||
strings into my tty every time I touch it by accident:
|
||||
#+begin_src shell
|
||||
rcctl -f start pcscd
|
||||
ykman config usb -d OTP
|
||||
rcctl -f stop pcscd
|
||||
#+end_src
|
||||
Next we create an non-resident =ed25519-sk= key.
|
||||
That is the key type used for FIDO keys:
|
||||
#+begin_src shell
|
||||
ssh-keygen -t ed25519-sk
|
||||
#+end_src
|
||||
FIDO keys consist of two parts: a key-handle and a private key.
|
||||
The private key stays on the FIDO token and is combined with the
|
||||
key-handle for signing operations.
|
||||
For a non-resident key the key-handle is stored on disk in the
|
||||
private-key file and is password protected.
|
||||
|
||||
=/etc/X11/xenodm/Xsession= starts [[http://man.openbsd.org/ssh-agent][ssh-agent(1)]] and calls [[http://man.openbsd.org/ssh-add][ssh-add(1)]] to
|
||||
add the standard identities to the ssh-agent.
|
||||
|
||||
I have to touch the token on every use of the =ed25519-sk= key.
|
||||
|
||||
Assuming the FIDO token works correctly, nobody can steal my private
|
||||
key remotely.
|
||||
|
||||
Theo de Raadt (deraadt@) pointed out a problem with the key at rest,
|
||||
when I suspend my laptop I want to remove the key from the agent and
|
||||
re-add it at first use on resume.
|
||||
We were puzzling around with this for a bit at =c2k24= but did not
|
||||
make too much progress.
|
||||
|
||||
* A Triumph in Modern Igoring
|
||||
Back home I remembered an option that I had to use on my macOS work
|
||||
laptop to make the ssh-agent work correctly: =AddKeysToAgent=
|
||||
|
||||
Having this in =/etc/apm/suspend= removes all keys from my agent on
|
||||
suspend:
|
||||
#+begin_src shell
|
||||
#!/bin/sh
|
||||
|
||||
for a in $(find /tmp -user florian -path '/tmp/ssh-*' -name 'agent.*'); do
|
||||
su florian -c "SSH_AUTH_SOCK=$a ssh-add -Dq"
|
||||
done
|
||||
#+end_src
|
||||
|
||||
Adding =AddKeysToAgent yes= as first line to =~/.ssh/config= then
|
||||
prompts me for the password of the key on first use and adds it to the
|
||||
ssh-agent again.
|
||||
|
||||
* Epilogue
|
||||
This works, but it should really work out of the box per default.
|
||||
This being OpenBSD, you can rest assured that we are working on it.
|
||||
Stay tuned...
|
||||
@ -8,13 +8,13 @@ upstream.
|
||||
git clone from the upstream repository on [[https://github.com/NLnetLabs/nsd][github]].
|
||||
#+begin_src shell
|
||||
git pull
|
||||
git diff NSD_4_6_1_REL..NSD_4_7_0_REL . ':!.cirrus.yml' ':!tpkg/*' \
|
||||
':!contrib/*' ':!compat/' ':!.gitignore' ':!doc/README.svn' \
|
||||
':!doc/CREDITS' \
|
||||
':!.buildkite/*' ':!makedist.sh' > ~/nsd_4.7.0_upstream.diff
|
||||
git diff NSD_4_3_6_REL..NSD_4_3_7_REL . ':!.cirrus.yml' ':!tpkg/*' \
|
||||
':!contrib/*' ':!compat/' ':!README.md' ':!.gitignore' \
|
||||
':!.buildkite/*' ':!makedist.sh' > ~/nsd_4.3.7_upstream.diff
|
||||
cd /usr/src/usr.sbin/nsd
|
||||
patch -Ep0 < ~/nsd_4.7.0_upstream.diff
|
||||
autoconf-2.71
|
||||
patch -Ep0 < ~/nsd_4.3.7_upstream.diff
|
||||
autoheader-2.69
|
||||
autoconf-2.69
|
||||
make -f Makefile.bsd-wrapper obj
|
||||
make -f Makefile.bsd-wrapper clean
|
||||
make -f Makefile.bsd-wrapper -j4
|
||||
@ -22,7 +22,7 @@ git clone from the upstream repository on [[https://github.com/NLnetLabs/nsd][gi
|
||||
We also need to update the version numbers in the man pages. For that
|
||||
we download the release tar ball and generate a diff:
|
||||
#+begin_src shell
|
||||
diff -bru . /home/florian/nsd-4.7.0/ | fgrep -v Only > sync.diff
|
||||
diff -ru . /home/florian/nsd-4.3.7/ | fgrep -v Only > sync.diff
|
||||
#+end_src
|
||||
The diff then needs to be partially applied, some changes are
|
||||
intentional.
|
||||
|
||||
203
openttd-srnw.org
203
openttd-srnw.org
@ -1,203 +0,0 @@
|
||||
#+TITLE: OpenTTD Self Regulating Networks
|
||||
#+DATE: 2024-01-13
|
||||
[[file:openttd-srnw/srnw_head.png]]
|
||||
* Prologue
|
||||
Every other year or so I decide to play a game or ten of
|
||||
[[https://openttd.org/][OpenTTD]]. Unfortunately I don't remember most of the things I figured
|
||||
out in the past so I have to start from first principles.
|
||||
In this case, first principles means reading the [[https://wiki.openttdcoop.org/Main_Page][openttdcoop Wiki]]
|
||||
[fn::which is quite dense] and watching a bunch of [[https://www.youtube.com/@LugnutsK][LugnutsK]]
|
||||
videos. For the topic at hand I would suggest
|
||||
[[https://www.youtube.com/watch?v=ZVtszocBmiY][Advanced OpenTTD - Self-Regulating Networks]].
|
||||
|
||||
* Self Regulating Passenger Network
|
||||
The idea of a self regulating passenger network is to grow a city and
|
||||
transport as many passengers out of the city as possible with the
|
||||
least amount of train orders.
|
||||
|
||||
openttdcoop calls it [[http://wiki.openttdcoop.org/Gametype:ICE_SBahn][Gametype:ICE SBahn]] / [[http://wiki.openttdcoop.org/Self-regulating_SBahn][Self-regulating SBahn]].
|
||||
|
||||
The idea is to have (SBahn) stations spread throughout the city that
|
||||
collect passengers. Once a train is fully loaded with passengers, it
|
||||
drives outside of the city to a transfer (ICE) station and transfers
|
||||
passengers to a waiting train that then transports the passengers to a
|
||||
far away city to make money.
|
||||
|
||||
We do not want to deal with each station individually but run the
|
||||
pick-up trains in a loop that automatically visits a station with
|
||||
a full load of passengers available.
|
||||
|
||||
To make all of this work we have:
|
||||
1. Inner-city SBahn stations
|
||||
2. Outside-city waiting loop for the pick-up trains
|
||||
3. Outside-city exit track to the transfer / ICE station
|
||||
4. Outside-city transfer / ICE station
|
||||
5. Injection mechanism from the ICE station to the waiting loop.
|
||||
|
||||
[[file:openttd-srnw/overview.png][file:openttd-srnw/overview_small.png]]
|
||||
|
||||
The pick-up trains have two orders:
|
||||
1. Go to transfer / ICE station and transfer passengers.
|
||||
2. Go to inject way-point.
|
||||
|
||||
The inject way-point leads to the waiting loop. From there the only
|
||||
way to reach the transfer station is through any of the SBahn
|
||||
stations. We have to arrange the SBahn stations in a way that they are
|
||||
only accessible when there is a full load of passengers available.
|
||||
|
||||
* City Layout
|
||||
For maximum growth and ease of putting train stations into the city we
|
||||
are using a 3x3 road grid. A spiral would be even better for city
|
||||
growth but it is annoying to put train stations into a spiral without
|
||||
disrupting the roads too much and creating dead-ends. Dead-ends are
|
||||
very bad for town growth.
|
||||
|
||||
We need to find a balance between station size[fn::One constraint here
|
||||
is how much space the station takes away from the city to build houses.],
|
||||
city coverage, and needed capacity at the transfer station. Using spread
|
||||
stations covering a 5x5 block[fn::A block is a 3x3 road grid.] and
|
||||
having six of those works fairly well. To be able to fit tunnels in we
|
||||
put two in a line and have three lines[fn::Three in a line would also
|
||||
leave enough space for tunnels. But then the waiting loop might become
|
||||
too big and it takes to long to try all the SBahn stations. This is
|
||||
especially a problem once the city is big and produces lots of
|
||||
passengers.]. This results in a city grid of 13x8 blocks.
|
||||
|
||||
[[file:openttd-srnw/city-layout.png][file:openttd-srnw/city-layout_small.png]]
|
||||
|
||||
* Station Construction
|
||||
We are building a spread station in a 3x3 block by putting a city
|
||||
station in each of the corners of the block. Holding =ctrl= while
|
||||
placing the building opens the "Join station" menu and we select an
|
||||
existing station.
|
||||
|
||||
[[file:openttd-srnw/spread-station.png][file:openttd-srnw/spread-station_small.png]]
|
||||
|
||||
The coverage of the station is not perfect but some of these parts
|
||||
will later be filled by the tracks of the train station. Note that the
|
||||
station also covers a block outside of the road grid, so each spread
|
||||
station covers a 5x5 block. This is important when putting in adjacent
|
||||
stations to not have them overlap.
|
||||
|
||||
[[file:openttd-srnw/station-coverage.png][file:openttd-srnw/station-coverage_small.png]]
|
||||
|
||||
For the rail network we dig two trenches outside of the city. Tunnels
|
||||
to those will connect the station to the waiting loop on one side of
|
||||
the city and the exit to the transfer station on the other side of the
|
||||
city.
|
||||
|
||||
We then remove some roads and the crossings[fn::This ensures that we
|
||||
can dig a complete hole, we will put the roads back in later.] on one
|
||||
side of the block and dig a 7x2 hole for the train tracks.
|
||||
|
||||
[[file:openttd-srnw/station-digging.png][file:openttd-srnw/station-digging_small.png]]
|
||||
|
||||
Next: Tunnels, tracks, signal, a way-point, and a train depot.
|
||||
|
||||
[[file:openttd-srnw/station-tunnels-and-tracks.png][file:openttd-srnw/station-tunnels-and-tracks_small.png]]
|
||||
|
||||
We then place a two platform, length five station, making sure we join
|
||||
it with the existing spread station.
|
||||
|
||||
[[file:openttd-srnw/station-placing-tracks.png][file:openttd-srnw/station-placing-tracks_small.png]]
|
||||
|
||||
The finished station looks like this:
|
||||
|
||||
[[file:openttd-srnw/station-anotation.png][file:openttd-srnw/station-anotation_small.png]]
|
||||
|
||||
We are going to put a dummy train onto the dummy track to pick up
|
||||
passengers and making sure a full load is available when a pick-up
|
||||
train arrives. We do not place exit signals at the station. When a
|
||||
dummy train is inside the station it blocks the whole station due to
|
||||
the crossing tracks at the end of the station. The way-point is going
|
||||
to be used to open the station for the pick-up trains.
|
||||
|
||||
* Train Orders
|
||||
** Dummy Trains
|
||||
We create a dummy train per station. These collect passengers and
|
||||
block the station for the pick-up trains. Once a full load of
|
||||
passengers has been collected the dummy train unloads them and
|
||||
unblocks the station. Now a full load of passengers is available for
|
||||
the pick-up trains.
|
||||
|
||||
We make the trains length four so that it can drive around in the
|
||||
station itself, which has length five. We are using "near end" and "far
|
||||
end" orders:
|
||||
|
||||
1. Go to near end of the station and full load
|
||||
2. Go to far end of the station and transfer
|
||||
3. Go to the way-point after the station.
|
||||
|
||||
Step three is what unblocks the station. Once the train has left the
|
||||
depot we have to remove the track right in front of it so that the
|
||||
dummy train cannot re-enter the depot. Alternatively we could put the
|
||||
depot right after the way-point and delete it after the train left it.
|
||||
|
||||
[[file:openttd-srnw/dummy-train-orders.png][file:openttd-srnw/dummy-train-orders_small.png]]
|
||||
|
||||
** Pick-up Trains
|
||||
The pick-up trains have two orders:
|
||||
1. Go via INJECT way-point
|
||||
2. Go to transfer station and transfer.
|
||||
|
||||
Unfortunately OpenTTD will create implicit orders every time the train
|
||||
passes through one of the SBahn stations. To prevent this we add a
|
||||
"Jump to order 1" instruction and then fill up the orders with an
|
||||
unreachable way-point. A train can only have 255 orders. Once the
|
||||
order list is full no implicit orders can be created.
|
||||
|
||||
[[file:openttd-srnw/pick-up-train-orders.png][file:openttd-srnw/pick-up-train-orders_small.png]]
|
||||
|
||||
* Self Regulating Network Signalling
|
||||
The interesting bit happens once the trains passed the INJECT
|
||||
way-point. They are now inside of the waiting loop and try to get
|
||||
to the transfer station. This is only possible through one of the
|
||||
SBahn stations.
|
||||
|
||||
When a train reaches the tunnel entrance to an SBahn station it has
|
||||
two choices, either continue in the waiting loop or enter the station.
|
||||
|
||||
If a train is already in the waiting bay of that station we want our
|
||||
train in the waiting loop to continue and try the next station. We put
|
||||
two-way block signals at the tunnel entrance which turns red when the
|
||||
waiting bay is occupied by a train. OpenTTD's path finder treats a red
|
||||
two-way signal as a dead-end[fn::This is only true if
|
||||
=yapf.rail_firstred_twoway_eol= is =True=. This is the default in
|
||||
OpenTTD 13.]. It has an infinite penalty and any other
|
||||
path will be better. Our train continues. This is the first block
|
||||
signal in the picture below.
|
||||
|
||||
At our second SBahn entrance the waiting bay is free and the block
|
||||
signal shows green, so a train should enter the waiting bay towards
|
||||
the station. However, the path finder sees that the path goes through
|
||||
a station. Going through a station incurs a penalty for that path and
|
||||
it might so happen that the path continuing in the waiting loop is
|
||||
considered better, our train would not enter the waiting bay.
|
||||
|
||||
We need to add a penalty to the waiting loop path to make the path
|
||||
through the station look better. We do this by adding a few reversed
|
||||
path signals after the choice.
|
||||
|
||||
[[file:openttd-srnw/srnw-signalling.png][file:openttd-srnw/srnw-signalling_small.png]]
|
||||
|
||||
* Bonus: Path Based Signal Priority Merge
|
||||
The [[https://wiki.openttdcoop.org/Priorities#PBS_prio][Priority - #openttdcoop wiki]] article shows how to construct a
|
||||
priority merge using path based signals. Since I will likely forget it
|
||||
and the traditional priorities are slightly more difficult to build I
|
||||
put a note here.
|
||||
|
||||
The way this works is that the tracks coloured in red form one big
|
||||
signal block that reaches all the way to the merger at the front. Once
|
||||
the mainline train enters this block the entry signal for the merging
|
||||
train turns red and the merging train has to wait for the mainline
|
||||
train to clear the block.
|
||||
|
||||
[[file:openttd-srnw/pbs-prio.png][file:openttd-srnw/pbs-prio_small.png]]
|
||||
|
||||
* Epilogue
|
||||
After many iterations I came up with this city layout and station
|
||||
design. It is probably not the best one and different trade-offs can
|
||||
be made. The most fun in OpenTTD is to figure these things out and not
|
||||
just copying a design from somewhere. When I come back to playing
|
||||
OpenTTD in a few years I can start from this and try to improve on
|
||||
this and do not have to re-discover all this the hard way.
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 5.2 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 673 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.2 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 610 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 6.1 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 554 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 4.7 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 709 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 4.9 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 681 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.6 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 450 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 6.6 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 634 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.6 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.1 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 929 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.8 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 849 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.8 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 778 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 2.5 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 575 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 1.9 MiB |
Binary file not shown.
|
Before Width: | Height: | Size: 688 KiB |
109
publish.el
109
publish.el
@ -13,6 +13,7 @@
|
||||
|
||||
(unless (package-installed-p 'use-package)
|
||||
(package-install 'use-package))
|
||||
|
||||
(require 'use-package)
|
||||
(setq use-package-always-ensure t)
|
||||
|
||||
@ -28,21 +29,68 @@
|
||||
:pin org)
|
||||
|
||||
(use-package htmlize)
|
||||
(use-package ox-rss)
|
||||
|
||||
(require 'ox-publish)
|
||||
(require 'ox-html)
|
||||
(require 'htmlize)
|
||||
|
||||
(defvar fo-base-directory "/var/www/htdocs/sha256.net/"
|
||||
"Where the org files live.")
|
||||
|
||||
(defvar fo-url "https://openbsd-build.home.narrans.de/sha256.net/"
|
||||
"URL where the site will be published.")
|
||||
|
||||
(defvar fo-title "sha256.net - Florian Obser"
|
||||
"Title of the site.")
|
||||
|
||||
|
||||
(setq org-html-doctype "html5")
|
||||
(setq org-html-htmlize-output-type 'css)
|
||||
(setq org-export-time-stamp-file nil)
|
||||
(setq org-html-validation-link nil)
|
||||
(setq org-html-head-include-default-style nil)
|
||||
(setq org-publish-project-alist
|
||||
'(("tlakh"
|
||||
:base-directory "/var/www/sha256.net/"
|
||||
:publishing-function org-html-publish-to-html
|
||||
:publishing-directory "/var/www/sha256.net/"
|
||||
|
||||
(defun fo/format-rss-feed-entry (entry style project)
|
||||
"Format ENTRY for the RSS feed.
|
||||
ENTRY is a file name. STYLE is either 'list' or 'tree'.
|
||||
PROJECT is the current project."
|
||||
(cond ((not (directory-name-p entry))
|
||||
(let* ((file (org-publish--expand-file-name entry project))
|
||||
(title (org-publish-find-title entry project))
|
||||
(date (format-time-string "%Y-%m-%d" (org-publish-find-date entry project)))
|
||||
(link (concat (file-name-sans-extension entry) ".html")))
|
||||
(with-temp-buffer
|
||||
(insert (format "* [[file:%s][%s]]\n" file title))
|
||||
(org-set-property "RSS_PERMALINK" link)
|
||||
(org-set-property "PUBDATE" date)
|
||||
(insert-file-contents file)
|
||||
(buffer-string))))
|
||||
((eq style 'tree)
|
||||
;; Return only last subdir.
|
||||
(file-name-nondirectory (directory-file-name entry)))
|
||||
(t entry)))
|
||||
|
||||
(defun fo/format-rss-feed (title list)
|
||||
"Generate RSS feed, as a string.
|
||||
TITLE is the title of the RSS feed. LIST is an internal
|
||||
representation for the files to include, as returned by
|
||||
`org-list-to-lisp'. PROJECT is the current project."
|
||||
(concat "#+TITLE: " title "\n\n"
|
||||
(org-list-to-subtree list 1 '(:icount "" :istart ""))))
|
||||
|
||||
(defun fo/org-rss-publish-to-rss (plist filename pub-dir)
|
||||
"Publish RSS with PLIST, only when FILENAME is 'rss.org'.
|
||||
PUB-DIR is when the output will be placed."
|
||||
(if (equal "rss.org" (file-name-nondirectory filename))
|
||||
(org-rss-publish-to-rss plist filename pub-dir)))
|
||||
|
||||
(defvar fo--publish-project-alist
|
||||
(list
|
||||
(list "sha256.net"
|
||||
:base-directory fo-base-directory
|
||||
:publishing-function 'org-html-publish-to-html
|
||||
:publishing-directory fo-base-directory
|
||||
:exclude (regexp-opt '("rss.org"))
|
||||
:section-numbers nil
|
||||
:with-author nil
|
||||
:with-email nil
|
||||
@ -52,11 +100,54 @@
|
||||
:time-stamp-file: nil
|
||||
:html-postamble "<p class=\"date\">Published: %d</p>
|
||||
<hr /><footer><nav><a href=\"/\">< Home</a></nav>
|
||||
Copyright © 2014 - 2024 Florian Obser. All rights reserved.
|
||||
Copyright © 2014 - 2023 Florian Obser. All rights reserved.
|
||||
</footer>"
|
||||
:html-head-include-default-style: nil
|
||||
:html-head "<link rel=\"stylesheet\" href=\"simple.min.css\" type=\"text/css\"/>
|
||||
<link rel=\"stylesheet\" href=\"htmlize.min.css\" type=\"text/css\"/>
|
||||
<link rel=\"stylesheet\" href=\"custom.css\" type=\"text/css\"/>")))
|
||||
<link rel=\"stylesheet\" href=\"custom.css\" type=\"text/css\"/>")
|
||||
(list "sha256.net-rss"
|
||||
:base-directory fo-base-directory
|
||||
:publishing-directory fo-base-directory
|
||||
:base-extension "org"
|
||||
:recursive nil
|
||||
:exclude (regexp-opt '("rss.org" "index.org" "404.org" "50x.org"))
|
||||
:publishing-function 'fo/org-rss-publish-to-rss
|
||||
:rss-extension "xml"
|
||||
:html-link-home fo-url
|
||||
:html-link-use-abs-url t
|
||||
:html-link-org-files-as-html t
|
||||
:auto-sitemap t
|
||||
:sitemap-filename "rss.org"
|
||||
:sitemap-title fo-title
|
||||
:sitemap-style 'list
|
||||
:sitemap-sort-files 'anti-chronologically
|
||||
:sitemap-function 'fo/format-rss-feed
|
||||
:sitemap-format-entry 'fo/format-rss-feed-entry
|
||||
:author "Florian Obser"
|
||||
:email "")
|
||||
(list "site"
|
||||
:components '("sha256.net"))))
|
||||
|
||||
(org-publish "tlakh")
|
||||
(defun fo-publish-all ()
|
||||
"Publish the blog to HTML."
|
||||
(interactive)
|
||||
(let ((org-publish-project-alist fo--publish-project-alist)
|
||||
(org-publish-timestamp-directory "./.timestamps/")
|
||||
(org-export-with-section-numbers nil)
|
||||
(org-export-with-smart-quotes t)
|
||||
(org-export-with-toc nil)
|
||||
(org-export-with-sub-superscripts '{})
|
||||
(org-html-divs '((preamble "header" "top")
|
||||
(content "main" "content")
|
||||
(postamble "footer" "postamble")))
|
||||
(org-html-container-element "section")
|
||||
(org-html-metadata-timestamp-format "%Y-%m-%d")
|
||||
(org-html-checkbox-type 'html)
|
||||
(org-html-html5-fancy t)
|
||||
(org-html-validation-link nil)
|
||||
(org-html-doctype "html5")
|
||||
(org-html-htmlize-output-type 'css))
|
||||
(org-publish-all)))
|
||||
|
||||
(fo-publish-all)
|
||||
|
||||
Loading…
Reference in New Issue
Block a user