Random forest

TOTP is easy

sarg@sarg.org.ru (Sergey Trofimov) — Fri, 01 Aug 2025 00:00:00 +0200

One-time passwords are everywhere. Let’s learn how to generate one yourself. The algorithm is simple: calculate an HMAC over an 8-byte counter and truncate it for display. Truncation involves using the last four bits of the hash to determine an offset, from which four bytes are extracted. The most significant bit of these four bytes is masked out (to avoid negative values), and the remaining 31 bits are treated as a big-endian integer. Finally, the decimal representation of this integer is truncated to the required length.

The counter must be synchronised between the generator and validator. For HOTP it is the sequential index of the password; for TOTP it is derived from current time.

The algorithm translates to this Python program:

import hmac, base64, hashlib, time

def hotp(secret: bytes, counter: int, length: int = 6):
    h = hmac.new(
        base64.b32decode(secret),
        counter.to_bytes(8, "big"),
        hashlib.sha1,
    ).digest()
    o = h[-1] & 0xF
    res = int.from_bytes(h[o : o + 4], "big") & 0x7FFFFFFF
    return str(res % 10**length).zfill(length)

def totp(secret: bytes, period: int = 30):
    return hotp(secret, int(time.time()) // period)

print(f"HOTP: {hotp('SDACNQ37BVMBQ3Q5', 1)}")
print(f"TOTP: {totp('SDACNQ37BVMBQ3Q5')}")

HOTP: 795526
TOTP: 467487

Literate Guix configuration

sarg@sarg.org.ru (Sergey Trofimov) — Sun, 22 Jun 2025 00:00:00 +0200

Intro

This file is a literate configuration of a Guix and Emacs system. Tangling it (C-c C-v t) produces home.scm , which could be then installed as usual.

Using this template you’ll be able to structure a home-environment definition in a way that is convenient to you. Besides regular noweb features, a new source code block language is implemented, allowing templated configuration files (see example below). Emacs configuration is included in the home-environment as a package (see the definition), which is built by tangling this org file.

A couple settings to ease the use in emacs:

set tangle parameters
eval and wrap cfg blocks automatically

#+PROPERTY: header-args :noweb-prefix no :noweb yes :eval no
#+PROPERTY: header-args:elisp :tangle config.el
#+PROPERTY: header-args:cfg :eval yes :cache yes :wrap "src text :noweb-ref guix-file"

# Local Variables:
# eval: (add-hook 'after-save-hook (lambda () (org-babel-tangle nil nil "scheme")) 0 t)
# eval: (add-hook 'org-babel-pre-tangle-hook #'org-babel-execute-buffer 0 t)
# End:

Template

Main template which tangles to home.scm.

#+begin_src scheme :tangle home.scm

(use-modules
 (gnu)
 (gnu services)
 (gnu home services)
 (gnu home services desktop)
 (guix gexp)
 (guix git-download)
 (guix build-system copy)
 (guix packages)
 (ice-9 receive))

(define (pkg x)
  (if (string? x)
      (receive (package out)
          (specification->package+output x)
        (if (string= "out" out) package
            (list package out)))
      x))

(define (chmod-computed-file f p)
  (computed-file
   (computed-file-name f)
   #~(begin #$(computed-file-gexp f) (chmod #$output #$p))
   #:options (computed-file-options f)))

(home-environment
 (services
  (cons*
   (simple-service
    'files home-files-service-type
    `((".guile" ,%default-dotguile)
      (".config/guix/channels.scm"
       ,(plain-file
         "channels.scm"
         (simple-format #f "~s" '(list <<guix-channel>>))))
      <<guix-file>>))
   <<guix-srv>>
   %base-home-services))

   (packages (map pkg (list <<guix-pkg>>))))

Channels

guix

#+begin_src scheme :noweb-ref guix-channel

(channel
 (name 'guix)
 (url "https://git.guix.gnu.org/guix.git")
 (branch "master")
 (introduction
  (make-channel-introduction
   "9edb3f66fd807b096b48283debdcddccfea34bad"
   (openpgp-fingerprint
    "BBB0 2DDF 2CEA F6A8 0D1D  E643 A2A0 6DF2 A33A 54FA"))))

Example content

package

#+begin_src scheme :noweb-ref guix-pkg

"emacs-next" "emacs-org"

service

#+begin_src scheme :noweb-ref guix-srv

(service home-dbus-service-type)

simple files

#+begin_src scheme :noweb-ref guix-file

("plain" ,(plain-file "file" "hello, world"))
("local" ,(local-file "." "file" #:recursive? #t))

templated files

A cfg source block with :dest parameter will produce a result, which when wrapped using :wrap "src scheme :noweb-ref guix-file", would be weaved automatically in the main template.

#+begin_src cfg :dest "tmpl"

this template gets converted to mixed-text-file
ANT unicode characters mark scheme code
some-program = 🐜 (pkg "msmtp") 🐜/bin/msmtp

("tmpl"
,(mixed-text-file
  "cfg-file"
"this template gets converted to mixed-text-file\n"
"ANT unicode characters mark scheme code\n"
"some-program = " (pkg "msmtp") "/bin/msmtp"))

Emacs config

ob-cfg.el

A helper script that renders cfg source code blocks.

;; -*- lexical-binding: t; -*-
(require 'rx)
(defun org-babel-execute:cfg (body params)
  "Wrap BODY in a (mixed-text-file). If PARAMS contains :file, use its
contents instead of BODY. If PARAMS contain :chmod, additionally wrap
with (chmod-computed-file)."
  (let-alist params
    (when .:file
      (with-temp-buffer
        (insert-file-contents-literally .:file nil)
        (setf body (buffer-string))))

    (concat
     (if .:dest (format "(%S\n," .:dest) "")
     (if .:chmod "(chmod-computed-file " "")
     "(mixed-text-file\n  \"cfg-file\"\n"
     ;; split string into parts 'string-part🐜scheme-part🐜'
     ;; - output each line of 'string-part' quoted.
     ;; - output 'scheme-part' as is
     ;; - note: add an empty 'scheme-part' at the end of input string
     ;;   so that the last 'string-part' is matched
     (replace-regexp-in-string
      (rx (minimal-match
           (group (zero-or-more anything))
           ?🐜 (group (zero-or-more anything)) ?🐜))
      (lambda (e)
        (let ((string-part (match-string 1 e))
              (scheme-part (match-string 2 e)))
          (concat
           (if (string-empty-p string-part) ""
             (string-join
              (mapcar
               (lambda (l) (prin1-to-string l nil '((escape-newlines . t))))
               (string-lines string-part nil t))
              "\n"))
           scheme-part)))
      (concat body "🐜🐜") t t)
     ")"
     (if .:chmod (format " #o%o)" (org-babel-interpret-file-mode .:chmod)) "")
     (if .:dest ")" ""))))

config package

#+begin_src scheme :noweb-ref emacs-config :noweb yes

(package
  (name "emacs-config")
  (version "0")
  (source (local-file "." "src" #:recursive? #t))
  ;; you really should put the files in git and enable filtering
  ;; #:select? (git-predicate ".")
  (build-system copy-build-system)
  (arguments
    (list
     #:install-plan ''(("." "" #:include (".el")))
     #:phases
     #~(modify-phases %standard-phases
         (add-before 'install 'tangle
           (lambda* (#:key inputs #:allow-other-keys)
             (substitute* "<<self()>>"
               (("\\(guix[/]pkg '([^ )]+)" _ pkg)
                (format #f "(concat ~s"
                        (or (assoc-ref inputs pkg)
                            (error (format #f "~a is required for emacs config but not present in inputs" pkg))))))

             (invoke "emacs" "-Q" "-l" "ob-cfg.el" "--batch" "--eval"
                     (simple-format #f "~s"
                                    '(progn
                                      (require 'ob-tangle)
                                      (setq org-confirm-babel-evaluate nil)
                                      (with-current-buffer
                                       (find-file-noselect "<<self()>>")
                                       (org-babel-tangle nil "early-init.el" "elisp")))))
             (delete-file "<<self()>>"))))))
  (inputs (map pkg '(<<emacs-dep>>)))
  (native-inputs (map pkg '("emacs-org" "emacs-next-minimal")))
  (description "emacs config")
  (home-page #f)
  (synopsis #f)
  (license #f))

#+begin_src scheme :noweb-ref guix-file

(".config/emacs" ,<<emacs-config>>)

config parts

#+begin_src text :noweb-ref emacs-dep

"msmtp"

#+begin_src elisp

(load (concat user-emacs-directory "ob-cfg.el"))
(require 'sendmail)
(setq sendmail-program (guix/pkg 'msmtp "/bin/msmtp"))

Pulumi + SOPS

sarg@sarg.org.ru (Sergey Trofimov) — Tue, 17 Jun 2025 00:00:00 +0200

Secrets can be provided to Pulumi programs via environment variables, configuration files, or Pulumi ESC. However, you might prefer to manage secrets externally for use in other programs. While I use pass for interactive secret management, I use SOPS for automation due to its text-based format and relative simplicity.

One method for passing secrets from SOPS is with the sops exec-env command, which adds secrets to the environment before executing a program. Pulumi can then read them with process.env["TOKEN"]. However, this might return null, causing the program to fail or, worse, producing unintended resource changes (e.g., setting a password to null 😱).

A superior approach leverages the structured (YAML) format in which SOPS stores secrets. Extract a schema from the secret file and convert it to TypeScript definitions:

npm install quicktype
sops -d --output-type json ../secrets.yaml | quicktype -t Secrets --lang ts > secrets.ts

For convenience, add this invocation to package.json:

{
  "scripts": {
     "secrets": "sops -d ..."
  }
}

In your Pulumi program, read secrets from SOPS and parse them into properly typed classes:

import { Convert } from './secrets';
import { execSync } from 'node:child_process';
import * as cloudflare from '@pulumi/cloudflare';

const secretsJson = execSync('sops -d --output-type json ../secrets.yaml');
const secrets = Convert.toSecrets(secretsJson.toString('utf8'));

const cloudflareProvider = new cloudflare.Provider('default', {
  apiToken: secrets.cloudflare.API_TOKEN,
});

Quicktype’s converter also validates the data, raising an error if a field is missing. This eliminates unexpected null values and improves the overall structure of your program.

Self-hosted DynDNS

sarg@sarg.org.ru (Sergey Trofimov) — Mon, 16 Jun 2025 00:00:00 +0200

Roll your own free dynamic DNS (DynDNS) service with these simple steps:

Host your DNS zone with a provider offering API access.
Deploy a DynDNS script to a FaaS platform.

For Cloudflare, follow these instructions:

Add your domain to Cloudflare.
Set up your development environment.
Deploy the DynDNS worker:
- Obtain the worker script and run npm i.
- Modify the domain in wrangler.jsonc.
- Add subdomain tokens to src/tokens.json.
- Execute wrangler deploy.

Verify operation with these commands:

# Sets the A/AAAA record to the IP address from the request.
curl https://dyndns.your.domain/your_token

# Explicitly sets the A/AAAA records.
curl https://dyndns.your.domain/your_token?ipv4=1.2.3.4&ipv6=::1

# Removes the A/AAAA records.
curl -XDELETE https://dyndns.your.domain/your_token

That’s all there is to it!

Cloudflare workers are cool

sarg@sarg.org.ru (Sergey Trofimov) — Sun, 15 Jun 2025 00:00:00 +0200

Writing a Telegram bot, my first option was to make it a Google Cloud Run function. The bot is stateless and in its core is a simple webhook responder. I quickly jotted down logic that only responded “Hello world” for every message. Even though it was about ten lines of Python, it took a couple minutes to deploy. I kind of expected that, knowing that Cloud Run functions are run in a docker container and building these could be slow due to many network requests. Cloud Run functions’ response time also increases for the first request, as it takes time to cold-start the container. Lastly, docker containers are bulky - a hundred megabytes to run 10 lines of code is off-putting.

Cloudflare Workers on the other hand boast support for 0ms cold starts. It turned out to be true, due to different approach they take. Instead of dedicated clusters to run containerized functions, Cloudflare puts functions on edge servers as V8 Isolates. The cold start is eliminated because V8 Engine is already running on the server. Deploying functions is also much faster, now that there is no need to build a container. Granted, a build step is still present if the program requires compilation (e.g. Typescript), but it is quick and happens on my laptop. Only the code itself needs to be uploaded (a couple kilobytes), as opposed to full runtime environment of a docker container.

sarg@thinkpad tgbot$ wrangler deploy --minify

 ⛅️ wrangler 4.20.0
───────────────────
Total Upload: 22.12 KiB / gzip: 7.50 KiB
Worker Startup Time: 14 ms
Uploaded tgbot (6.29 sec)
Deployed tgbot triggers (1.19 sec)

On-demand Wireguard VPN server on Hetzner

sarg@sarg.org.ru (Sergey Trofimov) — Fri, 13 Jun 2025 00:00:00 +0200

I need VPN so rarely that any subscription service becomes cost-prohibitive. Instead an on-demand VM instance could be provisioned in a cloud of choice. The cheapest VM on Hetzner costs just 0.007 €/hr. Let’s make it real.

Generate two sets of keys and a PSK

for i in A B; do
    KEY=$(wg genkey)
    PUB=$(echo $KEY | wg pubkey)
    echo -e "$i key: $KEY\n$i pub: $PUB"
done
echo "  psk: $(wg genpsk)"

A key: 8LfzsPSUfKRIF1nJRGsWquwQfYK1t4xtZ/tZd0Iw/nc=
A pub: W3FavAFRpwbwrf+C1swigClVe5HV9deUwqv+V2vGhgc=
B key: 8LXWaEVGhHwPLnXjMjFpZwHNUfj9lfaKWdKB1/gRg3U=
B pub: Tt9VwSHn7v4k7FrdPcgfv6dpIQ+kl/lj1DUvGGnKj18=
  psk: s34FX6nUoDOiPzj6ExWh92kgA03xk6YaQtf7qTRRO2M=

Put these in { placeholders } in files below.

Prepare a `cloud-init` provision script

wireguard module is only supported on ubuntu.

#cloud-config
wireguard:
  interfaces:
    - name: wg0
      config_path: /etc/wireguard/wg0.conf
      content: |
        [Interface]
        Address = 10.66.66.1/24,fd42:42:42::1/64
        ListenPort = 52817
        PrivateKey = { A key }
        PostUp = iptables -I INPUT -p udp --dport 52817 -j ACCEPT
        PostUp = iptables -I FORWARD -i eth0 -o wg0 -j ACCEPT
        PostUp = iptables -I FORWARD -i wg0 -j ACCEPT
        PostUp = iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
        PostUp = ip6tables -I FORWARD -i wg0 -j ACCEPT
        PostUp = ip6tables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
        PostDown = iptables -D INPUT -p udp --dport 52817 -j ACCEPT
        PostDown = iptables -D FORWARD -i eth0 -o wg0 -j ACCEPT
        PostDown = iptables -D FORWARD -i wg0 -j ACCEPT
        PostDown = iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
        PostDown = ip6tables -D FORWARD -i wg0 -j ACCEPT
        PostDown = ip6tables -t nat -D POSTROUTING -o eth0 -j MASQUERADE

        [Peer]
        PublicKey = { B pub }
        PresharedKey = { psk }
        AllowedIPs = 10.66.66.2/32,fd42:42:42::2/128

runcmd:
  - sysctl -p /etc/sysctl.d/nat.conf

write_files:
  - content: |
      net.ipv4.ip_forward = 1
      net.ipv6.conf.all.forwarding = 1
    owner: root:root
    path: /etc/sysctl.d/nat.conf
    permissions: '0644'

Deploy the VM

hcloud server create --name vpn \
    --type cax11 --image ubuntu-24.04 \
    --location hel1 \
    --ssh-key thinkpad \
    --start-after-create

Server 65612345 created
IPv4: 123.123.123.123

Prepare client config

[Interface]
Address = 10.66.66.2/32, fd42:42:42::2/128
PrivateKey = { B key }
ListenPort = 51820

[Peer]
PublicKey = { A pub }
PresharedKey = { psk }
AllowedIPs = 0.0.0.0/0, ::/0
Endpoint = { VM ip }

Start Wireguard

sudo wg-quick up ./wg0.conf

Cleanup after use

sudo wg-quick down ./wg0.conf
hcloud server delete vpn

Server vpn deleted

Trying Pulumi

sarg@sarg.org.ru (Sergey Trofimov) — Fri, 13 Jun 2025 00:00:00 +0200

I finally described my cloud infrastructure in a declarative configuration. It’s a modest setup: a DNS zone, this blog, and a couple of VM instances. Terraform, which I use at work, was my initial choice. I quickly imported the existing resources, but the resulting code didn’t spark joy. It felt too verbose and lacked expressiveness. HCL still relies on for_each and count for even basic flow control.

Looking for alternatives, I chose Pulumi because it supports configuration in general-purpose programming languages like Typescript, Go, or Java. Lisp dialects weren’t supported, so I settled on Typescript. Emacs with LSP support provided a great developer experience, offering contextual autocomplete and easy access to comprehensive documentation.

For secrets management, Pulumi offers ESC, which requires account registration and is dependent on their cloud. Since I don’t need RBAC or anything complex, I replaced it with SOPS. A small helper script (direnv/lib/sops.sh) injects the secrets into the shell environment when I cd into the pulumi directory.

Pulumi is a clear winner for me. Writing infrastructure code in a proper language is liberating, and I appreciate how pulumi stack neatly organizes my infrastructure view.

Current stack is dev:
    Managed by thinkpad
    Last updated: 59 minutes ago (2025-06-13 07:42:44.338221653 +0200 CEST)
    Pulumi version used: v3.175.0
Current stack resources (40):
    TYPE                                                            NAME
    pulumi:pulumi:Stack                                             terraform-dev
    ├─ components:index:Cloudflare                                  cloudflare
    │  ├─ cloudflare:index/zone:Zone                                sarg.org.ru
    │  │  ├─ cloudflare:index/dnsRecord:DnsRecord                   MX 2
    │  │  ├─ cloudflare:index/dnsRecord:DnsRecord                   CNAME mail
    │  │  ├─ cloudflare:index/dnsRecord:DnsRecord                   MX 3
    │  │  ├─ cloudflare:index/dnsRecord:DnsRecord                   TXT dmarc
    │  │  ├─ cloudflare:index/dnsRecord:DnsRecord                   MX 1
    │  │  ├─ cloudflare:index/dnsRecord:DnsRecord                   TXT spf
    │  │  ├─ cloudflare:index/dnsRecord:DnsRecord                   MX 0
    │  │  └─ cloudflare:index/dnsRecord:DnsRecord                   TXT dkim
    │  ├─ cloudflare:index/workersScript:WorkersScript              blog
    │  ├─ cloudflare:index/workersScript:WorkersScript              tgbot
    │  ├─ cloudflare:index/workersCustomDomain:WorkersCustomDomain  blog
    │  └─ cloudflare:index/workersCustomDomain:WorkersCustomDomain  tgbot
    ├─ components:index:Google                                      google
    │  ├─ gcp:organizations/project:Project                         default
    │  ├─ gcp:projects/service:Service                              services-apikeys.googleapis.com
    │  ├─ gcp:projects/service:Service                              services-serviceusage.googleapis.com
    │  ├─ gcp:projects/service:Service                              services-generativelanguage.googleapis.com
    │  ├─ gcp:projects/service:Service                              services-cloudresourcemanager.googleapis.com
    │  └─ gcp:projects/apiKey:ApiKey                                gptel
    ├─ components:index:Selectel                                    selectel
    │  ├─ openstack:networking/network:Network                      private
    │  ├─ openstack:compute/keypair:Keypair                         ssh_key
    │  ├─ openstack:networking/subnet:Subnet                        private
    │  ├─ openstack:networking/router:Router                        router
    │  ├─ openstack:networking/port:Port                            vpn
    │  └─ openstack:networking/routerInterface:RouterInterface      interface
    ├─ components:index:Hetzner                                     hetzner
    │  ├─ hcloud:index/sshKey:SshKey                                ssh_key
    │  └─ hcloud:index/primaryIp:PrimaryIp                          vpn_ipv6
    ├─ pulumi:providers:telegram                                    telegram
    ├─ components:index:Telegram                                    telegram
    │  ├─ telegram:index/botCommands:BotCommands                    diogenisbot
    │  └─ telegram:index/botWebhook:BotWebhook                      diogenisbot
    ├─ pulumi:providers:openstack                                   default_5_1_0
    ├─ pulumi:providers:hcloud                                      default_1_23_0
    ├─ pulumi:providers:cloudflare                                  default_6_2_1
    └─ pulumi:providers:gcp                                         default_8_33_0

Current stack outputs (0):
    No output values currently in this stack

Use `pulumi stack select` to change stack; `pulumi stack ls` lists known ones

Following git changelogs in elfeed

sarg@sarg.org.ru (Sergey Trofimov) — Sat, 08 Feb 2025 00:00:00 +0100

I wanted a simple solution to read daily commit logs of the repos I follow, but couldn’t find any. For a while I resorted to an org-mode block that I had to execute time to time:

#+begin_src sh :results drawer :var repo='guix'
cd ~/devel/ext/$repo
git fetch origin master
git show-branch _track > /dev/null || git branch -f _track origin/master~20
git log --date=short --pretty='%ad %s' _track..origin/master
git branch -f _track origin/master > /dev/null
#+end_src

But quickly it started bothering me and I’ve jotted down a Babashka script that does the same:

#!/usr/bin/env bb

(ns git2rss
  (:require [clojure.zip :as zip]
            [clojure.string :as str]
            [clojure.java.shell :as shell]
            [clojure.data.xml :as xml]
            [clojure.java.io :as io]
            [clojure.data.zip.xml :refer [xml1-> text]]))

(defn commits-for [gitdir last-seen before]
  (->> (shell/with-sh-dir gitdir
         (shell/sh "git" "fetch" "origin" "master")
         (shell/sh "git" "log"
                   (str "--format=%H%n%an%n%cI%n%s")
                   (str "--before=" before "T00:00:00Z")
                   (str last-seen "..origin/master")))
       :out
       str/split-lines
       (partition 4)
       (map (partial zipmap '(:commit :author :date :subject)))))

(let* [filename (first *command-line-args*)

       now (-> (java.time.Instant/now)
               (.truncatedTo java.time.temporal.ChronoUnit/SECONDS))

       yesterday (-> now
                     (.atZone java.time.ZoneOffset/UTC)
                     .toLocalDate
                     (.minusDays 1))

       feed (-> filename
                io/reader
                (xml/parse :namespace-aware false
                           :skip-whitespace true)
                zip/xml-zip)

       ;; state from the input file
       last-seen (or (xml1-> feed :git2rss :last-seen text) "origin/master~10")
       gitdir (xml1-> feed :git2rss :dir text)
       commit-url (xml1-> feed :git2rss :commit-url text)
       range-url (xml1-> feed :git2rss :range-url text)]

  (def commits (commits-for gitdir last-seen (.plusDays yesterday 1)))
  (println (count commits) " new commits.")
  (when (empty? commits) (System/exit 0))

  (def latest-commit (:commit (first commits)))

  (def new-entry
    (xml/sexp-as-element
     [:entry
      [:title (str "Commits for " yesterday)]
      [:id (str "urn:git2rss:" yesterday)]
      [:link {:rel "alternate"
              :href (format range-url last-seen latest-commit)}]
      [:updated now]
      [:content {:type "xhtml"}
       [:div {:xmlns "http://www.w3.org/1999/xhtml"}
        [:p "Commits"
         [:ul
          (for [c commits]
            [:li [:a {:href (format commit-url (:commit c))} (:subject c)]])]]]]]))

  (spit filename
        (-> feed

            (xml1-> :updated)
            (zip/replace (xml/element :updated nil (.toString now)))
            zip/up

            (xml1-> :git2rss :last-seen)
            (zip/replace (xml/element :last-seen nil latest-commit))
            zip/up

            zip/rightmost
            (zip/insert-right new-entry)

            zip/root
            xml/indent-str)))

The last touch was to register a scheduled job in Guix configuration and to fix file:// style link support in elfeed-org.

And voila:

Self-hosted org-mode tools

sarg@sarg.org.ru (Sergey Trofimov) — Thu, 09 Jan 2025 00:00:00 +0100

I have a couple org-mode based tools now:

youtube.org for saving liked videos
movies.org to keep a log of watched movies
blog.org for this blog
books.org to keep a reading log

Each of these has an API heading that defines interactive helpers relevant for the file. E.g.:

* COMMENT API
#+NAME: api
#+begin_src elisp
(define-minor-mode org-books-mode
  "Helper functions for books.org"
  :interactive nil
  :keymap (list
           (cons (kbd "C-c C-o") #'org-books-open-goodreads)))

(defun org-books-open-goodreads ()
  "Open goodreads for current entry."
  (interactive nil 'org-books-mode)
  (browse-url (format "https://goodreads.com/book/show/%s"
                      (cdar (org-entry-properties nil "goodreadsid")))))

(org-books-mode 1)
#+end_src

# Local Variables:
# eval: (sarg/eval-org-src-block "api")
# End:

Code Snippet 1: API section

Where sarg/eval-org-src-block is defined as:

(defun sarg/eval-org-src-block (name)
  "Eval elisp block NAME. To be used in org files in Local variables section."

  (save-excursion
    (org-babel-goto-named-src-block name)
    (let* ((block-info (org-babel-get-src-block-info))
           (lang (nth 0 block-info))
           (body (nth 1 block-info)))
      (if (or (string= lang "emacs-lisp")
              (string= lang "elisp"))
          (cl-loop with next = 0
                   with maxlen = (length body)
                   for (sexp . next) = (read-from-string body next)
                   do (eval sexp)
                   while (< next maxlen))

        (error "%s is not an emacs-lisp src block" name)))))

Code Snippet 2: sarg/eval-org-src-block

This way when such org file is opened, the API code block gets loaded and a buffer-local minor mode gets activated. The mode is marked as :interactive nil so it doesn’t pollute M-x. Helper functions are restricted to this mode as well with (interactive nil 'org-books-mode). I like that such an “API” is contained in the file itself, making it a kind of self-hosted program.

Reprovisioning Guix on a new machine

sarg@sarg.org.ru (Sergey Trofimov) — Wed, 15 Mar 2023 00:00:00 +0100

No matter how often you backup, if you never test your recovery procedures - you’d be surprised when it comes to it. So let’s make a recovery plan and give it a test.

Back up user files

Having configured restic-backup-service, it’s just one command:

restic-guix backup storage

Disaster happens

I see two options:

I just need to move my digital stuff to a new machine.
I’ve lost access to my current machine.

Let’s follow the first option for now and prepare everything to migrate smoothly.

Make a raw disk image

That’s as simple as running guix system image. The only trick is to add a couple more services to the operating-system.

(use-modules (gnu)
             (gnu system)
             (gnu services admin)
             (gnu services guix)
             (ice-9 textual-ports))

(define system (load "./system.scm"))
(define root-fs
  (first (filter (lambda (fs)
                   (string=? "/" (file-system-mount-point fs)))
                 (operating-system-file-systems system))))


(operating-system
  (inherit system)
  (services (cons*
             ;; auto-resize fs on boot
             (service resize-file-system-service-type
                      (resize-file-system-configuration
                       (file-system root-fs)))

             (simple-service 'activate-storage
                             activation-service-type
                             #~(begin
                                 (use-modules (guix build utils))
                                 (let ((dir "/storage")
                                       (%user (getpw "sarg")))
                                   (mkdir-p dir)
                                   (chown dir (passwd:uid %user) (passwd:gid %user))
                                   (chmod dir #o700))))

             ;; restore /storage
             ;; (simple-service
             ;;  'restore-storage-from-backup shepherd-root-service-type
             ;;  (list (shepherd-service
             ;;         (requirement '(user-processes file-systems file-system-/media/500GB))
             ;;         (start #~(lambda _ (invoke #$(file-append restic "/bin/restic")
             ;;                                    "-r" "/media/500GB/restic"
             ;;                                    "-p" "/media/500GB/restic/pass"
             ;;                                    "restore" "latest")))
             ;;         (stop #f))))

             ;; install user profile
             ;; (service guix-home-service-type
             ;;          `(("sarg" ,(load "./home.scm"))))

             (operating-system-user-services system))))

Code Snippet 1: system-image.scm

guix system image --image-type=efi-raw system-image.scm

The following derivations will be built:
  /gnu/store/q0gvw0s79wkjlanfq4yjjyf2f9ddk4iq-disk-image.drv
  /gnu/store/dp0p01bzgb9aarzx040iv7k5w1r7z5xx-genimage.cfg.drv
  /gnu/store/kr39hrsyvwp17ny5hcrj6ynfn56i2cq8-partition.img.drv
  /gnu/store/758yrvd3m7016i5vflzjyg9232j75k2r-system.drv
  /gnu/store/58xsyrsaqaqh3dsxzb7zc0h72v7ha3as-etc.drv
  /gnu/store/5kifgf0yqnki7k9wg0a8jlviqibirmxy-activate.scm.drv
  /gnu/store/rqyl2kcj5q84daw4qriza38l0vkrax0m-activate-service.scm.drv
  /gnu/store/5lnmmgrgpy58nnkcpy8z22a0kf54yv0k-boot.drv
  /gnu/store/ri65rz6rgh5kv6ff8g2q313fr9fjd5ac-provenance.drv
  /gnu/store/z62g9qwy9k65awaaxg2f3jzdadsk5iha-grub.cfg.drv
  /gnu/store/x1zyd14c5xnpd51ql6bcfdv0q06nla8a-partition.img.drv

building /gnu/store/ri65rz6rgh5kv6ff8g2q313fr9fjd5ac-provenance.drv...
building /gnu/store/58xsyrsaqaqh3dsxzb7zc0h72v7ha3as-etc.drv...
building /gnu/store/rqyl2kcj5q84daw4qriza38l0vkrax0m-activate-service.scm.drv...
building /gnu/store/5kifgf0yqnki7k9wg0a8jlviqibirmxy-activate.scm.drv...
building /gnu/store/5lnmmgrgpy58nnkcpy8z22a0kf54yv0k-boot.drv...
building /gnu/store/758yrvd3m7016i5vflzjyg9232j75k2r-system.drv...
building /gnu/store/z62g9qwy9k65awaaxg2f3jzdadsk5iha-grub.cfg.drv...
building /gnu/store/kr39hrsyvwp17ny5hcrj6ynfn56i2cq8-partition.img.drv...
building /gnu/store/x1zyd14c5xnpd51ql6bcfdv0q06nla8a-partition.img.drv...
building /gnu/store/dp0p01bzgb9aarzx040iv7k5w1r7z5xx-genimage.cfg.drv...
building /gnu/store/q0gvw0s79wkjlanfq4yjjyf2f9ddk4iq-disk-image.drv...

And the disk image is: /gnu/store/rj2i98lw7vjj0k48dbwfs0q9aqnd9nxk-disk-image

Let’s check its partition table:

fdisk -l $disk_image

Disk /gnu/store/rj2i98lw7vjj0k48dbwfs0q9aqnd9nxk-disk-image: 2.32 GiB, 2488737792 bytes, 4860816 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 389B45C2-0EC7-42EC-9D96-49B40F820311

Device                                                  Start     End Sectors  Size Type
/gnu/store/rj2i98lw7vjj0k48dbwfs0q9aqnd9nxk-disk-image1  2048   83967   81920   40M EFI System
/gnu/store/rj2i98lw7vjj0k48dbwfs0q9aqnd9nxk-disk-image2 83968 4860775 4776808  2.3G Linux filesystem

And filesystems labels are:

alias kpartx=$(guix build multipath-tools)/sbin/kpartx
kpartx -av $disk_image >/dev/null
lsblk -o NAME,FSTYPE,FSVER,LABEL,UUID /dev/loop0
kpartx -d $disk_image

NAME      FSTYPE FSVER LABEL      UUID
loop0
|-loop0p1 vfat   FAT16 GNU-ESP    572A-DDF3
`-loop0p2 ext4   1.0   Guix_image f6bc8ad8-ff1a-1754-eb2b-35d0f6bc8ad8

Note that the partition labels chosen by guix system image are not taken from the operating-system definition. They’re hardcoded in the guix sources and therefore I have to hardcode the same values in my operating-system. Without matching labels the system would just refuse to boot as they’re used in the EFI loader to find the boot partition and also in the fstab.

LABEL=Guix_image  /      ext4  defaults
LABEL=GNU-ESP     /boot  vfat  defaults

Putting disk image on a real machine

I have a Ventoy USB stick with Lubuntu Live CD on it. Let’s just copy the resulting image to the stick. Note, you might need to follow this recipe: Access USB stick on live ISO as by default the USB drive will not be accessible.

eval $(lsblk -P -o LABEL,MOUNTPOINT | grep Ventoy)
cp $disk_image $MOUNTPOINT/guixdisk.img
eject $MOUNTPOINT

And then reboot to the Live CD environment and install the image.

dd /media/lubuntu/Ventoy/guixdisk.img of=/dev/sda status=progress
reboot

Revitalising Guix

One-off shepherd services added to operating-system earlier take care of restoring /storage from backup and activate the user home profile. Check that everything worked as expected examining /var/log/messages.

echo "Wait guix-home-sarg to complete"
until herd status guix-home-sarg | grep 'It is stopped'; do echo -n .; sleep 1; done

passwd root
passwd sarg
login sarg

# app-specific restore commands
doom sync
mu init -m ~/.mail
mu index

logout # sarg
logout # root

wpa-manager.el and torrent-mode.el

sarg@sarg.org.ru (Sergey Trofimov) — Sun, 12 Mar 2023 00:00:00 +0100

There is a nice bluetooth-mode to scan and connect to bluetooth devices, but no such mode for wifi. It’s time to fix that. I’ve started implementing wpa-manager which controls wpa_supplicant over its D-Bus interface. Currently the manager supports scanning and connecting using WPA PSK. It already solves most of my use cases.

SSID                     BSSID                            Freq  Signal▼
SARG                     5B:32:C6:8F:55:03                5180   -47
FRITZ!Box 7362 SL        7D:3D:12:D4:C1:1C                2417   -53
ZYXEL-795                65:18:77:8E:6B:0C                2412   -56
WLAN-123488              27:49:1B:71:5D:F7                2412   -62
FRITZ!Box 7530 IG        F0:B4:4A:AB:B4:9F                2462   -63
WLAN-WWAAAB              83:F9:C3:23:C1:1C                2462   -65
FRITZ!Box 7430 DR        63:E7:BC:09:16:C0                2462   -73
WLAN-123488              B2:8A:27:3B:C7:39                5500   -75

Another package is torrent-mode which provides a tabulated view of a torrent file content. An example of how it looks like.

   Idx ▼Size     Name
   000  2.1k     dumpruninfo.txt
   001  4.8G     enwiki-20170220-abstract.xml
   002  741M     enwiki-20170220-abstract1.xml
   003  353M     enwiki-20170220-abstract2.xml
   004  259M     enwiki-20170220-abstract3.xml
   005  202M     enwiki-20170220-abstract4.xml
*  006  168M     enwiki-20170220-abstract5.xml
*  007  161M     enwiki-20170220-abstract6.xml
*  008  183M     enwiki-20170220-abstract7.xml
*  009  161M     enwiki-20170220-abstract8.xml
*  010  133M     enwiki-20170220-abstract9.xml

This mode supports selecting individual files which could then be fed into aria2 using a small defun:

(defun torrent-mode-download-selected (dest-dir)
  "Download ARG entries."
  (interactive
   (list (read-directory-name
          "Directory: "
          (or (dired-dwim-target-directory)
              aria2-download-directory)
          nil
          t))
   torrent-mode)

  ;; aria2.addTorrent([secret, ]torrent[, uris[, options[, position]]])
  (make-request
   aria2--cc "aria2.addTorrent"
   (aria2--base64-encode-file buffer-file-name)
   []
   `(:select-file ,(string-join (seq-map (lambda (el) (number-to-string (car el)))
                                         (tablist-get-marked-items))
                                ",")
     :dir ,(expand-file-name dest-dir))))

Making org-mode movie diary

sarg@sarg.org.ru (Sergey Trofimov) — Sun, 05 Mar 2023 00:00:00 +0100

End result is here: Movies

Putting it in org-mode

Small demo of my flow:

Your browser does not support the video tag.

File organisation:

#+STARTUP: logdrawer
#+TODO: TODO(t) NEXT(n) | DONE(d!) DROPPED(c@)

* DONE La Jetée (1962) :romance:scifi:
:PROPERTIES:
:rating: 4
:imdb: tt0056119
:END:
:LOGBOOK:
- State "DONE"       from              [2022-07-11 Mon]
:END:
* Watch List [0/1]
** TODO Big Buck Bunny (2008) :animation:short:comedy:
:PROPERTIES:
:imdb:     tt1254207
:END:

And here is a minor mode to automate inserting new entries.

(define-minor-mode org-movies-mode
  "Helper functions for movies.org"
  :keymap (list
           (cons (kbd "C-c C-i") #'org-movies-template)
           (cons (kbd "C-c C-o") #'org-movies-open-letterboxd)))

(defun org-movies-open-letterboxd ()
  "Open letterboxd for current entry."
  (interactive nil 'org-movies-mode)
  (browse-url (format "https://letterboxd.com/imdb/%s"
                      (cdar (org-entry-properties nil "imdb")))))

(defun org-movies-template ()
  "Convert imdb url at point to a diary entry."
  (interactive nil 'org-movies-mode)
  (let* ((ctx (org-element-context))
         (link (if (eq (car ctx) 'link)
                   (plist-get (cadr ctx) :raw-link)
                 (error "Not an org link")))
         (imdb (or (nth 1 (s-match
                           (rx "imdb.com/title/" (group "tt" (+ digit)))
                           link))
                   (error "Not an imdb link")))
         (apikey (auth-source-pass-get 'secret "Sites/omdbapi.com"))
         (movie (with-current-buffer
                    (url-retrieve-synchronously
                     (concat "http://omdbapi.com/?apikey=" apikey "&i=" imdb))
                  (goto-char url-http-end-of-headers)
                  (json-read-from-string
                   (decode-coding-string
                    (buffer-substring (point) (point-max)) 'utf-8))))
         (genres-str (alist-get "Genre" movie nil nil #'string=))
         (genres (-map (lambda (el) (downcase (s-replace "-" "" el)))
                       (string-split genres-str ", ")))
         (title (alist-get "Title" movie nil nil #'string=))
         (year (alist-get "Year" movie nil nil #'string=)))

    (pcase (org-element-type (org-element-at-point))
      ('headline (ignore))
      (_ (beginning-of-line)
         (kill-line)
         (org-insert-heading)))

    (org-edit-headline (concat title " (" year ")"))
    (org-set-tags genres)
    (org-todo "TODO")
    (org-set-property "imdb" imdb)))

(org-movies-mode 1)

Code Snippet 1: Minor mode with useful helpers

Letterboxd export

I’ve fallen into this trap again… I’ve relied on a 3rd party website to manage my data. Yes, it looks good and it’s UX friendly, it’s easy to fall prey. This time it was Letterboxd. I imported my TheMovieDB watchlist there and used it as a film diary for a couple of years. One day I finally found some time to sort my movie collection with tinyMediaManager and the icing on the cake was to add my ratings to the NFO files. To my disappointment, Letterboxd doesn’t include the imdb id in the export files, so I had to write a script to fetch all their short urls one by one and extract the imdb ids from the html.

import re
import requests
import fileinput

# takes file with https://boxd.it/71O short links
for row in fileinput.input():
    row = row.strip()
    d = requests.get(row)
    imdb = re.findall('(?<=imdb.com/title/)tt[^/]+', d.text)[0]
    print(row, imdb, sep=',', flush=True)

Importing ratings in NFO files

Then another script to put the ratings in the NFO files.

import re
import sys
import csv
import glob

r = csv.reader(open("/home/sarg/rated.csv"))
rates = {}

for n in glob.glob('/media/sarg/2TB/Movies/**/**/*.nfo'):
    imdb = re.findall('(?<=<id>)tt\d+', open(n).read())[0]
    if imdb in rates:
        rates.pop(imdb)
        d = re.sub('<userrating>0.0', '<userrating>' + str(rates[imdb]), d)
        f = open(n + '.new', 'w')
        f.write(d)
        f.close()

print("High rated movies I don't have:")
for imdb in rates:
    if rates[imdb]>7:
        print(f"https://imdb.com/title/{imdb}")

Exporting Liked Videos from Youtube

sarg@sarg.org.ru (Sergey Trofimov) — Sun, 26 Feb 2023 00:00:00 +0100

Over the years I’ve collected ~300 favourite videos and unfortunately some of them have been taken down. I have spent some time re-watching the old videos and still enjoy them. Few are just for laughs, some are educational and others are just art. As there are not too many of them, I can just download them all to my archives. 14 videos that were missing on Youtube I’ve recovered from https://web.archive.org

The resulting archive weighs in at just 13G, so it really is a no-brainer to keep it offline.

# https://gist.github.com/guidocella/a272b6e68f9c44532b011f6596e95c61
{
    echo '# Netscape HTTP Cookie File' # needed by youtube-dl

    QUERY="SELECT
            host_key,
            IIF(host_key LIKE '.%', 'TRUE', 'FALSE'),
            path,
            IIF(is_secure, 'TRUE', 'FALSE'),
            IIF(expires_utc == 0, 0, expires_utc / 1000000 - 11644473600),
            name,
            value
        FROM cookies;"

    # There is no attempt to url encode $1, but SQLite already handles
    # characters like spaces, so only ? % and # should cause issues.
    sqlite3 -readonly -separator $'\t' \
        "file:${1:-$HOME/.local/share/qutebrowser}/webengine/Cookies?nolock=1" \
        "$QUERY"
} > cookies.txt

Code Snippet 1: Export qutebrowser cookies for yt-dlp

yt-dlp --ignore-errors \
    --download-archive archive.txt \
    --cookies cookies.txt \
    'https://www.youtube.com/playlist?list=LL'

Code Snippet 2: Download everything from the Liked Videos playlist

📖 Cracking The Code To A Successful Interview

sarg@sarg.org.ru (Sergey Trofimov) — Thu, 28 May 2020 00:00:00 +0200

Goodreads - 3/5.

First impression: Overall the tone of the book is slightly arrogant. The author’s ego is soaking from every page.

After the read: The book is best suited for a sales position in corporate America. Overall the message of the book is: Be prepared, know what’s expected of you and give it to the inverviewer in advance. In chapter 9 there is a summary of REAPRICH method. Chapter 10-12,15 are unrelated to hiring process.

Chapter 1.

Hiring managers could be of 4 types:

the power tripper - tries to prove his own intellectual prowess
the defensive crouhcer - feels unsafe with poeple they fear could do their jobs better
method man - follows script and expects valid answers
rogue agent - not fitting in 3 types above, dark horse

REAPRICH - 8 step plan to win the interview. results / energy / attitude / process / relationships / interview the interviewer / close the interviewer / humanity

Chapter 2

research the company
polish resume
be tenacious

On the day of interview - be on time, dress well, shine your shoes, bring hard copy of your resume, give a firm handshake, look into the eye.

Don’t discuss personal problems, family, bad experiences with past employers.

Chapter 3. REA

Be enthusiastic (E - energy) and have positive A (attitude).

Chapter 4. P - Spotlight on creativity

You’re in great shape if the manager ends up saying, “I want you on board so you can do this for me or my team. I want this to be a part of our skill set and our arsenal as we move forward as an organization.”

Chapter 5. R - Relationships

One of the biggest fears of hiring managers is having a personality conflict on their team. Showing that you have the ability to create and maintain relationships alleviates this fear.

Exercise: Write down two long-term relationships such as bosses, coaches, mentors, or friends (one which affected your career). Answer the following questions in writing for each: How did you secure this relationship? Was it through performance? How did this person become interested in you? How did he or she see positive qualities in you? How did you build and nurture this connection?

Chapter 6. I - interview the interviewer

Ask something that you can’t find in public sources.

Tell me about some of your most successful people. What’s made them great?
How can I help your team? What is the best way my skills could add value?
If you hired me today, what would you have me help you and your team with in the first three months?

Chapter 7. Close the interview

Like in sales - try to close the deal.

Chapter 8. Humanity

Small talk in the end. Author suggests to directly ask if you’re recommended for hire.

Chapter 9. Recap.

The most successful interview candidates aren’t necessarily the most experienced, the most educated, or the most qualified for the job. They are the best prepared.

Chapter 14. Tough questions

Who are you?
Why should I hire you?
Give an example of a time you tried your best and failed.
Who was the worst boss (or employee) you ever worked with?
What four words would someone close to you use to describe you?
Who was your best mentor?
What do you feel you will bring to this organization?
Where do you want to be in five years?
What is your greatest weakness?
What is your ninety-day plan to be successful here?
Give me an example of where you took a difficult customer or situation and turned it into a win.

📖 Growing Object-Oriented Softwared, Guided by Tests.

sarg@sarg.org.ru (Sergey Trofimov) — Tue, 19 May 2020 00:00:00 +0200

I’ve finished reading Growing Object-Oriented Softwared, Guided by Tests by Steve Freeman and Nat Pryce. The book is a good introduction into TDD. First it goes through core concepts and then demonstrates how to implement TDD on a simple project.

My takeouts throughout the reading:

TDD is a core practice of XP. TDD is not about keeping defects from the users, but helping the team to understand the features and to deliver them reliably and predictably.
Making code accessible for testing often drives it towards being cleaner and more modular.
TDD cycle: red/green/refactor
Different kinds of tests:
- acceptance - exercises the functionality we want to build. Does the whole system work? Whenever possible should be end-to-end without directly calling internal code.
- integration - does our code work against code we can’t change?
- unit - do our objects do the right thing, are they convenient to work with?
External/internal quality: customer facing or developer/admin facing
OOP focuses more on the communication between objects than on the objects themselves. An object has a method of handling every type of message that it understands and in encapsulates some internal state.
Objects could have different semantics:
- values model unchanging quantities
- objects have identity that may change over time and model computational process

TDD best practices:

First make a minimal “skeleton” of an app and set up deployment infrastructure. E.g. for a web app it would be a one page site which shows a field in database.
Develop from outer boundary. E.g. write end-to-end acceptance test and start unrolling the logic from the incoming external message.
The development cycle goes like this. When implementing an object, we discover that it needs a service to be provided by another object. We give the new service a name and mock it out in the client object’s unit tests, to clarify the relationship between the two. Then we write an object to provide that service and, in doing so, discover what services that object needs. We follow this chain (or perhaps a directed graph) of collaborator relationships until we connect up to existing objects, either our own or from a third-party API. We think of this as “on-demand” design: we “pull” interfaces and their implementations into existence from the needs of the client, rather than “pushing” out the features that we think a class should provide.

Exporting watch list from TMDb

sarg@sarg.org.ru (Sergey Trofimov) — Sun, 26 Apr 2020 00:00:00 +0200

I keep list of movies I’ve watched on TheMovieDB. The service is great and I’ve used it for quite a long time (>700 movies in my list). I was using another service before, which unfortunately had closed without giving me a chance to export the data and I had to scrape my watchlist from the google cache.

To prevent that from happening again, I’ve exported my watchlist from TMDb and will use the service only as a secondary source with a nice UI.

TMDb has a decent API and there are several python wrappers for it. The script for export is pretty straightforward. Retrieve request_token, open a session (login) for your account and retrieve the list of rated movies from a paginated endpoint.

import tmdbsimple as tmdb
import json
import subprocess

tmdb.API_KEY = "bafca4e813ca8fc43b3c9f1e32d1932b"
res = subprocess.run(
    ["pass", "show", "Sites/themoviedb.org"], capture_output=True, check=True
)
password = res.stdout.splitlines()[0]

auth = tmdb.Authentication()
token = auth.token_new()["request_token"]

auth.token_validate_with_login(request_token=token, username="sarg", password=password)

session_id = auth.session_new(request_token=token)["session_id"]
acc = tmdb.Account(session_id)
acc.info()

movies = []
idx = 1
while True:
    res = acc.rated_movies(page=idx)
    movies = movies + res["results"]
    idx = idx + 1
    if idx > res["total_pages"]:
        break

idx = 1
while True:
    res = acc.watchlist_movies(page=idx)
    movies = movies + res["results"]
    idx = idx + 1
    if idx > res["total_pages"]:
        break

print(json.dumps(movies, indent=2))

Unfortunately TMDb does not expose Rated Date via API. It could be extracted from the CSV export though. Request the CSV export on the “My Ratings” page (under three dots icon). It would be sent by email. Download the file and use the following script to merge the json and csv.

import csv
import json

rates = { row["TMDb ID"]: row for row in csv.DictReader(open("ratings.csv")) }
films = json.load(open("movies.json"))

for film in films:
    if str(film['id']) in rates:
        film['rated_date'] = rates[str(film['id'])]['Date Rated']

print(json.dumps(films, indent=2))

Android apps that I use

sarg@sarg.org.ru (Sergey Trofimov) — Fri, 24 Apr 2020 00:00:00 +0200

My phone is running AOSP without google services and almost all of the applications I use come from F-Droid. Few exceptions are the apps for paid services that I use and they are installed from Aurora Store.

I use fdroidcl to install and update applications.
Backup is covered in the post “Backing up an Android phone”.
I use syncthing to share a directory between my laptops and the phone.
I have “Password Store” files in the shared directory in case I need access to passwords on the phone (rarely).
In AFWall+ I whitelist only selected apps. All others are denied to access the net.

F-Droid apps

Name	Description
AnkiDroid	A flashcard-based study aid
AntennaPod	Easy-to-use, flexible and open-source podcast manager and player
Aurora Store	Play store alternative
BinaryEye	Yet another barcode scanner for Android. Free, no ads and open source.
Calculator++	Scientific calculator with sleek interface and powerful capabilities
Fennec F-Droid	Browse the web
Goodtime	Minimalist Pomodoro Timer
K-9 Mail	Full-featured email client
Kvaesitso	A search-focused, free and open source launcher for Android
KOReader	Ebook reader supporting multiple formats like PDF, DjVu, EPUB, FB2
LinPhone	SIP (VOIP) phone
Memo Game	Find pairs of cards
MuPDF viewer	Lightweight document viewer
NewPipe x SponsorBlock	A free lightweight YouTube frontend for Android with SponsorBlock capabilities
Offi Stations	King of public transit planning!
OpenKeychain	Encrypt your Files and Communications. Compatible with the OpenPGP Standard.
OpenVPN for Android	OpenVPN without root
Orgzly Revived	Outliner for taking notes and managing to-do lists in Org-mode format
OsmAnd~	Global Mobile Map Viewing & Navigation for Offline and Online OSM Maps
PassAndroid	View Passbook files
Password Store	Manage your passwords
PipePanic	A plumbers favorite game
Puzzles	39 single-player logic games
QKSMS	Messaging app
QuickDic	Offline translation dictionary
Quinb	A multiplayer reaction game: train your mind and your reflexes while having fun.
SatStat	GPS, sensor and network status
SimpleSSHD	Easy to use SSH server
Syncthing	File synchronization
Telegram	Telegram is a messaging app with a focus on speed and security.
VLC	The best video and music player. Fast and “just works”, plays any file
Voice	Simple audiobook player

Name	Description
KISS launcher	Custom launcher
AFWall+	Control network traffic
AnySoftKeyboard	Keyboard
Amaze	Manage local files
andOTP	Time-based One-time Passwords (TOTP) from QR codes for two-factor authentication
Clock	A combination of a clock, alarm, stopwatch and timer.
Loop Habit Tracker	Track your habits and long-term goals
Binaural Beats	Meditation helper
Survival Manual	Learn how to survive
Soundboard	Play short sound samples on touch
Red Moon	Filter blue light from your screen for a better sleep
GPSLogger	Lightweight GPS Logging
Der Die Das	learning genders of German nouns
Open Camera	Camera App
UnifiedNlp (no GAPPS)	Location provider middleware (UnifiedNlp)
OneTwo	Provides you with dices, counter and timers
Meditation Assistant	Feature packed meditation session timer and recorder
LocalGsmNlpBackend	UnifiedNlp location provider (local GSM database)
Planning Poker	Planning Poker for phone, tablet and wear devices
NominatimNlpBackend	UnifiedNlp geocoding provider (MapQuest Nominatim)
RadioBeacon	Contribute to coverage maps

Backing up an Android phone

sarg@sarg.org.ru (Sergey Trofimov) — Fri, 24 Apr 2020 00:00:00 +0200

One day my wife asked me to help her backup about 30Gb of photos from her android phone. At first we had tried to copy files over MTP (in Win10). This idea quickly showed its incapacity. You see, Win10 wants to be nice and tries to calculate ETA. But 30Gb is a lot of files and over MTP it takes one request per file to get its size. So, we waited half an hour while Win10 was preparing to start the transfer and I had aborted this madness.

Here is a sane approach to do the job.

Install SimpleSSHD on the phone. Set "Start on Open" in app settings.

Grab adb-channel script and add this snippet to ~/.ssh/config

   Host sshelper
       Port 2222
       ProxyCommand adb-channel tcp:%p org.galexander.sshd/.SimpleSSHD 1

Copy your public key to the phone.

Please note that ssh-copy-id won’t work. Just ssh sshelper and then paste the key from ssh-add -L into authorized_keys. Details here.

Now backup with rsync as usual.

   rsync -e ssh -av sshelper:/sdcard/DCIM .

Optional

This setup assumes that you copy files over USB. But nothing stops you from starting SimpleSSHD manually and copy files over WiFi.
```
   rsync -e 'ssh -p 2222' -av 192.168.178.42:/sdcard/DCIM .
```

Now that newer Android versions (newer than 12 I believe) don’t allow apps to access files of other apps, I had to revisit my backup solution.

Now I simply use the adb-sync script, but I also do adb root to give the script full access to any files. The script is pretty smart and it works kind of like rsync, downloading only newer files. It fits perfect the use case.

Making a hugo powered blog in emacs

sarg@sarg.org.ru (Sergey Trofimov) — Wed, 22 Apr 2020 00:00:00 +0200

Starting a blog this days couldn’t be easier. Ten simple steps to get started.

Create a github pages repo.

Check it out locally, e.g. to ~/blog. Every file you put in the repo will be published automatically:
By default, Jekyll doesn’t build files or folders that:
- are located in a folder called /node_modules or /vendor
- start with _, ., or #
- end with ~
- are excluded by the exclude setting in your configuration file
Configure custom domain or just use the default <repo>.github.io.

Create ~/blog/_hugo/config.toml

    baseURL = "http://sarg.org.ru/"
    languageCode = "en-us"
    title = "Random forest"
    theme = "anubis" # see the next step
    publishDir = "../" # github pages look in the root folder by default

Checkout some theme.

    git submodule add   \
    https://github.com/Mitrichius/hugo-theme-anubis.git \
    _hugo/themes/anubis

Add _hugo/content to .gitignore.

This directory will be generated by ox-hugo, no need to store it in git.
Create org-mode file where all the content would be placed.

Use the instruction from ox-hugo documentation. But make sure that #+HUGO_BASE_DIR points to _hugo directory. You can put the file whenever you want, not strictly necessary in the repository.
Export the file with ox-hugo.

Press C-c C-e H A to export everything. Now _hugo/content should be populated. You might want to enable auto export on saving.
Check that your blog works locally.

Go to _hugo/ and run hugo server -D. Hugo should start on http://localhost:1313
Generate html for github pages.

Run hugo and it’ll create all the files in publishDir.
Commit and push all the changes. You’re done! 🎉

Advent of Code 2019

sarg@sarg.org.ru (Sergey Trofimov) — Wed, 01 Jan 2020 00:00:00 +0100

For the first time I’ve participated in “Advent of Code” contest. It promised to be a fun time and a good opportunity to learn a new programming language. I’ve taken Clojure from my bucket list and dived head first into the contest.

The first few tasks are deliberately very easy, so that participants firstly would get into the mood and would not detract by the bar raised too high, and secondly to let one to get acquainted with the platform and rules, set up the coding environment and get hands-on with the new language (if you one uses such).

The problems were of varying complexity and some of them required prior knowledge of algorithms. The hardest one were about searching a path in a maze. The cool twist was the addition of the IntCode virtual machine. Over the course of a few days the problems were shaping the definition of the IndCode VM. Participants were expected to implement and to debug it in order to solve the problem.

Clojure has shown strong and I’ve rarely struggled with the language. Debugging tools could be better though. All the solutions could be found here on Github.

For an additional fun I’ve produced animations for some of the problems. For that I’ve used awesome Quil library which implements Processing.org API.

Problem 13 - Care Package

Arkanoid AI should be implemented in IntCode. Each block in the game has its own unique value and the answer of the problem is the end score.

Problem 15 - Oxygen System

A robot have been deployed to a maze of unknown structure. By issuing movement and observation commands the maze should be fully explored and a marked cell should be found. Then the time to fill the maze with air should be calculated, given that the air is injected in the bottome left corner and it spreads each tick to the adjacent cells.

"Parental control" for emacs

sarg@sarg.org.ru (Sergey Trofimov) — Fri, 21 Sep 2018 00:00:00 +0000

It easily becomes automatic to check for notifications/new messages/new emails etc. To give my rational self some time to intercept compulsive behaviour I have made this elisp snippet.

(defun sarg/otp-for-confirm ()
  "Generates random hex string"
  (format "%06x%06x%06x"
          (random (expt 16 6))
          (random (expt 16 6))
          (random (expt 16 6))))

(defun confirm-advice (proc &rest args)
  "Asks to enter random string as confirmation before executing PROC."
  (interactive)
  (let* ((pass (sarg/otp-for-confirm))
         (input (read-string (format "Enter %s if you wish to proceed: " pass))))

    (if (string= input pass)
        (apply proc args))))

;; Apply the advice.
(advice-add 'ivy-telega-chat-with :around #'confirm-advice)

Exporting secrets from FreeOTP to Pass

sarg@sarg.org.ru (Sergey Trofimov) — Fri, 21 Sep 2018 00:00:00 +0000

I have broken my phone screen. Not a big deal but I use it for 2FA. While replacement is on the way I will extract credentials from FreeOTP and import them in password store. Be aware that it is completeley against the idea of 2FA, because now both password and otp are on the same device.

So, first I need to find where FreeOTP stores OTP secrets. Android apps usually use sqlite db for that, but not in this case. FreeOTP uses shared_prefs.

Pull prefs from the phone (root needed).

adb root
adb pull /data/data/org.fedorahosted.freeotp/shared_prefs/tokens.xml

Here is what they look like.

<?xml version='1.0' encoding='utf-8' standalone='yes' ?>
<map>
    <string name="Amazon Web Services:sarg@organization">{&quot;algo&quot;:&quot;SHA1&quot;,&quot;counter&quot;:0,&quot;digits&quot;:6,&quot;issuerExt&quot;:&quot;Amazon Web Services&quot;,&quot;issuerInt&quot;:&quot;Amazon Web Services&quot;,&quot;label&quot;:&quot;sarg@organization&quot;,&quot;period&quot;:30,&quot;secret&quot;:[52,19,105,64,67,-120,-45],&quot;type&quot;:&quot;TOTP&quot;}</string>
    <string name="tokenOrder">[&quot;Amazon Web Services:sarg@organization&quot;,&quot;Google:sarg@sarg.org.ru&quot;,&quot;GitHub:sarg&quot;]</string>
</map>

Storing JSON in XML - that’s a beaty. If you decode the value, it looks better:

{
    "type": "TOTP",
    "secret": [52, 19, 105, 64, 67, -120, -45],
    "period": 30,
    "label": "sarg@organization",
    "issuerInt": "Amazon Web Services",
    "issuerExt": "Amazon Web Services",
    "digits": 6,
    "counter": 0,
    "algo": "SHA1"
}

Now I need to:

parse xml
extract json from xml
extract secret from json and encode it with base32
construct otpauth:// uri using details from json

URI format is described in google-authenticator wiki.

Provision a TOTP key for user alice@google.com, to use with a service provided
by Example, Inc:

otpauth://totp/Example:alice@google.com?secret=JBSWY3DPEHPK3PXP&issuer=Example

For one-time scripts I prefer using python. There is a handy library for dealing with html/xml - BeautifulSoup. And I will consult with FreeOTP sources to get things right.

#!/usr/bin/env python3

import urllib.parse
import base64
import json
from bs4 import BeautifulSoup

inp = open('tokens.xml')
soup = BeautifulSoup(inp, 'lxml')

for tag in soup.find_all('string'):
    # it's a special setting, skip it
    if tag['name'] == 'tokenOrder':
        continue

    token = json.loads(tag.text)

    # secret is stored as signed byte array
    # so first make it unsigned by adding 256 modulo 256
    # and then convert to python's bytearray
    secret = bytearray(map(lambda x: (x+256)%256, token['secret']))

    issuer = token.get('issuerExt', '')
    print('otpauth://{type}/{issuer}?{args}'.format(
        type=token['type'].lower(),

        # issuer should be URL encoded
        issuer=urllib.parse.quote(
            ':'.join(filter(None, (issuer, token['label'])))),

        args=urllib.parse.urlencode({
            'secret': base64.b32encode(secret).decode('utf-8'),
            'algo  ': token['algo'],
            'digits': token['digits'],
            'period': token['period']
        })
    ))

inp.close()

Now what’s left is just to import everything into pass with pass otp insert -e. Or you can manually put otpauth:// string in an appropriate pass file. I did the latter way, because my already existing entries have had another names. Then check that everything went fine - pass otp 'Amazon Web Services/sarg@organization'

Random forest

TOTP is easy

Literate Guix configuration

Intro

Header

Footer

Template

Channels

guix

Example content

package

service

simple files

templated files

Emacs config

ob-cfg.el

config package

config parts

Pulumi + SOPS

Self-hosted DynDNS

Cloudflare workers are cool

On-demand Wireguard VPN server on Hetzner

Generate two sets of keys and a PSK

Prepare a cloud-init provision script

Deploy the VM

Prepare client config

Start Wireguard

Cleanup after use

Further reading

Trying Pulumi

Following git changelogs in elfeed

Self-hosted org-mode tools

Reprovisioning Guix on a new machine

Back up user files

Disaster happens

Make a raw disk image

Putting disk image on a real machine

Revitalising Guix

wpa-manager.el and torrent-mode.el

Making org-mode movie diary

Putting it in org-mode

Letterboxd export

Importing ratings in NFO files

Exporting Liked Videos from Youtube

📖 Cracking The Code To A Successful Interview

Chapter 1.

Chapter 2

Chapter 3. REA

Chapter 4. P - Spotlight on creativity

Chapter 5. R - Relationships

Chapter 6. I - interview the interviewer

Chapter 7. Close the interview

Chapter 8. Humanity

Chapter 9. Recap.

Chapter 14. Tough questions

📖 Growing Object-Oriented Softwared, Guided by Tests.

Exporting watch list from TMDb

Android apps that I use

F-Droid apps

Archive

Backing up an Android phone

Making a hugo powered blog in emacs

Advent of Code 2019

Problem 13 - Care Package

Problem 15 - Oxygen System

"Parental control" for emacs

Exporting secrets from FreeOTP to Pass

Prepare a `cloud-init` provision script