seyshee.dev

Self-hosting a tiny static site with nginx

2026-10-18T09:00:00Z

This site used to be a single index.html. It is now a few dozen HTML files produced by a Python script, and that's about as complicated as I want it to get. Here is the whole setup, mostly so that I remember it next time.

The build

There is no framework. A ~300-line build.py reads Markdown files with a tiny front matter block, renders them into one HTML template and writes everything into dist/. It also packs project tarballs and makes thumbnails for the gallery. The only dependency is Pillow.

I tried Hugo and liked it, but every time I came back after six months the theme had changed its config format. A script I wrote myself doesn't do that.

Deploying

Deploy is rsync to a VPS with a backup of the previous version:

ssh host "rsync -a /var/www/site/ /root/site-backups/site-$(date +%F-%H%M)/"
rsync -rlt --delete --chmod=D755,F644 dist/ host:/var/www/site/

Rollback is the same command in the other direction. I keep the last ten backups; the whole site is a few dozen megabytes, mostly images.

nginx

The server block is short. The interesting part is cache policy:

location / {
    try_files $uri $uri/ =404;
}
location /static/ {
    expires 1y;
    add_header Cache-Control "public, immutable";
}
location ~* \.(jpg|png|webp|tar\.gz)$ {
    expires 30d;
}

The stylesheet URL contains a hash of its content (style.css?v=1a2b3c4d), so it can be cached forever: when the file changes, the URL changes. HTML is not cached at all, which is fine, it is tiny.

Things I'd tell myself a year ago

try_files $uri $uri/ =404 plus directories with index.html give you clean URLs for free. No rewrite rules.
Turn on http2 and TLS 1.3 and forget about it. There is nothing to tune for a site this size.
Add a proper 404.html and error_page 404 /404.html;. The default nginx error page is ugly and tells everyone your exact nginx version.
server_tokens off; while you're at it.
A single-vCPU box with 2 GB of RAM serves static files faster than you'll ever need. The bottleneck is the uplink, not nginx.

That's it. Boring, which is the point.

Let's Encrypt renewals that never surprise you

2026-09-02T09:00:00Z

Certificates from Let's Encrypt are valid for 90 days and certbot renews them at 30 days left. In theory, you never think about it. In practice, every few years renewal silently breaks and you find out from a browser warning.

The three times it happened to me:

A firewall change closed port 80, so the HTTP-01 challenge could not reach the webroot.
Renewal worked, but nothing reloaded the web server, so it kept serving the old certificate from memory.
A second service used the same certificate files and was never restarted.

All three are fixable with a little discipline.

Use the webroot, and test it

certbot certonly --webroot -w /var/www/acme -d example.com -d www.example.com
certbot renew --dry-run

--dry-run talks to the staging environment and goes through the full challenge. Run it after any change to the firewall or the web server config. It takes ten seconds.

Deploy hooks

Anything in /etc/letsencrypt/renewal-hooks/deploy/ runs after a successful renewal. Mine is three lines:

#!/bin/sh
systemctl reload nginx
systemctl restart my-other-thing

Make it executable. certbot ignores files without the x bit, silently.

Monitor from the outside

The real fix for "I find out from a browser warning" is a check that looks at what clients actually see, not at files on disk. I wrote certpeek for that: it connects to each host, reads the certificate that is actually served and exits non-zero if it expires in less than N days.

$ certpeek --warn 21 example.com seyshee.dev
example.com     OK    67 days  (Let's Encrypt R11)
seyshee.dev     OK    74 days  (Let's Encrypt R10)

It runs from a systemd timer on a different machine and sends me a message if the exit code is not zero. That covers case 2 and 3 above, which no amount of looking at /etc/letsencrypt/live/ would have caught.

Reading TCP retransmits on a slow uplink

2026-07-21T09:00:00Z

My home connection is usually fine, but a couple of times a week uploads crawl. Before calling the ISP I wanted numbers. Linux already collects most of them, you just need to know where to look.

ss -ti

ss -ti prints internal TCP state for every connection. The interesting fields:

cubic wscale:7,7 rto:236 rtt:35.1/2.4 mss:1448 cwnd:10 ssthresh:7
bytes_sent:1849204 bytes_retrans:93312 retrans:0/66 lost:0

rtt:35.1/2.4 — smoothed round-trip time and its variance, in ms.
cwnd — congestion window in segments. If it stays small on a long transfer, the sender keeps hitting loss.
retrans:0/66 — currently outstanding / total retransmitted segments.
bytes_retrans / bytes_sent — the number I actually care about. Here it is ~5%, which is bad. On a healthy link it's well under 1%.

Reading this by eye gets old quickly, so I wrote retrans, which parses ss -tin and prints one line per connection with the retransmit ratio, sorted worst first. It can also run in a loop and print deltas.

tcpdump, when you need proof

If the ratio is high, a capture tells you whether loss is upstream or local:

tcpdump -i eth0 -w upload.pcap 'tcp port 443 and host 203.0.113.10'

Open it in Wireshark, filter on tcp.analysis.retransmission, and look at the timing. Loss that comes in bursts every few seconds usually means a queue somewhere is overflowing (bufferbloat). Random single losses look more like a bad line.

What it was

Bufferbloat on the router. Turning on SQM (cake with the upload limit set to ~90% of the real speed) brought the retransmit ratio down to 0.2% and made video calls stop stuttering while something is uploading. No call to the ISP needed.

Postgres advisory locks as a poor man's job scheduler

2026-06-03T09:00:00Z

We had a cleanup job that ran every ten minutes. Then the app went from one server to three, and so did the job. It was idempotent, mostly, but it was also slow and it held row locks, so three copies at once were noticeably worse than one.

The usual answers are "use a proper scheduler" or "pick a leader". Both are fine. But if you already have Postgres, there is a cheaper one: advisory locks.

The idea

SELECT pg_try_advisory_lock(42);

returns true for exactly one session at a time; everyone else gets false immediately. The lock is released when the session ends or when you call pg_advisory_unlock(42). The number is yours to choose — I hash the job name into a bigint.

In code

func runExclusive(ctx context.Context, db *sql.DB, name string, fn func(context.Context) error) error {
	conn, err := db.Conn(ctx) // the lock belongs to a session, so pin one connection
	if err != nil {
		return err
	}
	defer conn.Close()

	key := int64(fnv64(name))
	var ok bool
	if err := conn.QueryRowContext(ctx, "SELECT pg_try_advisory_lock($1)", key).Scan(&ok); err != nil {
		return err
	}
	if !ok {
		return nil // someone else is doing it
	}
	defer conn.ExecContext(context.Background(), "SELECT pg_advisory_unlock($1)", key)
	return fn(ctx)
}

The one thing that bites people: the lock is per session, so you must run lock, work and unlock on the same connection. With a connection pool, take a dedicated *sql.Conn as above.

Caveats

If you use PgBouncer in transaction mode, session-level advisory locks don't work. Use pg_try_advisory_xact_lock inside a transaction instead.
This gives you mutual exclusion, not scheduling. Every server still wakes up every ten minutes; only one of them does anything.
If the process dies, the connection drops and the lock is released. That's a feature.

Two years later this is still running. Boring, again.

Sunday plotting

2026-04-12T09:00:00Z

Some weekends I write small programs whose only output is a picture. The results live in the gallery. A few people asked how they are made, so here's the short version.

The toolbox

Everything is Python and Pillow. No numpy, no Processing, no GPU. Rendering one 3000×2000 image takes between five seconds and a minute, which is a perfectly good speed for something you look at for a week.

Pillow's ImageDraw does not antialias lines, so I draw at twice the size and downscale with LANCZOS. That's the single biggest quality improvement per line of code.

Noise

Most of the sketches start from 2D gradient noise. The whole implementation is about 25 lines: a shuffled permutation table, a table of random unit vectors, and a smoothstep between the four corners of a cell. Layer a few octaves (fractal Brownian motion) and you get something that looks like terrain:

def fbm(self, x, y, octaves=4):
    s, amp, f = 0.0, 0.5, 1.0
    for _ in range(octaves):
        s += amp * self(x * f, y * f)
        amp *= 0.5; f *= 2.0
    return s

From there:

Flow field — treat noise as an angle, drop thousands of particles, let each walk along the angle for a while and draw its path.
Contours — sample noise on a grid and run marching squares at a few dozen levels. Every fifth line is brighter, like on a real map.
Two peaks — the same noise as a horizon line, plus two gaussian bumps. If you've been to Yerevan you know which mountain it is.

I've cleaned up the noise and drawing helpers into noisepad in case anyone wants to play.

Why not numpy

I started with numpy and it was faster. But half the fun is reading the code a year later and still understanding it, and loops over pixels are easier to read than broadcasting tricks. Speed has never been the bottleneck — deciding which of the twenty variations to keep is.

I replaced cron with systemd timers and I'm not going back

2026-02-08T09:00:00Z

For years my servers had a crontab with ten lines and no idea whether any of them worked. Moving them to systemd timers took an afternoon. Here's what I got for it.

Two files instead of one line

# /etc/systemd/system/backup.service
[Unit]
Description=Nightly database backup
After=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/local/bin/backup.sh

# /etc/systemd/system/backup.timer
[Unit]
Description=Run backup nightly

[Timer]
OnCalendar=*-*-* 03:15
RandomizedDelaySec=20m
Persistent=true

[Install]
WantedBy=timers.target

Then systemctl enable --now backup.timer.

What you get

Logs. journalctl -u backup shows every run, its output and exit code. With cron I had >> /var/log/backup.log 2>&1 and hoped.
Missed runs. Persistent=true runs the job at boot if the machine was off at 03:15.
No thundering herd. RandomizedDelaySec spreads jobs across servers that share a config.
Status at a glance. systemctl list-timers shows the last and next run of everything.
Failure hooks. OnFailure=notify@%n.service in the [Unit] section, and a tiny template unit that sends a message. This alone was worth the move.

What you lose

The one-line convenience. For quick experiments I still use systemd-run --on-calendar=... which gives a transient timer without writing files.

OnCalendar syntax takes a minute to learn; systemd-analyze calendar "Mon *-*-* 09:00" prints the next trigger times, which helps.

A year of working from Yerevan

2025-12-14T09:00:00Z

It's been a bit more than a year since I moved to Yerevan. I promised a couple of friends I'd write down what working remotely from here is actually like, so here it is, in no particular order.

Internet

Better than I expected. Fiber is common in the center; I pay about 8,000 dram a month for a symmetric connection that's been down twice in a year, both times for under an hour. Mobile data is cheap and fast enough to work from a phone hotspot in a pinch.

Places to work

The city is full of cafes where nobody minds a laptop for a few hours as long as you keep ordering. My rule is one drink per ninety minutes. Coworking spaces exist and are fine, but I mostly work from home and go out for the afternoon.

Things I didn't expect

How walkable the center is. I haven't needed a car once.
The pink tufa stone everywhere — the tufa grid sketch is a small homage.
Ararat on clear mornings. You stop noticing it after a while and then one day it's just there, huge, and you stop on the street like a tourist.
Summers are hot. Really hot. August is a good month to be somewhere else.

Language

My Armenian is still at the "thank you, the bill please, how much" level. Russian and English cover almost everything. People are patient with the rest.

I'm staying.

In defense of boring backends

2025-10-05T09:00:00Z

Every few months I see a small project with a message queue, three microservices, a cache cluster and a service mesh, serving a hundred users. I understand the appeal — it's fun to build — but it is rarely what the project needs.

Here's the stack I default to, and I've yet to regret it:

One process. A Go or Python app that serves HTTP and runs background jobs in goroutines or a worker thread.
One database. Postgres. It is also the queue (SELECT ... FOR UPDATE SKIP LOCKED), the lock service (advisory locks), the full-text search and the cache for anything that isn't hot.
One reverse server. nginx in front, for TLS and static files.
One deploy script. Build, copy, restart, check. Keep the previous build to roll back.

When to add something

Add a component when you can name the problem it solves and you've measured that problem. "Postgres is too slow for this query" is a reason to add an index first, a read replica second, and a cache third. "We might need to scale" is not a reason.

What boring buys you

Fewer things to monitor. Fewer things to upgrade. Fewer ways to fail at 3 a.m. And — the part people underestimate — a system that a new person can understand in a day.

The most reliable systems I've worked on were the ones nobody talked about.

Backing up SQLite without stopping the app

2025-08-20T09:00:00Z

Copying a SQLite file with cp while the app is writing to it gives you a file that usually opens. Usually is not a word I want near backups.

Option 1: VACUUM INTO

Since SQLite 3.27:

VACUUM INTO '/backups/app-2025-08-20.db';

It writes a consistent, compacted copy of the database to a new file. It runs inside a read transaction, so writers in WAL mode keep working.

Option 2: the online backup API

sqlite3 /srv/app/app.db ".backup '/backups/app.db'"

This uses the online backup API, which copies pages incrementally and restarts if the source changes underneath. For big, busy databases VACUUM INTO tends to be more predictable; for small ones either is fine.

Then check it

A backup you haven't restored is a hope. My nightly job does:

sqlite3 "$OUT" "PRAGMA integrity_check;" | grep -qx ok || exit 1
zstd -q --rm "$OUT"

and once a month I restore the latest one on my laptop and click around.

WAL mode

If you haven't already, PRAGMA journal_mode=WAL; is the single best change for an app with concurrent readers. It also makes both backup methods above friendlier.