Lots of overcomplicated answers, imo.

Here’s what I did:

1. Set up a DNS server (I just used Pi Hole docker)
2. Tell router to use the DNS server
3. Register domain
4. Set up Caddy with https to use DNS challenge with registrar’s API key for Let’s Encrypt or similar.
5. Add subdomain DNS entries to the DNS server for each service. The records point to Caddy’s IP (e.g. jellyfin.<domain>.com)
6. Add each container to Caddy’s list

Now every device at home can hit all services by domain name over https. No need for any manual configuration on any client device (certs, hosts, etc.)

Bonus: Set up split DNS and subnet routing with Tailscale so that it uses the DNS server for that domain. Now any device connecting to the tailnet can connect to the services just like at home. No need to reconfigure apps, etc.

Simple and it works like magic.