Isn’t this the nth time you’ve done this?

Yes! And what always happens is that I give up having to deal with it for one reason or another, usually caused by a stupid decision I made at some point. Thankfully, with all that prior experimentation I’ve arrived at a few nice decisions:

1. You don’t need to use ArgoCD/FluxCD for something this simple
2. Maybe I don’t need to self-host everything I think looks cool
3. Paying a little bit of cash to make things easier is perhaps fine

1

This took me by surprise. The plan was to use FluxCD since it makes more sense for a small setup like what I’m doing. But the sheer amount of stuff that needed to be done just for that was too much. I dialed my requirements back a bit and decided that a well documented README with everything handled by helm manually would be sufficient. We will see in a few months how that decision has been.

2

This is a me problem. There are so many things that could be self-hosted. Taking a few seconds back to ask “why” and “how” together can usually stop any sort of impulse. To this end I started trying to devise a list of what is essential:

• A personal wiki
• CI for Codeberg
• RSS Aggregator (for reasons that will be discussed in a later blog post)

This is a weirdly short list, but that wiki (an Otterwiki) is doing a lot of heavy lifting (like where I am currently writing this) that I might write up exactly how at some point. My main questions to help narrow things down are:

• Does it replace a paid/dodgy/sloppy product?
• Is it allowed to go down for X days?
• Does it need to be hosted?
• Is it easy to recover after catastrophe?

Generally if the answer is all yeses, it goes onto my shortlist. A key word in that first question is ‘replace’. I don’t particularly want to add a new application to what I already have UNLESS it’s absolutely critical. In this case, the ForgeJo Actions for use on Codeberg is one of very few exceptions I will be making.

Alas, while still paring everything down, there are some required essentials on top of this:

• Status and alerting system
• Some SSO provider
• A tunnel

But these are running on the VPS machines (yes, plural) so maybe it doesn’t count.

Of course in addition to these I will host my own services here (currently just LFM-GRID and snagg). Hopefully more coming soon…

3

Unfortunately homelabbing isn’t free. Even with an old ‘dead’ laptop. And it gets less free when you want to make it more secure. So I’ve decided to pay the 11€ a month for a pair of VPSs at UpCloud (that’s a referral link BTW just in case I’ve convinced you :) ). One VPS runs the SSO, status, notification, and health services. The other runs Pangolin. They are separated for 2 reasons: firstly, Pangolin is hungry. Secondly, I would rather isolate my health checks and notifications so I can be informed when anything dies (which reminds me, I need a Dead Mans Switch on the status VPS).

Newt

How I’m using Pangolin is by running Newt at the system level and exporting my services by port-forwarding (automatically) on those machines. However, running AlmaLinux (with SELinux) proved to be a bit of a shit for Newt, mainly because I’m a SELinux noob. chcon -t bin_t newt did the trick. If I decide to also forward my music server (on a different machine) it’ll be quite a bit easier since that’s just boring-old-Debian.

Iocaine

I had an idea to put Iocaine everywhere. I ’liked’ Anubis, but considering how ‘sloppy’ it now is and how it interrupts humans instead of scrapers. For my status VPS I’m using Caddy as the reverse proxy because it was easy. Putting Iocaine on there was also very easy. How nice. On the Pangolin VPS I’m using Traefik (since Pangolin defaults to that). Thankfully there was a guide on installing Iocaine as middleware there. So now every site I forward via Pangolin has Iocaine running with it! I currently have it configured at its most basic but I am thinking of switching it to use ‘Nam Shub of Enki’ instead of the default QMK. I also need to forward the metrics somewhere easy so I can check them out easily.

Future

I will likely add some more things to my “essential services” list soon, with the current contender being a todo/planning tool, and a GTS instance is looking might tempting. I’m sure there’s many more quirks I will have to deal with (like proper secrets management, which I think I’ll be going down the SOPS road again) but right now things seem nice.

An idea

Someone I follow on Mastodon had the idea of powering off services when you don’t really need them, but for mental health reasons. I like this idea quite a bit, but since I plan on hosting some publicly accessible things I would have to focus on scaling down based on load instead of outright killing things. I will probably give Keda a go at some point, although not sure how to get that to behave with the Forgejo runner. Although making some ‘power-off/power-on’ endpoint could be kinda neat….

Apologies in advance for the Cosmere names

The Media Server (oathgate)

This is a NUC from 2015 that I’ve increased the RAM, the SSD and the HDD over the years. It sites behind the TV and serves Jellyfin and Navidrome internally. It has run Ubuntu Server since around 2020 when I decided that it should never be plugged into any monitor (except for debugging purposes). This time, I’ve decided to go vanilla Debian because, well, I don’t trust Ubuntu any longer, and to be fair the “please pay” warnings in Ubuntu Server have been giving me the shits for about a year.

I’ve also decided to replace Navidrome with LMS almost entirely out of virtue that it’s not sloppy. The added MusicNN stuff will be interesting (if it ever finishes processing).

In an ideal world this would be connected to some network-attached-storage. Alas, the media now sits on an external HDD. This is not ideal, but I can live with it for now. It’s not as if NASs and drives are going to get cheaper any time soon…

The Homelab (coppermind)

This is a lower-end gaming laptop from 2019. Last time I went through this dance I used proxmox and started up various VMs to do stuff. This became annoying and complicated to the point that I just gave up on it. This time we’re using Alma Linux because I quite like it for servers. I was thinking about a *BSD but decided that I wanted to put k3s on it, which ruled all of those out.

In my office I am yet to finalise where it will live. Currently it sits on a bookshelf behind me sitting precariously on a document storage tray from Ikea. This is part of a much large discussion/plan/expedition.

The curse of having a k3s set up very quickly is that my plan for what I’m going to put on there is getting quite large, and the pod sprawl might become quite an issue. I also will attempt, for the 4th time, to use FluxCD instead of ArgoCD because a) it’s lighter, b) pushing instead of pulling suits a low interaction system, and c) I get enough ArgoCD at work.

More on all of this later

The Devtop (trha)

gotcha, this is not Cosmerical, just music

I’ve already spoken about this here but it’s Chimera Linux for reasons. This laptop is however still having problems, but according to 9 years of forum posts it could be caused by X of 7 problems. Joy.

The Main (scadrial)

Continuing the thoughts from April my main machine should be changing. But as much as I like the allure of Gentoo, this machine I would kinda like to be stable, since it’s where the gaming happens (when it does). I was thinking either restarting with EndeavourOS or moving to CachyOS but they both suffer from slop and CachyOS is sponsored by a brand that I very much disagree with. I could just go vanilla Debian but Niri has some issues there. This is the hard one to figure out unfortunately. Gentoo is my number one choice here still, but this will keep eating at me. Alas, had I stuck to my plan where that dev laptop was going to be Gentoo while I learned it than this may be less of an issue. I guess now at least if Gentoo dies on this machine I can always use the devtop.

Too Many Things

Ok, so I have 4 different distros for 4 different machines. For posterity I guess I also have 2 Macs, but they are less for fun. And the Steam Deck, but I kinda wanna factory reset it since I did some dodgy shit when I first got it.

I have too many computers. The VPS that I’ll be setting up doesn’t count, but that’s Debian at least.

Recently I mentioned that I found an old laptop I found in my storage shelves and decided to do something with it. Initially the plan was OpenBSD but since I wanted to maybe put some games (Factorio) on it and have a working WM (this laptop has a Nvidia GPU), I decided to stay in Linux land. But this gave me an excuse:

I could finally try out Chimera Linux. I’ve had my eye on this since the maintainer made a very clear anti-LLM statement (you know I’m a sucker for good ethics). It also has a whole bunch of other interesting features:

• FreeBSD-like userland
• dinit instead of systemd
• apk as the package manager
• musl instead of glibc
• doas instead of sudo

However, this laptop had a dead battery and a known overheating problem. Resolving these was surprisingly easy: iFixIt had a compatible battery in stock (70eu ouchie) and new thermal paste and thermal tape was about 15eu. There’s a bunch of guides on how to make this laptop model more heat resilient which basically just boils down to “put a more thermal tape in certain places”. About 30 minutes of that, I had a laptop that could boot and run for several hours in Debian without randomly dying (foreshadowing). Now that the preliminary stuff was done I could move onto installing Chimera.

This was a far less painful process than that time I installed Gentoo in QEMU. However, there were some weird things. The installation instructions are almost amazing. They make assumptions and leave out some clarifying words that would have prevented 1 failed install. Thankfully, someone has a fork of Chimera which uses glibc instead of musl for compatability reasons, called “Casuarina”. Why “thankfully”? Well the maintainer of this rewrote the install guide and added in some clarifications and missing info (albeit they went with systemd-boot instead of GRUB but ironically that section helped).

There’s not much really to say with the install process. Having iwd work immediately was amazing; having to redo my partitions 3 times wasn’t fun. But the main issue was actually this laptops’ firmware not being particularly nice with GRUB. The --removable flag solved that though after one entirely unneeded reinstall.

Then there was the next stage of getting a greeter to work. Short story: greetd and tui-greet are a wonderful combo. Installed niri and got my dotfiles. A few quirks dealing with the FreeBSD-like userland (this actually improved some scripts) and I got my swaybg and swayidle scripts working. Some more usual things and it’s all good to go.

Yep.

I was hesitant about dinit after my time really enjoying OpenRC. That was silly of me. dinit is wonderful. I was able to setup custom services in 3 lines per service. Insane.

I knew the userland and musl changes would cause some problems once I started trying to build some projects but it seems to only be a single OpenSSL dep doing this, so that shouldn’t be too annoying too resolve…?

Unfortunately, there is one mystery problem that I remember this laptop having before, except now I can recreate it! Sometimes it will just randomly reboot and nothing shows up in the logs. If I visit <newsblur.com> in any browser and wait 5 - 10 seconds, it will happen. I have a suspicion it’s GPU related, but I need to validate that somehow. This wouldn’t be much of a problem if Newsflash worked, but it doesn’t which is most peculiar. I will continue trying to figure this one out because I need my RSS feeds (although I plan on moving away from Newsblur after 13 years since the slop coding is now strong there). This no longer happens. I am now more perplexed

A final gripe with Chimera is again due to musl and the strictness of cports (these are both things I very much agree with however): I have to use Flatpak for certain things. And I don’t like Flatpak because it is very good at wasting space. It has been a few years since I used it so maybe it’s less ridiculous now, but we’ll see. There’s also a delightful bug with Niri currently where no Flatpak apps can access the clipboard or vice-versa. Having to install Bitwarden via Flatpak makes it a bit useless…

Overall, I like Chimera so far. It’s making me think in yet another way but also needing to find replacements for things (hurry up and finish plz #5506). Let’s see what happens when I start actually developing with it. It’s a slight shame since I wanted to move away from Niri and Waybar (ravaged by slop) on a machine I would use often before making whatever my replacements. Maybe I’ll just do that in a VM instead. And then eventually I’ll change my main PC to use Gentoo. One day.

A warning: There’s a good chance that in the first section of this post I will state some things that appears as fact. If it doesn’t have a source linked to it, that likely means it’s something I have read but not saved where I read it. I read a lot of things online but don’t save them because I never think I need to refer to them (unless I’m doing that deliberately). If you disagree with something and have a source(s) I would be delighted to read them. This warning is basically here since having any view on systemd, coding LLMs, or ‘fascism in OSS communities’ tends to attract unsolicited opinions

The Story So Far…

🎵 carry on my wayward son 🎵

I’ve been an Arch Linux user for a few years now. Before that it was Debian (but as a surplus machine to a pair of MacOS & Windows machines). I’ve also mucked around with other distros over the years (putting crunchbang! on an EeePC is something I weirdly remember). However, due to some recent happenings, my personal views don’t necessarily align with using Arch as my primary distro. Of course it does make some things easier (mainly gaming and updating) but here is where you will either a) stop reading, b) get angry at me or c) carry on with trepidation. I don’t like where systemd is heading (or has been) for a while now. In the same week 2 things happened: they allowed LLM generated code, and they allowed that pre-age-verification-compliance code. The former is an infuriating debate that won’t stop (you know my side at least) and I’ll try not to dwell on it here (it will show up again though). The latter is a worrying slide away from Linux principles of what people were warning about nearly a decade ago when systemd somehow kept being the init system of choice for nearly every distro. People more eloquent than me have written very good pieces about this, which I unfortunately never saved to link here.

A Visual Aside

Simultaneously I have been looking for a new desktop environment. I love KDE’s philosophies but the software just works in ways I don’t like. This is not a new thing though, as I’m sure the KDE/GNOME fight is with for every Linux user at one time or another. I don’t like GNOME’s gentle shift to LLMness (albeit that is yet to be proven). This whole thing was to be done during a full system refresh (there’s a slight issue in my bootloader I also want to get rid of). However, during trial some DEs I ended up switching to Niri, because it does everything I want it to in the way I want. This will become important later.

The Easy Way…

‘Out of the fire’ if you will

As a ‘sensible’ person I did explore some alternatives with the primary goal of ‘removing systemd from Arch’. What exists are 2 things that I would not be happy using, and a third that just did not work.

The Bible Failed Us (or how maybe I’m not that great)

The thing that did not work would have been the easiest solution, in the sense that it would have impact my daily life very little, was to rip out systemd from my installation and replace it with OpenRC. Heck, there’s even a page on the venerable ArchWiki with instructions on how to do this.

It did not work, I had to revert everything, and I lost about 45 minutes of my life. I (perhaps stubbornly) decided that I would wrap this whole ordeal into a full system wipe that I’ve been wanting to do.

If you have successfully done this, I would be keen to hear!

So What Does Google Say?

Artix. Even guides that say “do not follow the ArchWiki” recommend using Artix (or it’s repos at the very least). Unfortunately, Artix suffers from a slight fashy issue I am told. There is no direct evidence of this I could find but core devs stating they are in the middle and accepting of anyone echoes Framework’s “big tent” post (if you are unfamiliar with that, please search for it and why it was written). As I cannot vet these concerns either way, I do and will not trust it.

So What Does a Different Search Engine Say?

ArchBang. Huh, this was new. I love a low effort developer page. I agree with a lot of what the dev does. Unfortunately on check the repo I was hit with the Github slop warning¹. Checked for the usual suspects and sigh. Re-read some of the devs posts. They’re not quite full in with the LLM-do-everything-crowd but they use it more than I’m comfortable with. I back this up with that this is also a very hobbyist distro and not a daily-driver.

Ahh shit

The Hard Way

‘And into the frying pan’

I spend a bit of time on the “anti-LLM” side of the internet a.k.a Mastodon and sometimes I find an interesting post leading to an interesting source of information. This particular one brought to my attention that a few distros had taken strong stances on LLM code, that is outright refusing it. Of interest to me were Gentoo (obviously) and Chimera. Chimera scratches a load of certain itches but it doesn’t look too friendly with one of my daily-driver requirements: gaming. And so Gentoo entered the “maybe I should really do this” discussions in my head. I have little experience with Gentoo-running folk, except for that once guy I was friends with back in uni who spend the whole day compiling Gentoo on his laptop and had to sit in awkward places around campus (2010 was a simpler time). But also why are these 2 of interest to me? Well they don’t use systemd. Gentoo defaults to OpenRC (but does offer easy methods to use systemd), while Chimera uses dinit.

Welp then, Gentoo it is. For those unfamiliar, Gentoo has quite the reputation for being ‘difficult’. This is due to the fact you ‘basically’ configure it from scratch. And since I’m a paranoid person that doesn’t want to be without a primary PC for a while, I thought I’d try it out in QEmu first.

Quizzical Emu (wrong on both counts)

The Gentoo community is quite nice it seems as the Handbook for AMD64 is pretty good. As long as you actually follow it. Full disclosure: I use virt-manager to help with QEmu stuff, which is a bit of a double-edged sword. My goal here is to get Gentoo booting into Niri and get some services designed for systemd running.

Attempt 1

Following the Handbook is easy, just a lot of typing. Foolishly decided to use the dist-kernel and about an hour in (while I’m in another room) I run out of disk space.

• Lesson 1: more disk space

Attempt 2

Ok, VM drive now has 3 times the space. Find a comment saying “for your first Gentoo install maybe use the dist-kernel-bin to save on time”. Good advice. All is going well until setting up GRUB and I get a load of weird EFI related warnings. Oh right, I guess I need to use non-EFI boot drive for QEMU. That makes sense. This attempt was also hampered by me fucking up the USE flags for installkernel THREE separate times.

• Lesson 2: actually know what the system is compatible with
• Lesson 3: Measure twice, cut once

There was an attempt 2.5 where I restored a snapshot and tried again but I forgot to emerge sys-kernel/installkernel after setting the dracut USE flag. Oops

Attempt 3

Get all the way through and Gentoo boots! Big relief (I enjoy an easter egg as a celebration). Now to install Niri. Following the Gentoo wiki this is easy. I execute dbus-run-session niri --session and everything halts.

My inexperience with QEmu and lots of things now becomes a problem here. First thought is that maybe this VNC viewer isn’t capable of handling graphics. So I try configuring the display to Virtio with no 3D acceleration. Same result. This time I enable 3D acceleration and get greeted with an error saying I need to configure a GL backend. Fair enough, but in virt-manager this isn’t possible, it needs a command-line option. Eventually I find that /var/log/libvirt/qmeu/NAME_OF_VM.log stores the qemu command that virt-manager called. Try it out with the suggested fix and still nothing. Now begins the digging. I find a lot of stuff about libvirt and QEmu and OpenGL having myriad issues with myriad causes. Eventually I track down a similar problem on a Github issue for libvirt. Ultimately this works (there is some stupidity that I do applying this fix that makes it take longer than it should).

And now I have Gentoo running Niri in QEmu!

Fun Times Ahead*

Now I need to see if I can get some other things behaving nicely, specifically SDDM, a daemon I use, doas and maybe a couple of other things. I would also maybe test Steam but that’s definitely pushing the limits here. I also want to try building and managing a tool from source that has no Gentoo flavour.

I had technically set doas up during the install phase but with a very annoying config:

permit :wheel

I would rather like it to behave like how sudo defaults: 2 retries, remember rights for 5 minutes. Well the retry prompt doesn’t seem to exist but persistence does:

permit persist :wheel

Nice.

Why doas? I’ve heard good things about it (and also capsudo) so I thought I’d try it out

SDDM was also easy thanks to the docs (yet again). Tiny Niri issue but not a big deal. Now for the first daemon: jottad. I use Jottacloud as my Dropbox-replacement. As much as I have minor issues with it, it’s pretty good and the companies overall mission is good (except the lack of OSS but that’s pretty normal with cloud storage companies).

jottad

This is a little more involved. Firstly, you need to get the bins from their site. The tarball actually contains some default config and basic docs, which was very unexpected. However, it’s built around systemd. So that means writing an OpenRC script, which is weirdly tiny.

#!/bin/bash
command="/usr/bin/jottad"
pidfile="/run/${RC_SVCNAME}.pid"

After moving the bins to /usr/bin and the default config to /etc/jottad/, rc-service start jottad does…nothing. No logs. Just nothing.

Reading some docs I find this line from the OpenRC Service Script Writing Guide:

The interpreter for service scripts is #!/sbin/openrc-run. Not using this interpreter will break the use of dependencies and is not supported

Oops. I switch to that and I’m greeted with * WARNING: jottad is already starting but nothing actually happens. A quick search for this and I end up back on the Gentoo OpenRC page. I just need to run openrc to see what’s going wrong, which seems to be start-stop-daemon: --stdout and --stderr are only relevant with --background. Straightforward I guess. I add the line anddd it starts! So the complete script is as follows (extra notes in comments):

#!/sbin/openrc-run

depend() {
  need net # jottad needs to look at 127.0.0.1, I didn't try 0.0.0.0 that would make this not needed
}

command="/usr/bin/jottad"
pidfile="/run/${RC_SVCNAME}.pid"
command_args="stdoutlog"                # tell jottad to use stdout for logs
output_log="/var/log/${RC_SVCNAME}.log" # need to put logs somewhere
error_log="/var/log/${RC_SVCNAME}.err"  # need to put errors somewhere
command_background=true                 # from error message above
command_user="root:jottad"              # uhh this might be wrong but I'll deal with that later

Unfortunately I now need a way to handle updates. That will come later once I’m fully Gentoo’d (that sounds gross).

To test jottad I kinda need to install a browser (Jottacloud uses token auth and copy/paste in QEmu ain’t there), and since I’ll be doing that anyway I might as well and install Librewolf (but the bin version, I don’t feel like waiting for several hours). But first…

fuzzel

Let’s get some more of my ’normal’ functions going here. I use fuzzel as my launcher because I ran into a lot of issues with the other one I was using (that I forget the name of) and I just decided “simple is best”. And here is where I learned about “unmasking”. Conceptually it makes sense, but practically it would be really nice if there was an easy way to see why and when a package was masked. The wiki to the rescue.

Librewolf

Man, that Gentoo wiki really is something. It’d be better than the ArchWiki iff more people used Gentoo I think. Anyway the Librewolf page is pretty good. And it works.

Running out of things tricky to test now. Everything else I use is pretty standard, except maybe my dotfile management.

chezmoi

Oh, it’s in Guru. No biggy. Easy. I guess this didn’t deserve it’s own section.

Steam

From the steamed hams

Again, the wiki entry steps in clutch with it’s slightly-poorly-worded paragraph about ‘pure Wayland systems with a global -X flag’ (guess what I did). Alas, emerge still blocks me due to dbus having it’s ABI_X86 flag set to 64. A quick search to validate if I was correct and I was halfway right. Instead of setting the ABI_X86 USE, I just needed to sigh enable the X USE flag for it. Now I can install steam-launcher…

…Why is it compiling llvm?… This was the only surprising thing to happen and it’s because I didn’t actually check what was being pulled in.

4 episodes of One Piece later and we’re done. Well, after a restart really. I’m not going to try and get WINE or Proton working just here, because a) it will take a while and b) testing it will be a bit of a pain. But I can do something…

I think that’s that then…

Conclusion

Am I ready to make this on my main PC? Maybe still with the dist-kernel. But in that case maybe I should practice with the source kernel in QEmu (albeit that will take many hours to compile). I do have an unused laptop (“unused” meaning it has Proxmox on it but it’s been dormant for a couple of months) that would make a great weekend project and a real testbed (i.e. test full functionality by plugging it into my setup and using it for a couple of hours/days).

The setting up of various USE flags and licenses etc. is really cool, BUT bringing a system back to working after a catastrophic failure would be no mean feat. I’m not really sure what the best approach to backing up and restoring alllll those files would be. I think that before I start the full system wipe, I need to resolve that. And also investigate some sort of ‘rollback’ management. The ability to completely break your system by accident is high. This scares me a bit, mainly because I don’t want to spend 5+ hours setting things up from scratch if I fuck something up. And well I totally forgot about when the profile updates. That I have no idea about. I just need to pay attention to the eselect news list outputs.

As I alluded to back at the start of this, I’m also in the middle of changing my display environment and entire workflow (this includes switching to vertical tabs and still migrating to helix from vim). There are still some things that need to worked on there, but most of those things are effectively trivial. This is the first time in 16 years I’ve done without a ‘Quake-style’ terminal and that takes a bit of work to undo. I still also have a bunch of other things I need to do, such as restoring my web services.

If, somehow, you’ve made it this far and are also curious about trialling Gentoo: do it! Maybe just use this post as inspiration and not a source of truth.

Some Other Miscellaneous Learnings

1. Mirrors can sometimes be borked. I need to see if there’s a way to reduce the timeout. I wish there was a mirror-ranking system that could be done once a week or even daily. Maybe I need to use the rsync versions instead
2. Searching for packages can be a bit of mess, perhaps there’s a way to make this easier…
3. There is a lot more manual work than I anticipated, but mostly coming from other places (see package management). It’s interesting, there’s some really easy solutions you could do but they would take away from the spirit of Gentoo. I just wish the some more critical information was more readily available
4. Setting up packages take a lot longer than you expect, even when everything is working properly. Find some other things to do while you wait

I’m sure I learned some other things while doing this (I know I did) but I think these are the main things to consider when switching to Gentoo. It’s not the groundbreaking learning experience I’ve seen people say online, but it’s certainly not a passive experience.

This was mostly written while having Soulless’s “Shiny In Purity” on repeat. Cracking album, give it a spin

• All ongoing projects under one repo
• All homelabbing stuff is split into 2: one public, one private (secrets are still handled properly)
• One for my writing and knowledge-acquisition
• pages (this is new and will serve a purpose later)

This is a fairly boring process by hand, and that is why bash exists! Here’s the script I used to dump whole repos into subdirs in a new repo, while preserving all their history. This is extra beneficial if the project becomes good enough to warrant going back into its own repo. This script is actually pretty straight forward but is *nix only, thanks to the xargs. You can probably use git filter-repo but I wanted a result quickly and someone on SO had such as nixxy solution.

#!/bin/bash

REPO=$1
REPOMIG=$REPO"migration"

git remote add $REPO ssh://git@codeberg.org/{YOUR_USERNAME}/$REPO.git
git fetch $REPO
git branch $REPOMIG $REPO/main
git checkout $REPOMIG
mkdir $REPO
git ls-tree -z --name-only HEAD | xargs -0 -I {} git mv {} $REPO
git commit -m "migrated $REPO to $REPO/"
git checkout main
git merge --allow-unrelated-histories -m "Assimilated $REPO" $REPOMIG 
git push -u origin main
git remote rm $REPO

Now I just need to make a script that does the opposite (although I don’t actually know when that would come in handy).

Apologies if you’re in dark mode and the codefence is blinding you. My CSS appears to no longer be working, but that’s what I get for not checking this blog for 10 months.

So I finally did it. I bought a ZSA Voyager. And also a shit load of keyswitches. Let’s get the big question out of the way first: do I like it? Yes, but… It’s a great keyboard, even if my comparison points are not very mechanical. The ‘but’ though does a LOT of heavy lifting. But let’s start with the ‘why’.

In previous posts I’ve talked about wanting a split keyboard that helps my back (yes unlike most people I see, my main issue is posture and not wrist/finger pain). I am a large individual and hunching over all day everyday isn’t great. Also I’ve just wanted a split ergo keyboard that looks cool. On these fronts, it’s a success. And really I just like things that are functional, so maybe that was all the motivation I needed all along.

But now the bad. There’s 2 aspects here. The first is about relearning how to touch type and things of that ilk. The second is about the board itself.

The (Mavis) Beacons are lit!

Eons ago (years that start with 19 and a few beyond that) I had typing classes. Really. Some of these were at school, some my mum signed me up for (I started having to use a laptop at school in 1998/year 5 because dyslexia and dysgraphia and needing to type at a reasonable pace was required), but they never really got embedded. Sure I know the basics of it but like most people I developed my own style over time because it was easy and fast enough. I really wish now that I had actually stuck with it properly because that would have probably saved a lot of time (literally) over the years, but also now would prevent this post from existing. Because holy shit, I am fucking slow now that I have to use a new form-factor of keyboard with a new layout of keycaps.

Of course I didn’t go into this journey blind, I watched and read quite a few things to help me decide. Hell, like three days before buying this I almost bought the Dygma Raise 2, and thank fuck I didn’t (more in the board section). One thing that no-one seemed to cover was the absolute mental block that happens when having to relearn typing. Everyone talks about how it’ll be slow and painful and slow, but this mental block is THE hardest thing. Specifically, having to kinda think about where your fingers should go to press what symbol, especially if its on another layer, not only disrupts my typing but actively discourages me from doing things. For example, I had to migrate a lot of code at work (re-monolithing a service for reasons) and I kinda did it half arsed because having to fix and type things just elicited some level of anxiety in me. It’s also in my personal projects, which being in Rust sure requires a LOT of typing. However, I am stubborn. So I power through because I want to master this thing, and not have another thing put into a box stored in the shelves behind me.

And yes, this entire time I’m sticking with QWERTY for the simple fact I need to use other keyboards at times (in both US QWERTY and Nordic ISO forms). Maybe when I master this I’ll explore Colemak-DH or Graphite for some real fun and speed and not having my right little finger be a jerk (after months of pain). Although sticking with QWERTY has allowed some bad habits to stick around like “pressing K with right little finger” and “pressing A with the left middle” but they should go away after some time…

An extension to all this is that the Voyager has only 52 keys. That means layers and programming it the way that works best for me. I’m still figuring out whats best but right now it’s not too different from the default, but I’m still working on it (where space and backspace + delete are is a constant source of frustration for my fingers). Speaking of the board…

Taking a trip

(Voyager, get it?)

I opted for the Pro Red keyswitches because I read somewhere they were the quietest. I don’t know what scale those people operate on but they sure as hell weren’t. They were deafening. Thankfully a post tipped me off to some very quiet switches called “Ambient Nocturnals”. From what I had read about these, they are practically silent but very lightweight. Mechboards had a heap of those in stock, but also the slightly heavier ‘Twilight’s. I bought 5 sets of each (fucking spenno I know) and I was so glad I did, because it turns out they are indeed silent. And also the Nocturnals are indeed slightly too light for me. However some of the Twilghts died in transit (3 actually) so I have a few Nocturnals in here and it’s actually noticeable. BUT the important thing is that it’s almost as quiet as my Logitech MX Keys S, which is amazing and I don’t cringe at every keypress now.

Above I mentioned I was a large individual, and that corresponds to my hands as well. A reduced key, low-profile, small-footprint board should have given me some hints about this but the Voyager is small. That has been quite a challenge. Have a picture to show my concerns.

However, it’s been useful to my left thumb which is slightly smaller than it should be due to an accident when I was 10. There’s not much I can do about this apart from getting used to it, which is slowly happening.

One main complaint I have with this though is how I wish for a third thumb key. That would make so many things just that bit easier, but I can see why they stuck with two. Okay, one more complaint was that I wish it could store profiles. I jump between Linux (daily personal), Mac (work), and Windows (gaming I guess) frequently. I guess a solution is for one system to have the ’true’ layout and then use layout software on the others to line things up. Basically this is just an ALT, CTRL, Meta/CMD problem. I’m sure that someone out there has figured it out…

Onwards

It’s only been a week with this, so there’s still a lot more work to go. I have only switched to my MX Keys once and that was out of a need to do something quickly. The main thing I have been changing around is my trackball. I’m still getting used to that, but during this week I have mainly used my mouse because I needed something to be normal and fast. I am now back to trackballing and it’s fine. Just gotta learn and get used to it all. I of course have been spending a bunch of time with monkeytype doing long stretches of english and moderate bursts of Rust for symbol training.

Just gotta keep voyaging (ick I know).

As such, ‘Locke’s Stupid Clockify Analyser’(LSCA) was born. My initial goal was simple: get all my Clockify data and plot it against various things. Getting to that point was easily achievable via a spreadsheet, so it needed more. I have no idea where the following came from, but at some point I thought correlating my working time with various weather would be interesting.

After probably around 10 - 12 hours of coding I came to a natural conclusion point with this project, so now I’ll explain it a bit in depth then go through the results.

LSCA

LSCA can be split into two parts: data wrangler and plotter. The data wrangler does the following:

1. Imports a Clockify CSV export
2. Flattens the entries into single days
3. Gets weather information for that day at my lat/lon
4. Store it in an SQlite DB

Pretty simple. The weather data is all from <open-meteo.com> which fucking rocks.

The plotter is where most of the coding time went. I’ll explain shortly, but this basically does what it sounds like:

1. Run some SQL query
2. Do some data post-processing
3. Draw some plots

And if you thought I did this in some sensible language for doing this type of stuff, you’d be wrong since I used Go. Why? I wanted to do something small (and finish it) in Go because I’ve been using it more and more at work (and this project actually really helped with that).

Components

• For the queries I used go-resty, which I’m very familiar with. However, I tried using v3 but it refused to behave, so I went back to the comfortable v2
• For the CSV parsing I used gocsv because I can directly unmarshal a file into an struct type with little pain. Big recommend
• For the SQLite stuff I used go-sqlite3 which is fine. It requires CGOENABLED at compile time which I don’t particularly like, but it works and doesn’t shove any ORM in my face (which funnily enough could have been useful)
• For plotting I used gonum’s plot package. It’s fine. Kinda. I does sort of a job sort of well, but it’s clearly not built as a total matplotlib replacement

The Feature Creep

At some point, prior to writing the plotting code, I planned out a way for this tool to be bigger and better and eventually enough that it would be running on my homelab (still in progress) so I could update it at anytime. Once the plotting began, it was very clear that these new goals couldn’t be achieved. Additionally, once I started seeing some of the results, I realised that updating it all the time wasn’t particularly valuable.

And the feature creep went away. Possibly to return at a later date, but likely not.

Results

The whole purpose of this project was basically for this section, so strap in. I picked a few queries that I thought to help elucidate my working habits and maybe show them in a new light. This “few” ultimately turned out to be six, since things like max working time per year’ is boring, while queries that utilise 5+ dimensions were not suitable (again, I’ll explain later).

This data consists of 3 complete years of tracked data with 833 working days. A couple of interesting analyses had to be thrown out from the start since they were too hard to separate from noise. These are things like “how often would I return after a headache and for how long” or “how did a public holiday affect my work week”.

Firstly, lets look at the average working time per weekday.

It’s not very interesting BUT it contradicts my thoughts. Monday through Thursday I basically work just over 8 hours a day. How novel. Friday’s I uhhh kick off a bit early, but not too early. I was surprised at how both how not much over 8 hours the four days are but also how practically identical they are. Internally, I feel that Monday and Thursday are my longest working days. This says otherwise, kinda.

Secondly, let’s expand this a bit and see how it looks per the three years of data I have.

My Mondays and Wednesdays across the 3 years are basically the same. Tuesday and Thursdays differ a little more but only by a few minutes. I do not know why my Tuesdays in 2023 were so much longer but whatever. Friday’s on the other hand. In 2022, I would stop on average just under 7 hours of work (6.8). The rise by an hour in 2024 makes sense as the number of responsibilities I had increased dramatically (and there were quite a lot of 15-1600 meetings due to timezone differences).

Between these two charts, it appears that I work basically the amount of time I’m paid for (plus a little more). Huh.

Thirdly, and still thinking of working time, let’s look at how the weather affects my working time.

“Rain rain go away, come again another day” seems to an adage that would work here. Except for when it’s heavy. The most confusing one here is ‘partly cloudy’ but I suspect that is due to only 2.9% of entries sharing that weather code. But basically aside from those rainy days, it’s always roughly 8 hours a day. Yawn.

Ok, now lets look at some starting and finishing times. Things are a little more interesting but thanks to a couple of outliers, the results are very boring. Starting with the earliest time I started work per week day per year.

I should really take out anything before 6am as those entries are insomnia driven. But if you squint real hard you can see a trend that during the course of the week I start earlier and earlier. What about the latest time I started work per week day per year?

Some more outliers that could have been scrubbed (or maybe plotted in a much different way), but it’s not very informative either way. But at least there are not 2200+ ending times. I will say though in 2022 I was much stricter about my working habits. Seems like I’ve just given up since (again LOTS more responsibilities).

In hindsight, these as line graphs is just wrong and I should have used something wayy more appropriate.

Finally, we come to the noise graph that I was hoping would show something at least a bit interesting. This is working duration and daylight duration per year.

Absolutely ZERO correlation. Again, if you squint you MIGHT see a slight dip during summer (oh I wonder why) but ultimately the amount of daylight in a day doesn’t really affect my working habits.

What’s the big takeaway from all these charts? I work enough, maybe a tiny bit too much. The weather and sun doesn’t affect it, and sometimes I’ll push hard to get things done. These conclusions I basically already knew, albeit I don’t work as long as I thought I did.

Post-Mortem

The absolute main thing from this project was: use the right tool for the right job. Everything up until plotting made sense. Trying to make interesting and informative plots in Go is not easy. It might also be not possible for the data I have. But I don’t want to spend more time trying to find out. Were I to do this project again (or maybe when I revisit this in January 2026) I would do all the analysis in Python, because doing that stuff there is dead simple and the range of tools available for you is just astounding. Gonum’s plot is a great package and the work put in for it is amazing, but it’s for a very specific need that just wasn’t mine.

My coming to the conclusion of this project was basically due to the awkwardness in plotting. I did leave out one query I thought was interesting, namely “after an extended break, how long did it take to return working at full capacity” but after thinking how to do it in what I had built was just a non-starter.

What Did I Learn?

• More Golang stuff. Generics are much more painful in Go than I thought
  • Structuring a Golang project
• Dealing with well documented public APIs. I don’t get to do that with work (they aren’t well documented)
• WMO codes are kinda cool but my god what a messy standard
• I’m still bad a choosing the right plot for the right data. I can partially blame gonum for that I guess but still
• USE THE RIGHT TOOL FOR THE RIGHT JOB!

If you’re interested the code is here. It’s not that great but it might be of interest.

And oh my, did ‘Lutris’ deliver. For my experience, it was like 5 steps:

1. Download the latest Scrivener EXE
2. Install it using ‘Lutris’ (the + in the top-left of the window)
3. Configure it to point to the executable: lutrisScrivenerDir/drive_c/Program Files/Scrivener3/Scrivener.exe
4. Delete the textToSpeech directory in lutrisScrivenerDir/drive_c/Program Files/Scrivener3
5. Run Scrivener

The weird step here is deleting the textToSpeech directory but that tip came from this forum post.

The next steps are going to be harder, or so I thought. I store my Scrivener files on my Dropbox. I already use Maestral to have fully-functional Dropbox on my Linux machine and have my .scrivxs stored there. Turns out, the file-picker in WINEd-Scrivener actually exposes root so, all I really had to do was point where I wanted and done.

Now I can actually use Scrivener on all my machines mainly thanks to Lutris doing all the heavy lifting.

(I’m aware that this isn’t really a ‘dev’ thing but I thought maybe it could help some others)

Having 0 idea where to start, I did some digging and found almost exactly what I wanted: a ’time to read’ snippet from https://github.com/Froodooo/bear-plugins/blob/main/bear/reading-time.js. Ideally, this is intended to live on each post and give the reader some info of how long it’d take to read a post. But in the guts of this there is what I need: a way to get the word count, and a way to display it on the page.

I made a few changes to the original script and ended up with the following:

<script> 
if (document.getElementsByName("body_content").length > 0) {
  const wordCount = document.getElementsByName("body_content")[0].textContent.trim().split(/\s+/).length;
  document
    .querySelector("main")
    .insertBefore(
      document.body.appendChild(
        Object.assign(
          document.createElement("p"), {
          className: "word-count",
          innerHTML: `Word Count: ${wordCount}`
        })),
      document.querySelector("main").childNodes[parseInt(document.currentScript.getAttribute("data-before-child") ?? 4)]);
}
</script>

This has some flaws though:

• It’s not particularly accurate. But that’s not important
• It’s static. This is the main drawback with it, is that it requires a page refresh to update. I did play around with KeyboardEvents for quite a while and got something almost working, but decided that what I have is good enough
• I have no idea if it’s good JS. Since I made only a few changes to the original, I’m relying on the possible fact that Froodooo writes good JS. Well, if I get an angry email from Herman, I’ll know something’s up.

But so far it’s been great and quite informative. It’s also given me some new ideas of other scripts that I could do (spoiler: check the email button below).

I took quiet a break from the self-hosting stuff mainly due to reorganising everything and work getting mega busy from late 2023 to late 2024. So what’s it look like now?

• k3s instead of docker, because its so much easier to manage
  • tried FluxCD but managed to break it almost immediately, so switched to ArgoCD which I know very well
• Server is now running Proxmox with 2 AlmaLinux nodes. Server is struggling a bit and will probably need to be replaced :(
• Outside access is still missing mainly due to the new need to tunnel to a remote server and expose via that. Why? Security basically

Most of the same goals apply really. Having a Wiki to store long-term useful info. CI for Codeberg. Media hosting and so on and so forth. I don’t really want to make the whole list here but instead talk about little bits and pieces, because hoo boy there sure are a lot of little problems everywhere.

Let’s start with the latest one: filesystem consistency. One goal I have is that I can sync my Bandcamp library and allow uploading of music to a central place, which is then accessible either via browser or SubSonic API. Additionally, it requires the ability to be backed up and accessible as ‘regular’ files. The way this was done:

• Create a local-path PersistentVolume to a specific path (e.g. /mnt/storage/music) on a specific node
• Force a PVC to use this PV (extremely easy surprisingly)
• Have a CronJob run the ‘Bandcamp Sync’ tool periodically and save into that PV
• Music server uses that PV as it’s storage
• Backup script on VM host copies the dir to backup storage

This actually worked for a little while, say 2 or 3 days. Until one day I tried to ls the dir and was met with reading directory '.': Bad message. Initially terrifying but straight forward to fix thankfully. Except the fix lost+found‘ed the entire ‘music’ dir, which is fine but I don’t want to have to run fsck every few days. For now I have scrapped the music stuff and now I need to find the culprit since there seems to be nothing online about this type of issue.

But what are the potential culprits? Let’s go from bottom up:

• The disk/server is shot (unlikely)
• The disk being written to by both Proxmox and a VM caused issues
• Treating part of the disk as both a Kubernetes PersistentVolume and a regular volume led to various inconsistencies
• K3S’s local-path StorageClass has issues with the VM disk setup

I am not great at fixing Linux filesystem issues so I don’t really know but to me any inconsistency in data is brought about by either bad concurrent writes or misunderstanding of the destination. There’s a lot of slow testing here but thankfully it’s quite easy to do: just run the ‘Bandcamp Sync’ for a few hours then ls the target dir.

There’s a pair of ’easy’ solutions in my mind but they make either backing up harder or create too much overhead for my tiny 9 y/o server. Thankfully, part of me wanted to heavily document this homelab set up this time, so once I fix this I can write up the solution (iff I remember).

Unfortunately, I still have 3 or 4 more other problems beyond this that also need to be tackled (including one really annoying physical one) but this I think is the most important as it’s one of the key features I want in my homelab.

• some wiki (wikijs, dokuwiki or mycorrhiza)
• commento for adding a comment section to my personal blog
• jenkins so i can run ci from my codeberg (more about that later) [but also maybe woodpecker instead]
• some tools to help my partner out with video stuff
• either wallabag or shaarli for bookmark management (i’m a little annoyed with raindop.io)
• and probably something to replace this gh pages stuff

the yunohost image seems like it’d be easy to get most of this going. although i have seen a comment that suggests it does some silly things with systemd

some form of vanilla linux with docker or even k3s/microk8s would also be fine

but why?

i deal with too many services and a lot of the nice replacements tend to be require self-hosting. i also want to reduce my dependence on obsidian and also be able to access it at work.

i could also add some fedi* stuff but we’ll see. that stuffs expensive on the ol’ bandwidth

the missing thing?

backups. if i do any of this, i need solid ways to back everything up. i will make a post about that (assuming i ever even get around to all this)

very quick sketch

i thought it best to also sketch it out. there’s a lot missing (volumes, server services) and kinda wrong (the ‘vpn’ section points the wrong ways), but this is the bulk of my idea. i guess i finally have a project with a defined goal? nice

randomly thought of a good name for this: lockelhost

but an even better idea for the domain name

Until next time, where I’ll have this posting setup either fixed (i.e. not written in a Github window) or entirely replaced (goodbye self-hosted?)

0 update on the game. Since work came back this year, it’s been a bit tricky to work on it. I’ve been focusing on getting my routines and schedules and relaxing back on track. That’s not to say I haven’t done any outside-of-work coding.

My problem:

I exercise on a stationary bike. I like to track my heartrate, and my pace. Pace on a stationary bike is a bit weird but it’s a metric I use to see if body is getting more used to it (I go for long periods without exercising). And I like to track that on Strava so when I do actual rides, I can better compare.

Solution:

Smart-watch. Tracks heart rate and exercise specifics. Pace is missing but the good, important stuff is there. But I really want to record my distance to calculate my pace.

Can I enter my distance in somewhere? For Polar, yes you can. Great! Oh, an allergic reaction means you have to try a dozen different watch bands. Thanks to an underwhelming 3rd party market, there aren’t any straps that are big enough for my watch.

Hrmm, which smartwatch does have that? Fitbit! Cool, back to Fitbit </s>. Oh it now has “Spinning” as a workout option.

Can I enter my distance? No. Hrmm, what if I used the “Ride” workout? No.

So how can I get this into Strava? Oh, I can have Fitbit automatically upload exercise data to Strava.

Oh why isn’t it working? “Spinning” doesn’t record a heart-rate so Strava won’t recognise it, and “Ride”, since I’m stationary, doesn’t record a distance.

The (actual) problem:

When I was using Polar, my process would involve: sync watch, load Polar dashboard on browser, open recorded exercise, for lap 1 alter the distance to match what I did, export exercise as a TCX file, upload to Strava.

But Fitbit, I can’t do this, but I can export an exercise as a TCX file. So, that’s what I did and I was horrified. If your activity is set to “Spinning”, the TCX file only records the start time and the end time. However, if it’s set to “Ride”, it exports a segment every 5 seconds with the current heart rate, but a distance of 0. Oh easy right, lets just use “Ride” and manually add the distance to the TCX file.

Not so. Strava calculates distance based on the total distance accumulated from each segment. So if I rode for 30 minutes, I would need to alter 360 segments and do something like distance/(time*60/5) for each segment. That’s too annoying, especially by hand. But there is a field called DistanceMeters I can use. Oh nope. Since “Ride” also records lat/lon, Strava will think there’s distance in that segment if it sees the lat/lon. Easy, remove all the lat/lon and segment distances.

And that’s the solution: Use the “Ride” workout, scrub all the lat/lon and distances from the segments, and add the total distance to the parent DistanceMeters. Of course, this isn’t that easy to do. But still, tcxa (TCX Anonymiser) exists and does just that.

tcxa.exe -i input-file.tcx -o output-file.tcx -d distanceInMetres

It’s in Rust, it’s ugly as hell, but it does exactly what I want.

Next step:

This is too ugly. My next version of this will instead call the Fitbit API, get the HR and exercise data, ask for the distance, and output a TCX to upload to Strava (ideally all in a browser). Next step from there is remove the manual upload, but that’s going to be a long time. This way I can stop using “Ride” and instead use “Spinning”, which means I won’t upload any positional data to Fitbit.

We’ll see when that happens…

The game:

I’ll get back to it soon. I’ve been enjoying playing games a bit too much as well, but that has been good I think.

Until next time, where my head will be mashed by trying to do something wacky with K8S.

Since last time, we now have:

• Cutscenes
• Rotation
• > 1 height tiles!
• Level importing/exporting
• A level editor!

In addition to these things, I’ve also spent a bunch of time in spreadsheets figuring out rules and stats. We’re actually getting somewhere which is nice. I won’t write much about the tile height stuff. Just that it was a pain because they kept drawing in insane locations. However, I’m still not 100% set on the method I ended up with. It’s likely in the future I’ll investigate the cooler method I was trying which allowed any height and width structures made of tiles. Does height actually matter though in gameplay yet? Nope.

Rotation

This was a biggy, and also kinda simple once I realised I was reading from the wrong matrix… There are 4 rotations now and it is really cool being able to see the battlefield from these angles. There are some issues though e.g. some tiles lose their meaning in other orientations. I’ll figure that out later and decide if it’s really worth keeping.

Cutscenes

They are in a traditional visual novel style. They have their own JSONs as well and I can do an awful lot with them already. I won’t post a picture because it is very ugly with placeholders and flat fills. Maybe soon. The cutscene player has a typewriter mode and automatic text-wrapping so I don’t need to add \n in the dialogue manually (unless I want to). This was pretty easy to do, although I spent lot of time having the text being cut off really early. Spent a good 30 minutes investigating that to only find the wrap value was set to the height of the textbox and not the width. Ooops. I do need to spend some more time with fonts because I just can’t find one I like.

Level editor & importing/exporting

This was borne out of necessity. It actually begun with level generation. I started writing a random level generator because a) infinite levels, and b) I’m not a level designer. After a day¹ or so of work, I began to realise that maybe generation would get me 75% of the way and the rest would need to be tweaked manually. So I wrote an exporter for the levels. It took about 20 minutes and most of that time was spent deliberating how they should be stored. I chose a way that was ~27% more space efficient than the simplest way and still maintains some readability. Considering my maps are currently 20x20 i.e. 400 tiles, it’s still not so readable. Entries kinda look like:

{
  "tile_sprite_id": { "coords": ["x1,y1", "x2,y2"] }
}

At this point I was seriously thinking of editing the points in the JSON. Yes, that was insane. And that’s how the level editor came to be. Again, it was very simple, about an hour of work and I could load in the level JSON, click on each tile to cycle through the available tile sprites and then export (albeit to stdout but still). I guess I’m going to have to learn some level design, or just play more tactics games. I still have a few things I need to consider & implement such as depth, cover and line-of-sight.

I’m back at work as of tomorrow and back to my old routines, so there’ll be a couple of hours work on this a week which kinda sucks, especially since some hard bits are coming up and I can’t just do them in front of the TV. Also actually thinking about narrative again and since this will be part of the universe I set my book in, I need to check my notes.

Until next time (probably in another 2 weeks. Maybe)

We have:

• Units!
  • Animation
  • Movement
  • Turns
• A map..
• Parallax backgrounds
• A bunch of UI gunk
  • Buttons!

At least, that’s what you can see. I’ve spent quite an amount of time working on backend-y things to help once I get the gameplay sorted (or tbh started). Basically, lots of JSON files. For example, here is the main battle scene:

This took some degree of thinking (by which I mean I couldn’t do this with the TV on) and there are JSONs to describe pretty much every aspect of the game and its assets. This actually paid off when I wanted to try a new UI layout with more boxes and text (as is shown in the GIF). But of course I could add hundreds of options to the JSON but I think once the dialogue scenes are done, I’ll have the JSONs’ schemas mostly done.

However, since the last post, two things caused more hassle than anything else: the parallax background and the text.

The parallax backgrounds

Since I long decided (by accident) to consider every draw to be explicitly layer images up images (which very closely matches Ebitengine’s paradigm but I make it worse), I don’t actually have a system that resembles a camera. It’s all just sequential draws that compose an image that is then drawn to the screen. This will soon bite me with the last feature I want to add but I’ll deal with that over Xmas. So, what is the problem here? Well, parallax images are animations layered with each layer playing or moving at different speeds to give the illusion of depth. How do I do it here? I take the background images, tile them so they take up the screen width or height + 1 of the raw tile width/height as a buffer and then tween their draw positions along at different rates. It’s not a good way of doing it but it works really well!

The text

Ebitengine’s text module is actually pretty good and would have been fine for this project. Unfortunately it always looked kind of jank when drawn, even in nearest-neighbour style draw modes. I assumed this to be an issue with the module itself so I started using etxt which is a wonderful library that aims to fix all the typeface and font problems in Ebitengine’s own text module. This garnered me the same result. Perplexed, I did some reading and it turns out this was actually a limitation in the golang image module itself (I have forgotten exactly what though). Solution: bitmap fonts. Ebitengine has a module for that but only with 2 or 3 typefaces. So I built my own. It takes some of the ideas of etxt and jams bitmaps into it. Long way to go before I’m happy with it in this project but if I was to find anything to rip out and make available it would be this.

So what happens now? A lot of things. Gameplay first, then the cutscenes. I know I’m not doing anything ground breaking but I haven’t been this interested in making a game for 9 years and as much as I think golang is a language best not used (I had hoped to do this in Rust), I gelled really well with ebitengine. Maybe my other idea will be in Rust (w/ Bevy). Oh and shaders, I have to properly learn about them now.

Until next time (probably in another 2 weeks)

BUT! In my maybe an hour a day of wrangling with golang and ebitengine I’ve actually gotten somewhere. There has also been a LOT of refactoring to get things to behave nicer and faster¹. Maybe once this goes a bit further I’ll do a quick architecture overview but it’s going to look a lot like normal ECS & “scene-based” game-dev architectures because, well, that’s basically what I’ve been doing². There’s one thing my brain sucks at and that’s geometry. Since I can’t visualise stuff (Aphantasia and all) doing geometric calculations sucks. So I spent a lot of my time refactoring in such a way I would be able to do fewer of them. And I got it down to 2 from 5. Pretty good. But what was that main hurdle:

Yep, looks pretty okay I would say. Highlighting tiles is pretty important in what I have planned (TB/S RPG probably but maybe more). But as you can see, there is still minor hiccough. There’s a LOT still to come (my to-do list³ is ever expanding) but this is a big, big hurdle to get over.

Of course I did make it hard for myself by trying to have the tiles generate their highlighted state automatically but it was very ineffective, but I’m keeping that code for later since it might work in other contexts.

I also have an idea of how this game should sound but we are a looooong way from that.

I guess next week there may be something more, unless I get bored or frustrated.

I wanted to do something in Rust

Then Godot 4 (whenever that is stable)

But now due to…external ideas, go

More specifically ebitengine (terrible name, nice engine with a wacky concept)

We’ll see where this goes, if anywhere

I don’t appear very often, especially here BUT since the last post:

• Got job at Elisa Polystar
• I tried to make a VIC-20 emulator. It kinda worked until trying to get keyboard stuff to behave but you it runs VIC-20 stuff which is nice. Also it was written in Kotlin because work got me on that bandwagon.
• Promoted to senior developer

That’s about it considering the purpose of this blog.

SO SO SO SO SO SO

What am I doing outside of work for dev things? Well, I’m trying to think of a project that uses:

• Rust
• Wasm
• Typescript
• RabbitMQ
• AWS

Apart from that, I have nooooooo idea what to do. Maybe I’ll write and discuss ideas here.

Assuming I remember that here is here.

Until then..

Deploying Twine stories as executables is slightly annoying, especially when working across multiple platforms. In the “Reclaiming Fibres” post, I devised a structure for simplifying this while keeping it logical for VCSs. However, lots of Twine users don’t use any VCS and it’d be easier to use a standalone application to package things for release.

So I see 2 main problems here:

1. How can we make this easy while still being platform and user independent?
2. Interoperability with the Twine2 structure

Problem 1 is where the bulk of this post lies.

I still plan on using electron to package the executables. This requires NPM/NPX to do. As such, writing this in something that depends on NPM/NPX is slightly problematic for the non-technical user. This pushed me to make it in something else, however, the ease of use consideration made this choice easy. I decided on Python as prior experience with GUI development in C++ and Java is painful. A package for Python called “PySimpleGUI” makes this VERY easy. The problem now becomes a design problem.

So, this tool (or set of tools really) has to accomplish certain tasks:

• Set up the Electron app builder directory
• Move all the necessary Twine, HTML, and other files to the correct directory
• Add the correct data to the various JSON files
• Build the app for Platform X
• Build the app for Web

In it’s current state, it treats each new build as an entirely new Electron app. Which means lots of NPM downloads, and it doesn’t alter the Electron config files. My goal is to have this as a single program, with two main functions: 1) setup the Electron app builder directory, 2) build an executable.

Cool. Great. But back up to problem 2. How do we cater for people that only user Twine2? Of course, this shadows another problem but alas. I think the best port of call for this is to rely on the Twine2 user to “Publish to HTML”.

The proposed workflow for a Twine 2 user is as follows:

1. Work on story
2. Publish to HTML to any directory
3. Open my app, select the HTML file, choose a directory to build in
4. Build directory, copy HTML to it
5. Alter settings (if needed)
6. Build executable

In the case the entire story is self-contained as an HTML file, done! But of course, thats not always the case. I fear it may even be very seldom the case.

What happens if you have images and audio included? We now run into a problem that Twine2 handles differently compared to using Tweego and CLI. Paths are absolute compared to relative. In the latter, this is easy (assuming the author is logical) because a common structure is usually followed:

twine_project_dir/
    story.html
    imgs/
        an_image.png
    audio/
        some_audio.mp3

Easy. Copy twine_project_dir to the Electron directory. Twine2 is not so simple. Potential solutions are:

• Parse the story file, copy all the images and audio to local directories, replace the paths
• Make the author conform to the above structure manually
• Prompt the user to move their audio and images to the Electron directory and update their story

None of these options are nice and it will take a while to think this through.

I’ll make a post once I’ve finished this app (currently known as YATE) to see which design decisions I ended up making.

I’ve just spent a week getting acclimatised to Unity (it’s been 7 years since I used it last…) for a project I want finished by the 1st of March.

I don’t feel like divulging too much since otherwise I’ll probably not get on with it, BUT, it involves a lot of audio synthesis and Twine.

Audio synthesis in Unity is…interesting. After a week it makes a LOT more sense, but this is most likely me understanding how Unity -actually- works. Of course, Unity has a HUGE community and audio synthesis tools exist. But as is always the way, the good ones require $$$. Some not-very-thorough digging resulted in some open source synthesis tools¹ and pretty much work as is. “Pretty much” is where it gets funky.

Enter sequencing. I started this by making a clock. That…wasn’t enough. Random dead open source project to the rescue². Now I had audio file sequencing. Useful, but not what I needed. A whole lot of tweaking and testing later I know have a set of sequencers and synthetic instruments. Nothing fantastic, but I’m gradually making it more modular and you can guess where this is going.

BUT NOT YET

I want to do this project (or at least get a proof-of-concept out) before going overboard into software modular synthesis³. A secondary likely splinter project that will appear is expanding Cradle⁴ (the Twine > Unity framework) to do some more things.

Alas, where it stands right now requires more work into thinking how other parts work before blowing up my todo list. So instead I’ll be (attemping to) jam over the next few days.

Why?

So I can print it off and have it take up as much of as A4 page as required.

Now I’ve done it.

Here’s the gist for it: https://gist.github.com/LockeBirdsey/746c0f99b3b75960c8bc9b6d86a0f576

Now I have an infinite calendar that can be printed. Yay?

• pngToICNS
• pngToICO

I have to get testing these things on Windows (and no, not just WSL) rather soon.

So I’m trying to get them back up and working again. And boyyy is it taking a while. So I might run through the steps. It has been a LONG time since I checked out the Twine scene, and as such, I’m rather behind on developments.

From Twine 1 to Twine 2

Fibres was made a long time ago (2012 or 2013). Twine 1 (or more importantly Twee2) was the cutting edge. The story format (Sugarcube) had some limitations, so custom macros were all the rage. Porting from Twine 1 to 2 was easy. A tool that advertised as such could be downloaded and ran. Did it work? Of course not. It was from 2015, what did I expect?¹ Take 2: Maybe Twine’s export to Twee code feature would produce a file that could be imported into Twine 2. Ei/non/no. Twine 2 can only import compatible HTML files. Take 3: Generate the HTML from Twine 1 and import into Twine 2. Ei/non/no. Silent failures on both in browser and downloaded versions of Twine 2 ². What to do?

Well, tweego exists³. And can do some cool things. Like convert between versions of Twee code, but most importantly export to Twine 1 or 2 archives. C’est magnifique!

tweego my-poor-twee-file.tw2 -a -o output-to-twine2.html

Import into Twine 2 and suddenly it appears.
Oh god, the issues.

So those custom macros I mentioned? Well, the JS for them no longer works. Bollocks I hear you say. Never. Twine macros are Story Format dependent. Fibres used Sugarcube, a Twine 1 format. A increased captial version exists for Twine 2 called SugarCube. As it just so wonderfully turns out, my custom macros are now core in SugarCube. Just with different names and parameters. Regex made short work of that.

An interesting issue that was probably exclusive to me was a passage link failing on a number of chapters. Specifically:

[[-->|Passage]]

The documentation for linking is quite straight forward: [[Text|Link]]. So why does --> fail?
Well, how are Twine2 files stored? HTML. And what is a comment in HTML? That’s right <!--comment-->
Easy fix, just replace –> with literally anything else.

And Fibres comes to life. Glorious.

Intermission<blink><marquee>

A Caveat: Twine Version Control

A big surprise to me is that NONE of my Twine projects were in any form of version control outside of Dropbox. Something that needed to be fixed in the middle of all this. Because of how Twine 2 behaves, this is not a straightforward solve.

The main issues:

• Twine 2 runs as an Electron app, and doesn’t include any features for selecting where stories may live.
• Well, having to use Twine 2 since it’s a really nice tool and I don’t want to edit branching stories in Sublime Text.

Stage 1: Get the essentials into a repo.
Twine 1 source file, Twee2 file, and the Tweego converted Twee3 file. Cool. Bananas. A starting point.

Stage 2: Use Twine 2 to edit and update these files Twine 2 does use a /nice/ directory for storing stories which makes this easier, namely $HOME/Documents/Twine
We need to prepare our Twee file for use in it.

tweego Fibres.twee --archive-twine2 --output=$HOME/Documents/Twine/Stories⁴

Now we can open in Twine 2.

*edits*

How can we bring it back and save the changes, both layout and textual, to our VCS of choice?
Not hard.

mv $/Documents/Twine/Stores/Fibres.html our/vcs/repo

And then update our source file.

tweego Fibres.html -d -o Fibres.twee

This is all rather straightforward. There is additional issue that Twine 2 creates: When Fibres archive was opened, Twine 2 saves an archive that uses the StoryTitle parameter. Which in this case wasn’t Fibres but instead Fibres[Part 1]. Oops. Easy fix.

Now we’re all happy. Generate the Twine 2 archive, copy to the Twine 2 story directory, work on the file, copy the file back and decode to Twee3 code. E.A.S.Y

From Twine generated html to an executable in the year of our lord 2020

Hrmm, this was the hard part. Why? Because lots of different tools have lots of little issues that break. I will have nightmares about ENOTDIR for a while to come.
The plan: make an executable of a Twine game to put on itch.io. I’m aware I could deploy for web there (or quite frankly anywhere), but I want distributables because then a JS update won’t come along and cripple it (a la Awaken on GameJolt until recently).

Approach 1: Use someone else’s Twine/HTML game tool.
Simples. There’s a few listed. Some are very old and unmaintained. Some are overkill (one day I’ll try you Cradle). A promising one called HTMLE exists. It however doesn’t work nicely at uses a very old nw.js build. Not ideal.

Approach 2: Do my own NW.js build
This did not look particularly fun and lots of people have moved away from NW.js to…

Approach 3: Electron seems good. Let’s use that
Twine 2 uses Electron, so if it’s good enough for them, it’s good enough for me. Simplified creating of Electron apps is ironically not very simple due to NPM/NPX vs Yarn issues. Three main tools exist: electron-packager, electron-builder, and electron-forge. I tried them in this order. Why did I try 3? Because the first two kept throwing up either issues during build or execution of the final output. electron-forge however worked almost perfectly once I realised that I still had some stray .yarn directores and thus purged them. So now I have a distributible Fibres. Yay! The goal reached!

Ha.

Can we include working audio in this app?
Yep.

Oh good.

electron-forge does require some boilerplate to be saved to make life easier. And thats the only addition to the repository. Excellent. Well, now we have a build script that moves stuff around so that out final executables contain everything and work correctly. Just need to remember to update that packages.json file every so often.

The full workflow.

From an old Twine 1 game to a stable, safe Twine 2 one with distributing

I think it’s worth preserving, at least for my sake, how to do this. I have a LOT of Twine 1 files. There’s a number of tools to be made from this, but I’ll save that for a future bout of hypomania.

How does the directory in version control look now?

	/
	--src
        --audio
        --imgs
        index.html
        Fibres.twee
    --twine1
        Fibres.tws
        Fibres.tw2
    --dist
        --fibres
            -Package.json
            -**other stuff**
        Makefile

Of course, saving audio and images to VCS is an issue because of size, so they currently remain in Dropbox. For lack of a VPS, this must be the way.

Whether or not this is helpful to anyone remains to be seen, but at least it is for me.

The links to the saving graces of the day:

• Tweego https://www.motoslave.net/tweego/
• electron-forge https://github.com/electron-userland/electron-forge

I’ll eventually submit a pull request when I’m happy with it.
It was actually my first time writing in Ruby. But picking up new languages is easy.

I tried to make it as bash friendly as possible. The only weird tool it uses is ffmpeg. I should make a version for mencoder at some point. The gist is here: https://gist.github.com/LockeBirdsey/9566de6787f87079df319c94f6592adc

• Number 1: easy. Simple set of corpora, generate a (mostly) logical sentence and post.
• Number 2: less easy. Download tweets from x accounts, merge tweets, use a markov chain, generate shitposts.

I think I have ideas for more. I have a weird idea for something as well.