<Cadey> Spoiler alert! This references details for my GopherCon EU talk Reaching the Unix Philosophy's Logical Conclusion with WebAssembly. This may ruin some of the cosmic horror feeling that I tried to inspire in the audience by gradually revealing the moving parts and then crashing them all together into one "oh, oh god, why, no" feeling when the live demo happened. My talk page and the video will go up in August.

One of the fun parts about doing developer relations work is that you get to write and present interesting talks to programming communities. Recently I travled to Berlin to give a talk at GopherCon EU. It was my first time in Berlin and I've enjoyed my time there (more details later). This year at GopherCon EU I gave a talk about WebAssembly. Specifically how to use WebAssembly in new and creative ways by abusing facts about how Unix works. During that talk I covered a lot of the basic ideas of Unix's design (file i/o is device IO, the filesystem is for discovering new files, programs should be filters) and then put all the parts together into a live demo that I don't think was explained as well as I could have done it.

Today I'm going to go into more details about how that live demo worked in ways that I couldn't becaise I was on a time limit for my talk.

<Mara> If you just want to read the code, look here in the /x/ repo.

Dramatis Personae

I glossed over this diagram in the talk, but here's the overall flowchart of all the moving parts in my live demo (for those of you on screen readers, skip this image description because I'm going to explain things in detail):

There's two main components in this demo: yuechu and aiyou (extra credit if you can be the first person to tell me what the origin of those names are). yuechu is an echo server, but it takes all lines of user input and then feeds them into a WebAssembly program. The output of that WebAssembly program is fed back to the user. You can change the behavior that yuechu does by changing the WebAssembly program that it uses as a filter.

aiyou is a WebAssembly runtime that exposes the network as a filesystem. It doesn't really do anything special and doesn't pass through some fundamentally assumed things like command line args and other filesystem mounts. It really just is intended to act as an echo client for my demo. The most exciting part of it is the ConnFS type, which exposes the network as a filesystem.

<Aoi> I get it! If sockets really are files, then the network can be a filesystem, technically!

Otherwise most of this is really boring Rust and Go code. The real exciting part is that it's embedding Rust code into a Go process without having to use the horrors of CGo.

ConnFS

In Wazero, you can mount a filesystem to a WASI program. You can also use one of the Wazero library types to mount multiple filesystems into the same thing, namespaced much like they are in the Linux kernel. In Linux these filesystems are usually either implemented by kernel drivers, or programs that use FUSE to act as a filesystem as far as the kernel cares.

In Go, we have io/fs.FS as a fundamental building block for making things that quack like filesystems do. io/fs is fairly limited in most cases, but the ways that Wazero uses it can make things fun. One of the main ways that io/fs falls over in the real world is that files opened from an io/fs filesystem don't normally have a .Write method exposed.

However, an io/fs file is an interface. In Go, interfaces are views onto types so that you can expose the same API for different backend implementations (writing a file to another file, standard out, and connections). The File interface doesn't immediately look like it has a .Write method, but there's nothing that says there can't be a .Write method under the interface wrapper.

In Wazero, if you have your files implement the .Write call, they will just work. Write calls in WASI will just automagically get fed into your filesystem implementation.

In my talk I said that these methods are common to both sockets and files:

• open()
• close()
• read()
• write()

So you can use this to shim filesystem operations over to network operations. I did exactly this with ConnFS in my demo program aiyou.

<Aoi> Uhhh, isn't this going to fail horriffically in the real world when any network hiccups at all happen? This doesn't seem like it would be the most stable in the long term.

<Cadey> Well, yes this isn't going to work very well in the real world. However, this demo is a really unique case because it connects to localhost (so you don't have to worry about network stability) and only runs for about 30 seconds each run (so you don't have to worry about the philosophical horror of long-lived network connections). In practice, you can use /dev/tcp in bash to do most of the same thing as ConnFS.

<Aoi> Why am I not surprised that bash does something cursed like that.

The server

All that's left in the stack is the echo server, which really is the boring part of this demo. The echo server listens on port 1997 (the significance of this number is an exercise for the reader and definietly not the result of typing a random four digit number that was free on my development box) and every time a connection is accepted it tries to read a line of input from the other side. When it gets a line of input, it runs that through the WebAssembly program and returns the results to the user.

That's about it really.

Programs are like functions for your shell

This lets you use programs as functions. Stdin and flags become args, stdout becomes the result. I go into more detail about this in this talk based on this article. This is something we use at Tailscale for our fediverse bot. Specifically for parsing Mastodon HTML.

So realistically, if you can use something as stupid as the network as a filesystem, you can use anything as a filesystem. The cloud's the limit! But do keep in mind that any complicated abomination of a code-switched mess between Go and Rust can and will have a cost and if you use this ability irresponsibly I retain the right to take that power away from you. Don't ask how I'd do it.

Some pictures

I've never been to Berlin before and I took some time to take pictures with my dslr. I think the results are pretty good. I've attached some of my favorites:

It's been great fun. I'd love to come back to Berlin in the future. I'm considering getting some of the better photos printed and might sign some to send to my patrons. Let me know what you think!

I'm going to upload more of the photos to my blog later, I need to invent a new "photo gallery" feature for my blog engine. I could use something like Instagram or Google Drive for this, but I really like the tactility of having everything on my infrastructure. I'll figure something out.