I saw that Meta released the Llama 3 AI model, and people seem excited about it, so I decided to give it a try.

I don’t have much experience running open-source AI models, and I didn’t see a lot of documentation about how to run them. I tinkered with it for a few hours and got Llama 3 working with Ollama, so I wanted to share my instructions.

Provisioning a cloud server with a GPU

To run this experiment, I provisioned the following server on Scaleway:

Highlights

• I worked with the TinyPilot team to lock down access to deployment secrets without interfering with our workflows.
• I learned from my mistakes to limit downtime when migrating services between platforms.
• I wrote my first compiler, albeit an extremely simple one.

Goal grades

At the start of each month, I declare what I’d like to accomplish. Here’s how I did against those goals:

Seven years ago, I built my first home server. It made my software development work faster and more enjoyable, so I’ve gotten more into the home server scene. I built a custom storage server, another development server, and a dedicated firewall.

At some point, my wife gently observed that my office was filling with unsightly wires. “What?” I asked. “This is a normal amount of wires.” But then I looked around and realized it was kind of a lot of wires…

Highlights

• We completed the first-ever TinyPilot release where I didn’t perform any release task directly.
• Publishing a release through delegation helped identify many undocumented or poorly conceived steps in our release process.
• I’m continuing to enjoy writing a bytecode interpreter in Zig.

Goal grades

At the start of each month, I declare what I’d like to accomplish. Here’s how I did against those goals:

For the past few months, I’ve been curious about two technologies: the Zig programming language and Ethereum cryptocurrency. To learn more about both, I’ve been using Zig to write a bytecode interpreter for the Ethereum Virtual Machine.

Zig is a great language for performance optimization, as it gives you fine-grained control over memory and control flow. To motivate myself, I’ve been benchmarking my Ethereum implementation against the official Go implementation.