I write a lot about technologies that I found hard to learn about. A while back my friend Sumana asked me an interesting question – why are these things so hard to learn about? Why do they seem so mysterious?

For example, take DNS. We’ve been using DNS since the 80s (for more than 35 years!). It’s used in every website on the internet. And it’s pretty stable – in a lot of ways, it works the exact same way it did 30 years ago.

But it took me YEARS to figure out how to confidently debug DNS issues, and I’ve seen a lot of other programmers struggle with debugging DNS problems as well. So what’s going on?

Here are a couple of thoughts about why learning to troubleshoot DNS problems is hard.

(I’m not going to explain DNS very much in this post, see Implement DNS in a Weekend or my DNS blog posts for more about how DNS works)

it’s not because DNS is super hard

When I finally learned how to troubleshoot DNS problems, my reaction was “what, that was it???? that’s not that hard!“. I felt a little bit cheated! I could explain to you everything that I found confusing about DNS in a few hours.

So – if DNS is not all that complicated, why did it take me so many years to figure out how to troubleshoot pretty basic DNS issues (like “my domain doesn’t resolve even though I’ve set it up correctly” or “dig and my browser have different DNS results, why?“)?

And I wasn’t alone in finding DNS hard to learn! I’ve talked to a lot of smart friends who are very experienced programmers about DNS of the years, and many of them either:

• didn’t feel comfortable making simple DNS changes to their websites
• or were confused about basic facts about how DNS works (like that records are pulled and not pushed)
• or did understand DNS basics pretty well, but had the some of the same knowledge gaps that I’d struggled with (negative caching and the details of how dig and your browser do DNS queries differently)

So if we’re all struggling with the same things about DNS, what’s going on? Why is it so hard to learn for so many people?

Here are some ideas.

a lot of the system is hidden

When you make a DNS request on your computer, the basic story is:

1. your computer makes a request to a server called resolver
2. the resolver checks its cache, and makes requests to some other servers called authoritative nameservers

Here are some things you don’t see:

• the resolver’s cache. What’s in there?
• which library code on your computer is making the DNS request (is it libc getaddrinfo? if so, is it the getaddrinfo from glibc, or musl, or apple? is it your browser’s DNS code? is it a different custom DNS implementation?). All of these options behave slightly differently and have different configuration, approaches to caching, available features, etc. For example musl DNS didn’t support TCP until early 2023.
• the conversation between the resolver and the authoritative nameservers. I think a lot of DNS issues would be SO simple to understand if you could magically get a trace of exactly which authoritative nameservers were queried downstream during your request, and what they said. (like, what if you could run dig +debug google.com and it gave you a bunch of extra debugging information?)

dealing with hidden systems

A couple of ideas for how to deal with hidden systems

• just teaching people what the hidden systems are makes a huge difference. For a long time I had no idea that my computer had many different DNS libraries that were used in different situations and I was confused about this for literally years. This is a big part of my approach.
• with Mess With DNS we tried out this “fishbowl” approach where it shows you some parts of the system (the conversation with the resolver and the authoritative nameserver) that are normally hidden
• I feel like it would be extremely cool to extend DNS to include a “debugging information” section. (edit: it looks like this already exists! It’s called Extended DNS Errors, or EDE, and tools are slowly adding support for it.

Extended DNS Errors seem cool

Extended DNS Errors are a new way for DNS servers to provide extra debugging information in DNS response. Here’s an example of what that looks like:

$ dig @8.8.8.8 xjwudh.com
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 39830
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
; EDE: 12 (NSEC Missing): (Invalid denial of existence of xjwudh.com/a)
;; QUESTION SECTION:
;xjwudh.com.			IN	A

;; AUTHORITY SECTION:
com.			900	IN	SOA	a.gtld-servers.net. nstld.verisign-grs.com. 1690634120 1800 900 604800 86400

;; Query time: 92 msec
;; SERVER: 8.8.8.8#53(8.8.8.8) (UDP)
;; WHEN: Sat Jul 29 08:35:45 EDT 2023
;; MSG SIZE  rcvd: 161

Here I’ve requested a nonexistent domain, and I got the extended error EDE: 12 (NSEC Missing): (Invalid denial of existence of xjwudh.com/a). I’m not sure what that means (it’s some DNSSEC Thing), but it’s cool to see an extra debug message like that.

I did have to install a newer version of dig to get the above to work.

confusing tools

Even though a lot of DNS stuff is hidden, there are a lot of ways to figure out what’s going on by using dig.

For example, you can use dig +norecurse to figure out if a given DNS resolver has a particular record in its cache. 8.8.8.8 seems to return a SERVFAIL response if the response isn’t cached.

here’s what that looks like for google.com

$ dig +norecurse  @8.8.8.8 google.com
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11653
;; flags: qr ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;google.com.			IN	A

;; ANSWER SECTION:
google.com.		21	IN	A	172.217.4.206

;; Query time: 57 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Jul 28 10:50:45 EDT 2023
;; MSG SIZE  rcvd: 55

and for homestarrunner.com:

$ dig +norecurse  @8.8.8.8 homestarrunner.com
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 55777
;; flags: qr ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;homestarrunner.com.		IN	A

;; Query time: 52 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Fri Jul 28 10:51:01 EDT 2023
;; MSG SIZE  rcvd: 47

Here you can see we got a normal NOERROR response for google.com (which is in 8.8.8.8’s cache) but a SERVFAIL for homestarrunner.com (which isn’t). This doesn’t mean there’s no DNS record homestarrunner.com (there is!), it’s just not cached).

But this output is really confusing to read if you’re not used to it! Here are a few things that I think are weird about it:

1. the headings are weird (there’s ->>HEADER<<-, flags:, OPT PSEUDOSECTION:, QUESTION SECTION:, ANSWER SECTION:)
2. the spacing is weird (why is the no newline between OPT PSEUDOSECTION and QUESTION SECTION?)
3. MSG SIZE rcvd: 47 is weird (are there other fields in MSG SIZE other than rcvd? what are they?)
4. it says that there’s 1 record in the ADDITIONAL section but doesn’t show it, you have to somehow magically know that the “OPT PSEUDOSECTION” record is actually in the additional section

In general dig’s output has the feeling of a script someone wrote in an adhoc way that grew organically over time and not something that was intentionally designed.

dealing with confusing tools

some ideas for improving on confusing tools:

• explain the output. For example I wrote how to use dig explaining how dig’s output works and how to configure it to give you a shorter output by default
• make new, more friendly tools. For example for DNS there’s dog and doggo and my dns lookup tool. I think these are really cool but personally I don’t use them because sometimes I want to do something a little more advanced (like using +norecurse) and as far as I can tell neither dog nor doggo support +norecurse. I’d rather use 1 tool for everything, so I stick to dig. Replacing the breadth of functionality of dig is a huge undertaking.
• make dig’s output a little more friendly. If I were better at C programming, I might try to write a dig pull request that adds a +human flag to dig that formats the long form output in a more structured and readable way, maybe something like this:

$ dig +human +norecurse  @8.8.8.8 google.com 
HEADER:
  opcode: QUERY
  status: NOERROR
  id: 11653
  flags: qr ra
  records: QUESTION: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

QUESTION SECTION:
  google.com.			IN	A

ANSWER SECTION:
  google.com.		21	IN	A	172.217.4.206
  
ADDITIONAL SECTION:
  EDNS: version: 0, flags:; udp: 512

EXTRA INFO:
  Time: Fri Jul 28 10:51:01 EDT 2023
  Elapsed: 52 msec
  Server: 8.8.8.8:53
  Protocol: UDP
  Response size: 47 bytes

This makes the structure of the DNS response more clear – there’s the header, the question, the answer, and the additional section.

And it’s not “dumbed down” or anything! It’s the exact same information, just formatted in a more structured way. My biggest frustration with alternative DNS tools that they often remove information in the name of clarity. And though there’s definitely a place for those tools, I want to see all the information! I just want it to be presented clearly.

We’ve learned a lot about how to design more user friendly command line tools in the last 40 years and I think it would be cool to apply some of that knowledge to some of our older crustier tools.

dig +yaml

One quick note on dig: newer versions of dig do have a +yaml output format which feels a little clearer to me, though it’s too verbose for my taste (a pretty simple DNS response doesn’t fit on my screen)

weird gotchas

DNS has some weird stuff that’s relatively common to run into, but pretty hard to learn about if nobody tells you what’s going on. A few examples (there are more in some ways DNS can break:

• negative caching! (which I talk about in this talk) It took me probably 5 years to realize that I shouldn’t visit a domain that doesn’t have a DNS record yet, because then the nonexistence of that record will be cached, and it gets cached for HOURS, and it’s really annoying.
• differences in getaddrinfo implementations: until early 2023, musl didn’t support TCP DNS
• resolvers that ignore TTLs: if you set a TTL on your DNS records (like “5 minutes”), some resolvers will ignore those TTLs completely and cache the records for longer, like maybe 24 hours instead
• if you configure nginx wrong (like this), it’ll cache DNS records forever.
• how ndots can make your Kubernetes DNS slow

dealing with weird gotchas

I don’t have as good answers here as I would like to, but knowledge about weird gotchas is extremely hard won (again, it took me years to figure out negative caching!) and it feels very silly to me that people have to rediscover them for themselves over and over and over again.

A few ideas:

• It’s incredibly helpful when people call out gotchas when explaining a topic. For example (leaving DNS for a moment), Josh Comeau’s Flexbox intro explains this minimum size gotcha which I ran into SO MANY times for several years before finally finding an explanation of what was going on.
• I’d love to see more community collections of common gotchas. For bash, shellcheck is an incredible collection of bash gotchas.

One tricky thing about documenting DNS gotchas is that different people are going to run into different gotchas – if you’re just configuring DNS for your personal domain once every 3 years, you’re probably going to run into different gotchas than someone who administrates DNS for a domain with heavy traffic.

A couple of more quick reasons:

infrequent exposure

A lot of people only deal with DNS extremely infrequently. And of course if you only touch DNS every 3 years it’s going to be harder to learn!

I think cheat sheets (like “here are the steps to changing your nameservers”) can really help with this.

it’s hard to experiment with

DNS can be scary to experiment with – you don’t want to mess up your domain. We built Mess With DNS to make this one a little easier.

that’s all for now

I’d love to hear other thoughts about what makes DNS (or your favourite mysterious technology) hard to learn.

Hello! Here’s a new entry in the “cool software julia likes” section.

A little while ago I started using a Mac, and one of my biggest frustrations with it is that often I need to run Linux-specific software. For example, the nginx playground I posted about the other day only works on Linux because it uses Linux namespaces (via bubblewrap) to sandbox nginx. And I’m working on another playground right now that uses bubblewrap too.

This post is very short, it’s just to say that Lima seems nice and much simpler to get started with than Vagrant.

enter Lima!

I was complaining about this to a friend, and they mentioned Lima, which stands for Linux on Mac. I’d heard of colima (another way to run Linux containers on Mac), but I hadn’t realized that Lima also just lets you run VMs.

It was surprisingly simple to set up. I just had to:

1. Install Lima (I did nix-env -iA nixpkgs.lima but you can also install it with brew install lima)
2. Run limactl start default to start the VM
3. Run lima to get a shell

That’s it! By default it mounts your home directory as read-only inside the VM

There’s a config file in ~/.lima/default/lima.yaml, but I haven’t needed to change it yet.

some nice things about Lima

Some things I appreciate about Lima (as opposed to Vagrant which I’ve used in the past and found kind of frustrating) are:

1. it provides a default config
2. it automatically downloads a Ubuntu 22.04 image to use in the VM (which is what I would have probably picked anyway)
3. it mounts my entire home directory inside the VM, which I really like as a default choice (it feels very seamless)

I think the paradigm of “I have a single chaotic global Linux VM which I use for all my projects” might work better for me than super carefully configured per-project VMs. Though I’m sure that you can have carefully configured per-project VMs with Lima too if you want, I’m just only using the default VM.

problem 1: I don’t know how to mount directories read-write

I wanted to have my entire home directory mounted read-only, but have some subdirectories (like ~/work/nginx-playground) mounted read-write. I did some research and here’s what I found:

• a comment on this github issue says that you can use mountType: “virtiofs” and vmType: “vz” to mount subdirectories of your home directory read-write
• the Lima version packaged in nix 23.05 doesn’t seem to support vmType: vz (though I could be wrong about this)

Maybe I’ll figure out how to mount directories read-write later, I’m not too bothered by working around it for now.

problem 2: networking

I’m trying to set up some weird networking stuff (this tun/tap setup) in Lima and while it appeared to work at first, actually the tun network device seems to be unreliable in a weird way for reasons I don’t understand.

Another weird Lima networking thing: here’s what gets printed out when I ping a machine:

$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
ping: Warning: time of day goes back (-7148662230695168869us), taking countermeasures
ping: Warning: time of day goes back (-7148662230695168680us), taking countermeasures
64 bytes from 8.8.8.8: icmp_seq=0 ttl=255 time=0.000 ms
wrong data byte #16 should be 0x10 but was 0x0
#16	0 6 0 1 6c 55 ad 64 0 0 0 0 72 95 9 0 0 0 0 0 10 11 12 13 14 15 16 17 18 19 1a 1b
#48	1c 1d 1e 1f 20 21 22 23
ping: Warning: time of day goes back (-6518721232815721329us), taking countermeasures
64 bytes from 8.8.8.8: icmp_seq=0 ttl=255 time=0.000 ms (DUP!)
wrong data byte #16 should be 0x10 but was 0x0
#16	0 6 0 2 6d 55 ad 64 0 0 0 0 2f 9d 9 0 0 0 0 0 10 11 12 13 14 15 16 17 18 19 1a 1b
#48	1c 1d 1e 1f 20 21 22 23
ping: Warning: time of day goes back (-4844789546316441458us), taking countermeasures
64 bytes from 8.8.8.8: icmp_seq=0 ttl=255 time=0.000 ms (DUP!)
wrong data byte #16 should be 0x10 but was 0x0
#16	0 6 0 3 6e 55 ad 64 0 0 0 0 69 b3 9 0 0 0 0 0 10 11 12 13 14 15 16 17 18 19 1a 1b
#48	1c 1d 1e 1f 20 21 22 23
ping: Warning: time of day goes back (-3834857329877608539us), taking countermeasures
64 bytes from 8.8.8.8: icmp_seq=0 ttl=255 time=0.000 ms (DUP!)
wrong data byte #16 should be 0x10 but was 0x0
#16	0 6 0 4 6f 55 ad 64 0 0 0 0 6c c0 9 0 0 0 0 0 10 11 12 13 14 15 16 17 18 19 1a 1b
#48	1c 1d 1e 1f 20 21 22 23
ping: Warning: time of day goes back (-2395394298978302982us), taking countermeasures
64 bytes from 8.8.8.8: icmp_seq=0 ttl=255 time=0.000 ms (DUP!)
wrong data byte #16 should be 0x10 but was 0x0
#16	0 6 0 5 70 55 ad 64 0 0 0 0 65 d3 9 0 0 0 0 0 10 11 12 13 14 15 16 17 18 19 1a 1b
#48	1c 1d 1e 1f 20 21 22 23

This seems to be a known issue with ICMP.

why not use containers?

I wanted a VM and not a Linux container because:

1. the playground runs on a VM in production, not in a container, and generally it’s easier to develop in a similar environment to production
2. all of my playgrounds use Linux namespaces, and I don’t know how to create a namespace inside a container. Probably you can but I don’t feel like figuring it out and it seems like an unnecessary distraction.
3. on Mac you need to run containers inside a Linux VM anyway, so I’d rather use a VM directly and not introduce another unnecessary layer

OrbStack seems nice too

After I wrote this, a bunch of people commented to say that OrbStack is great. I was struggling with the networking in Lima (like I mentioned above) so I tried out OrbStack and the network does seem to be better.

ping acts normally, unlike in Lima:

$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=113 time=19.8 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=113 time=15.9 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=113 time=23.1 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=113 time=22.7 ms

The setup steps for OrbStack are:

1. Download OrbStack from the website
2. In the GUI, create a VM
3. Run orb
4. That’s it

So it seems equally simple to set up.

that’s all!

Some other notes:

• It looks like Lima works on Linux too
• a bunch of people on Mastodon also said colima (built on top of Lima) is a nice Docker alternative on Mac for running Linux containers

Hello! In 2021 I released a small playground for testing nginx configurations called nginx playground. There’s a blog post about it here.

This is an extremely short post to say that at the time I didn’t make it open source, but I am making it open source now. It’s not a lot of code but maybe it’ll be interesting to someone, and maybe someone will even build on it to make more playgrounds! I’d love to see an HAProxy playground or something in a similar vein.

Here’s the github repo. The frontend is in static/ and the backend is in api/. The README is mostly an extended apology for the developer experience and note that the project is unmaintained. But I did test that the build instructions work!

why didn’t I open source this before?

I’m not very good at open source. Some of the problems I have with open sourcing things are:

• I dislike (and am very bad at) maintaining open source projects – I usually ignore basically all feature requests and most bug reports and then feel bad about it. I handed off maintainership to both of the open source projects that I started (rbspy and rust-bcc) to other people who are doing a MUCH better job than I ever did.
• Sometimes the developer experience for the project is pretty bad
• Sometimes there’s configuration in the project (like the fly.toml or the analytics I have set up) which don’t really make sense for other people to copy

new approach: don’t pretend I’m going to improve it

In the past I’ve had some kind of belief that I’m going to improve the problems with my code later. But I haven’t touched this project in more than a year and I think it’s unlikely I’m going to go back to it unless it breaks in some dramatic way.

So instead of pretending I’m going to improve things, I decided to just:

• tell people in the README that the project is unmaintained
• write down all the security caveats I know about
• test the build instructions I wrote to make sure that they work (on a fresh machine, even!)
• explain (but do not fix!!) some of the messy parts of the project

that’s all!

Maybe I will open source more of my tiny projects in the future, we’ll see! Thanks to Sumana Harihareswara for helping me think through this.

Hello! On Wednesday, we released a new zine: How Integers and Floats Work!

You can get it for $12 here: https://wizardzines.com/zines/integers-floats, or get an 13-pack of all my zines here.

Here’s the cover:

the table of contents

Here’s the table of contents!

Now let’s talk about some of the motivations for writing this zine!

motivation 1: demystify binary

I wrote this zine because I used to find binary data really impenetrable. There are all these 0s and 1s! What does it mean?

But if you look at any binary file format, most of it is integers! For example, if you look at the DNS parsing in Implement DNS in a Weekend, it’s all about encoding and decoding a bunch of integers (plus some ASCII strings, which arguably are also arrays of integers).

So I think that learning how integers work in depth is a really nice way to get started with understanding binary file formats. The zine also talks about some other tricks for encoding binary data into integers with binary operations and bit flags.

motivation 2: explain floating point

The second motivation was to explain floating point. Floating point is pretty weird! (see [examples of floating point problems]() for a very long list)

And almost all explanations of floating point I’ve read have been really math and notation heavy in a way that I find pretty unpleasant and confusing, even though I love math more than most people (I did a pure math degree) and am pretty good at it.

We spent weeks working on a clearer explanation of floating point with minimal math jargon and lots of pictures and I think we got there. Here’s one example page, on the floating point number line:

it comes with a playground: memory spy!

One of my favourite ways to learn about how my computer represents things in memory has been to use a debugger to look at the memory of a real program.

But C debuggers like gdb are pretty hard to use at first! So Marie and I made a playground called Memory Spy. It runs a C debugger behind the scenes, but it provides a much simpler interface – there are a bunch of very simple example C programs, and you can just click on each line to view how the variable on that line is represented in memory.

Here’s a screenshot:

Memory Spy is inspired by Philip Guo’s great Python Tutor.

float.exposed is great

When doing demos and research for this zine, I found myself reaching for float.exposed a lot to show how numbers are encoded in floating point. It’s by Bartosz Ciechanowski, who has tons of other great visualizations on his site.

I loved it so much that I made a clone called integer.exposed for integers (with permission), so that people could look at integers in a similar way.

some blog posts I wrote along the way

Here are a few blog posts I wrote while thinking about how to write this zine:

• examples of floating point problems
• examples of problems with integers
• some possible reasons for 8-bit bytes

you can get a print copy shipped to you!

There’s always been the option to print the zines yourself on your home printer.

But this time there’s a new option too: you can get a print copy shipped to you! (just click on the “print version” link on this page)

The only caveat is print orders will ship in August – I need to wait for orders to come in to get an idea of how many I should print before sending it to the printer.

people who helped with this zine

I don’t make these zines by myself!

I worked with Marie LeBlanc Flanagan every morning for 5 months to clarify explanations and build memory spy.

The cover is by Vladimir Kašiković, Gersande La Flèche did copy editing, Dolly Lanuza did editing, another friend did technical review.

Stefan Karpinski gave a talk 10 years ago at the Recurse Center (I even blogged about it at the time) which was the first explanation of floating point that ever made any sense to me. He also explained how signed integers work to me in a Mastodon post a few months ago, when I was in the middle of writing the zine.

And finally, I want to thank all the beta readers – 60 of you read the zine and left comments about what was confusing, what was working, and ideas for how to make it better. It made the end product so much better.

thank you

As always: if you’ve bought zines in the past, thank you for all your support over the years. I couldn’t do this without you.

A few years ago I gave a short talk (slides) about myths that discourage people from blogging. I was chatting with a friend about blogging the other day and it made me want to write up that talk as a blog post.

here are the myths:

• myth: you need to be original
• myth: you need to be an expert
• myth: posts need to be 100% correct
• myth: writing boring posts is bad
• myth: you need to explain every concept
• myth: page views matter
• myth: more material is always better
• myth: everyone should blog

myth: you need to be original

This is probably the one I hear the most often – “Someone has written about this before! Who’s going to care about what I have to say?“.

The main way I think about this personally is:

1. identify something I personally have found confusing or interesting
2. write about it

The idea is that if I found it confusing, lots of other people probably did too, even though the information might theoretically be out there on the internet somewhere. Just because there is information on the internet, it doesn’t get magically teleported into people’s brains!

I sometimes store up things that I find confusing for many months or years – for example right now I’m confused about some specific details of how Docker networking works on Mac, but I haven’t figured it out enough to be able to write about it. If I ever figure it out to my satisfaction I’ll probably write a blog post.

Sometimes when I write a blog post, someone will link me to a great existing explanation of the thing that I hadn’t seen. I try to think of this as a good thing – it means that I get a new resource that I couldn’t find, and maybe other people find out about it too. Often I’ll update the blog post to link to it.

A couple of other notes about this one:

• technology changes, and the details matter. Maybe the exact details about how to do something have changed in the last 5 years, and there isn’t much written about the situation in 2023!
• personal stories are really valuable. For example I love my friend Mikkel’s Git is my buddy post about how he uses Git. It’s not the same way that I use it, and I like seeing his approach.

a bit more about my love for personal stories

I think the reason I keep writing these blog posts encouraging people to blog is that I love reading people’s personal stories about how they do stuff with computers, and I want more of them. For example, I started using a Mac recently, and I’ve been very annoyed by the lack of tracing tools like strace.

So I would love to read a story about how someone is using tracing tools to debug on their Mac in 2023! I found one from 2016, but I think the situation with system integrity protection has changed since then and the instructions don’t work for me.

That’s just one example, but there are a million other things on computers that I do not know how to do, where I would love to read 1 person’s story of exactly how they did it in 2023.

myth: you need to be an expert

The second myth is that you need to be an expert in the thing you’re writing about. If you’ve been reading this blog, you probably know that I’ve written a lot of “hey, I just learned this!” posts over the years, where I:

1. Learn an interesting thing (“hey, I didn’t know how gdb works, that’s cool!”)
2. Write a short blog post about what I learned (how does gdb work?)

You actually just need to know 1-2 interesting things that the reader doesn’t. And if you just learned the thing yesterday, it’s certain that lots of other people don’t know it either.

myth: posts need to be 100% correct

I try to my make my posts mostly correct, and I’ve gotten a bit better at that over time.

My main strategy here is to just add qualifiers like “My understanding is..” or “I think..” before statements that I’m not totally sure of. This saves a lot of time fact checking statements that I’m honestly not sure how to fact check most of the time.

Some examples of “I think…s” from my past blog posts:

I think people are replacing “how many golf balls can fit in the Empire State Building” with more concrete [interview] questions about estimating program runtime and space requirements.

I think the most important thing with bridges is to set up the route tables correctly. So far my understanding is that there are 2 route table entries you need to set: …

Etsy uses PHP, which I think means they can’t have long-lived persistent TCP connections

I think the MTU on my local network is 1500 bytes.

I still don’t know if all of those statements are true (is it true that PHP programs can’t have long-lived persistent TCP connections? maybe not!), so the qualifiers are useful. I don’t really know anything about PHP so I don’t have much interest in fact checking that PHP statement – I’m happy to leave it as an “I think” and potentially correct later it if someone tells me it’s wrong.

I do tend to overdo the “I think that…” statements a bit (bad habit!) and sometimes I need to edit them out when actually it’s something I’m 100% sure of.

myth: writing boring posts is bad

The reality of publishing things on the internet is that interesting things get boosted, and boring things get ignored. So people are basically guaranteed to think your posts are much more interesting that they actually are, because they’re more likely to see your interesting posts.

Also it’s hard to guess in advance what people will think is interesting, so I try to not worry too much about predicting that in advance. I really Darius Kazemi’s How I Won The Lottery talk on this topic about how putting things on the internet is like buying lots of lottery tickets, and the best way to “win” is to make a lot of stuff.

myth: you need to explain every concept

It’s common for people writing advanced posts (like “how malloc works”) to try to include very basic definitions for beginners.

The problem is that you end up writing something that feels like it wasn’t written for anyone: beginners will get confused (it’s very hard to bring someone from “I have no idea what memory allocation is” to “in depth notes about the internals of malloc” in a single blog post), and more advanced readers will be bored and put off by the overly basic explanations.

I found that the easiest way to start was to pick one person and write for them.

You can pick a friend, a coworker, or just a past version of yourself. Writing for just 1 person might feel insufficiently general (“what about all the other people??“) but writing that’s easy to understand for 1 person (other than you!) has a good chance of being easy to understand for many other people as well.

writing has gotten harder as I get more experienced

Someone who read this mentioned that they feel like writing has gotten harder as they get more experienced, and I feel the same way.

I think this is because the gap between me and who I’m writing for has gotten a bigger over time, and so it gets a little harder for me to relate to people who know less about the topic. I think on the balance having more experience makes my writing better (I have more perspective!), but it feels harder.

I don’t have any advice to give about this right now. I just want to acknowledge that it’s hard because someone who read a draft of this mentioned it.

myth: page views matter

I’ve looked at page view analytics a lot in my life, and I’ve never really gotten anything out of it. Comments like this one mean a lot more to me:

Hey, @b0rk. Just wanted to let you know that this post really helped me to improve my skill of understanding a complex concept. Thanks! :)

If it helps one person, I figure I’ve won. And probably it helped 10 other people who didn’t say anything too!

myth: more material is always better

I appreciate the work that goes into extremely deep dive blog posts, but honestly they’re not really my thing. I’d rather read something short, learn a couple of new things, and move on.

So that’s how I approach writing as well. I’ll share a couple of interesting things and then leave anything extra for another post. For me this works well because short posts take less time to write.

This one is obviously a personal preference: short posts aren’t “better” either, I just like them more.

But I often see people get tripped up by wanting to include EVERYTHING in their blog post and then never publishing anything and I think it’s worth considering just making the post shorter and publishing it.

some notes on pedantic/annoying comments

Someone who read a draft of this mentioned struggling with comments that are pedantic or annoying or mean or argumentative. That one’s definitely not a myth, I’ve read a lot of comments like that about my work. (as well as a lot more comments where people are being constructive, but those ones aren’t the problem)

A few notes on how I deal with it:

• The “don’t read the comments” advice has never worked for me, for better or for worse. I read all of them.
• I don’t reply to them. Even if they’re wrong. I dislike arguing on the internet and I’m extremely bad at it, so it’s not a good use of my time.
• Sometimes I can learn something new from the comment, and I try to take that as a win, even if the thing is kind of minor or the comment is phrased in a way that I find annoying.
• Sometimes I’ll update the post to fix mistakes.
• I’ve sometimes found it helpful to reinterpret people being mad as people being confused or curious. I wrote a toy DNS resolver once and some of the commenters were upset that I didn’t handle parsing the DNS packet. At the time I thought this was silly (I thought DNS parsing was really straightforward and that it was obvious how to do it) but I realized that maybe the commenters didn’t think it was easy or obvious, and wanted to know how do it. Which makes sense! It’s not obvious! Those comments partly inspired implement DNS in a weekend, which focuses much more heavily on the parsing aspects.

As with everything I don’t think this is the “best” way to deal with pedantic/annoying comments, it’s just what I do.

myth: everyone should blog

I sometimes see advice to the effect of “blogging is great! public speaking is great! everyone should do it! build your Personal Brand!“.

Blogging isn’t for everyone. Tons of amazing developers don’t have blogs or personal websites at all. I write because it’s fun for me and it helps me organize my thoughts.

that’s all for now!

Probably I’ll write another meta post about blogging in a couple of years since apparently that’s what I do :)

Thanks to Ed, Jeff, Brian, Hazem, Zachary, and Miccah for reading a draft of this