Code Documentation Epiphany

In the past decades I have written a bunch of software. And most was rather ephemeral stuff, like hobby projects or university assignments. In most cases there was no need for any sort of formal documentation. I would understand the code as I am working on it, and once I have abandoned it, it would never be touched again anyway.

During my PhD this has changed. I have worked on three software projects which were meant to be used in the work group after I am gone. And because I knew it from the start, I tried my best to document it. At the time I had the feeling that the documentation should be very close to the code, in the spirit of Donald Knuth's literal programming. I have used tools like Doxygen, Epydoc, Sphinx or JavaDoc to create HTML pages from the docstrings in the code. I only had to write a comment for each function, and then everything would be documented!

In reality I found that this documentation was not really useful. It wasn't motivating to write. The generated website didn't look beautiful by default, though with Sphinx one can find beautiful themes. At the time I didn't really see the systematic problem, I just thought that if I were to write better docstrings, it would become great.

And then for my PhD I needed to write something in Wolfram Mathematica. And as I learned the language, I looked for equivalents of tools that I usually use. There was no documentation generator, no real docstring functionality. I was annoyed by the language. And so I just started out with a Markdown document. And I have documented the functions there, but also the design decisions, use cases, caveats. Something that you can only do in a long-form document. The end result has been a 40 page manual. It has a common theme, has a structure, and you can actually work with that thing.

By not having yet another documentation generator of dubious use for my case, I have finally had the freedom do really dive into long-form documentation. And it was the best document project of the time. The functions in the code had virtually no comments, but they were short and readable functions. They have somewhat sensible names and are hopefully self-explanatory.

For my current hobby project, I therefore have also refrained from using any docstrings. My goal is to have small functions with expressive names. Grouped into modules in logical way. Function arguments with clear names and meaningful type annotations. And all the documentation will be in Markdown documents. This makes the documentation much more worth reading. I haven't written much about the inner workings of the code there, but I might do so at some point.

And I do the same at work. I have completely refrained from using a documentation generator and I also do not write docstrings. First I write high-level documentation as that seems to be much more important in a complex code. Once I have this, and I am able to isolate a few small classes, I might add a few docstrings with very technical details.

In the end I am really happy to have separate high-level documentation with screenshots, flow charts and links on one side, and clean code on the other side.