Updated Python Tooling

As I start to do more Python than before, it is time to update my tool chain. This is a listing of the current state.

During my Master thesis, I have written C++. During the PhD I worked with C++, R and the Wolfram Language. Python wasn't really promonent at that time. At work I do both C++ and Python, and for Vigilant Crypto Snatch I also use Python. Before I started more work on the hobby project, I looked into modern Python tooling and found a series called hypermodern Python. I want to share my current Python toolchain.

Although I have been using Vim for quite a while, using a modern IDE at work has convinced me to make the jump. So at home I now use PyCharm Community Edition. It is amazing how much static code analysis that tool delivers, and how much easier development is. I wouldn't want to go back to a plain text editor.

For dependency management I used to just have a read-me file stating the packages. I sometimes used a setup.py, but I am not sure whether any of my projects ever had a requirements.txt. Now I use Poetry, which neatly handles all the dependencies for me, it also builds the whole Python package.

Command line interfaces can be done with Python's built-in argparse library. Some people prefer Click, but I still prefer argparse.

For plotting, I have used ggplot in R for the last couple of years. The declarative grammar for visualizations is just amazing. With Python I had used Matplotlib, as that seems to be the canonical plotting library. I just found that it lacks this declarative approach that ggplot has. Seaborn appears to fill the gap, but it just doesn't. So I was super happy when I found Altair, which has an amazing declarative approach and also produces interactive plots. This is really great in combination with Jupyter Lab.

Creating interactive notebooks with Jupyter is nice for developers, but doesn't help users. They benefit a lot from Steamlit, which allows me to quickly build interactive web interface with Python and present data to the user.

I used to do documentation with something in the code, like Sphinx, until I had my documentation epipany. Using Sphinx one can create hand-written documentation. But I found that MkDocs, specifically Material for MkDocs is really amazing and produces beautiful documentation very quickly from a bunch of Markdown files.

For testing, I used the unittest module from the Python standard library. Pytest allows to write tests in even easier ways using lots of introspection. This way one doesn't have to remember assertion macros, but just uses assert. There is no need for fixture classes, one can just write generator functions and they get linked up by argument names automatically.

Python fortunately has an official style guide, PEP 8. Most IDEs have some formatting, but it just isn't as great as Clang-Format. I have looked at YAPF, which is pretty good. But Black really nails it. There are no configuration options, it just formats the code in a really sensible way.

For logging I use the logging module from Python, with the added library coloredlogs, which makes the output really colorful and much easier to read.

All of this together makes developing Python much better, and it was great to start with.