Component Facade without Incomplete Imports

My current hobby Python project is the Vigilant Crypto Snatch. This has grown to a size which made me refactor it a few times by now. Recently I've refactored it towards the clean architecture. I have introduced a two-level module structure such that I have components like configuration, marketplace, telegram and so on.

In a very reduced way, each of these components contains interface classes, concrete implementations and factory functions. They are then used from outside. This is what the dependency graph looks like (made with PlantUML), although I cannot show free functions in UML:

There are a few ways that one can do the import statements in Python. And I have been bitten by cyclic imports and got frustrated about that. When searching the web for cyclic imports in Python, one will find many articles about dumb incarnations of the problem. Somebody really has a cyclic dependency in their components, and then there is no way to get it right. But with Python one can get cyclic imports even when one doesn't have cyclic dependencies. And this is what this post is about.

Direct from imports

One safe way of playing is the following. I'll show the contents of all files in this example now.

File component/interface.py:

print(__file__, "start")

class Interface:
  pass

print(__file__, "end")

File component/concrete.py:

print(__file__, "start")

from component.interface import Interface

class Concrete(Interface):
  pass

print(__file__, "end")

File component/factory.py:

print(__file__, "start")

from component.concrete import Concrete
from component.interface import Interface

def make_instance() -> Interface:
  return Concrete()

print(__file__, "end")

File component/__init__.py:

print(__file__, "start")
print(__file__, "end")

File __main__.py:

print(__file__, "start")

from component.factory import make_instance

make_instance()

print(__file__, "end")

We effectively have an empty component/__init__.py. I have only added these print(__file__) such that we can see the import statements. When we run it, we see that it branches out into the various files and imports them.

 ❯ python __main__.py
__main__.py start
component/__init__.py start
component/__init__.py end
component/factory.py start
component/concrete.py start
component/interface.py start
component/interface.py end
component/concrete.py end
component/factory.py end
__main__.py end

We can also visualize this:

So although we never explicitly import component, the component/__init__.py is run whenever a submodule of component is loaded. This will be the source of complication when we add code to the __init__.py later.

In the component/interface.py I have first imported concrete and then interface. But because concrete imports interface as well, we branch into that. Let us change the order of the imports to this:

from component.interface import Interface
from component.concrete import Concrete

Then the output of the program looks like this:

 ❯ python __main__.py
__main__.py start
component/__init__.py start
component/__init__.py end
component/factory.py start
component/interface.py start
component/interface.py end
component/concrete.py start
component/concrete.py end
component/factory.py end
__main__.py end

This looks a bit cleaner:

In the end, both variants work just fine. This is good, because the imports can then be sorted alphabetically and it is robust. Should the program depends on the order of the imports, it would break sooner or later.

Facade import

The thing that I don't like about this approach is that in the __main__.py I have to import component.factory. I need to know about the structure within the component to use it. I also cannot move classes and functions within the component without changing client code. What I would like to have is a facade such that the code which uses the component only needs to import component. This can be done by adding things to the component/__init__.py. Let's try that!

File component/interface.py:

print(__file__, "start")

class Interface:
  pass

print(__file__, "end")

File component/concrete.py:

print(__file__, "start")

from component import Interface

class Concrete(Interface):
  pass

print(__file__, "end")

File component/factory.py:

print(__file__, "start")

from component import Concrete
from component import Interface

def make_instance() -> Interface:
  return Concrete()

print(__file__, "end")

File component/__init__.py:

print(__file__, "start")

from .concrete import Concrete
from .interface import Interface
from .factory import make_instance

print(__file__, "end")

File __main__.py:

print(__file__, "start")

from component import make_instance

make_instance()

print(__file__, "end")

This then crashes, because it cannot retrieve the Interface from component/__init__.py, but that hasn't been finished loading yet.

❯ python __main__.py
__main__.py start
component/__init__.py start
component/concrete.py start

Traceback (most recent call last):
  File "/run/media/mu/LAUFWERK/Python-Imports/cyclic_import_test/facade-import-2/__main__.py", line 3, in <module>
    from component import make_instance
  File "/run/media/mu/LAUFWERK/Python-Imports/cyclic_import_test/facade-import-2/component/__init__.py", line 3, in <module>
    from .concrete import Concrete
  File "/run/media/mu/LAUFWERK/Python-Imports/cyclic_import_test/facade-import-2/component/concrete.py", line 3, in <module>
    from component import Interface
ImportError: cannot import name 'Interface' from partially initialized module 'component' (most likely due to a circular import) (/run/media/mu/LAUFWERK/Python-Imports/cyclic_import_test/facade-import-2/component/__init__.py)

We can fix this, if we change the order of imports in the components/__init__.py such that it is carefully ordered by the dependencies:

from .interface import Interface
from .concrete import Concrete
from .factory import make_instance

Then it runs through:

❯ python __main__.py
__main__.py start
component/__init__.py start
component/interface.py start
component/interface.py end
component/concrete.py start
component/concrete.py end
component/factory.py start
component/factory.py end
component/__init__.py end
__main__.py end

This looks very clean, the __init__.py imports one module after the other, and they do not need to branch out themselves because everything has already been loaded.

This however feels a bit brittle. Reordering the imports alphabetically will break it. This might be a viable way to go, but I don't like its brittleness.

Facade with from imports within

We can make a compromise by using the same facade, but using direct imports within the component. Then then files look as follows.

File component/interface.py:

print(__file__, "start")

class Interface:
  pass

print(__file__, "end")

File component/concrete.py:

print(__file__, "start")

from component.interface import Interface

class Concrete(Interface):
  pass

print(__file__, "end")

File component/factory.py:

print(__file__, "start")

from component.concrete import Concrete
from component.interface import Interface

def make_instance() -> Interface:
  return Concrete()

print(__file__, "end")

File component/__init__.py:

print(__file__, "start")

from .concrete import Concrete
from .interface import Interface
from .factory import make_instance

print(__file__, "end")

File __main__.py:

print(__file__, "start")

from component import make_instance

make_instance()

print(__file__, "end")

This also works fine:

❯ python __main__.py
__main__.py start
component/__init__.py start
component/concrete.py start
component/interface.py start
component/interface.py end
component/concrete.py end
component/factory.py start
component/factory.py end
component/__init__.py end
__main__.py end

The visualization for that is this:

We can again order the imports by the dependencies and get a more streamlined import traversal:

❯ python __main__.py
__main__.py start
component/__init__.py start
component/interface.py start
component/interface.py end
component/concrete.py start
component/concrete.py end
component/factory.py start
component/factory.py end
component/__init__.py end
__main__.py end

This is the picture that we already had before:

However, this approach does not depend on the order of the import statements, which makes it more robust. I think that I like this the most.

Conclusion

One can have components which are structured within but appear behind a flat facade for the user. In this way one has additional freedoms to refactor the component. One just has to avoid using the facade within the component, otherwise Python will not be able to resolve the imports.