Adding CCXT to Vigilant Crypto Snatch

The Vigilant Crypto Snatch software had been refactored into the Clean Architecture a while ago. This process started when we wanted to also support the Kraken marketplace next to the Bitstamp marketplace. Then I switched over from the clikraken package to krakenex to improve the support with Kraken.

Recently somebody mentioned the CCXT package which provides a unified interface to over a hundred of exchanges. Adding this to our software would significantly increase the applicability. So I have looked into it. The license of CCXT is MIT, so it fits perfectly to my code.

Because it promises a unified interface to all markets, it basically does the same as my internal marketplace module. One might consider getting rid of that, and using CCXT directy. But that won't be a good idea. What if they change their API? What if I don't like it? What if they change their license model? And most importantly: Why should I change my other code only because I add a new marketplace library?

Not pythonic

I have looked at the documentation, and quite early one can read that it is primarily a JavaScript library. The Python code seems to be generated from that, and you can already see that the library is not very “pythonic”. One can see that when an instance for an exchange is created. In JavaScript is looks quite ideomatic:

const exchangeId = 'binance'
    , exchangeClass = ccxt[exchangeId]
    , exchange = new exchangeClass ({
        'apiKey': 'YOUR_API_KEY',
        'secret': 'YOUR_SECRET',
    })

The constructor is passed an object (a Python dict), and also the ccxt library object is accessed via [] with a string key.

In Python this looks really peculiar:

exchange_id = 'binance'
exchange_class = getattr(ccxt, exchange_id)
exchange = exchange_class({
    'apiKey': 'YOUR_API_KEY',
    'secret': 'YOUR_SECRET',
})

It is the same code, just translated. But using getattr is a smell that you are not doing it right. Sure, there are applications for getattr, but these should be special implementation details. Having this as the top-level usage of your API is really strange. And then passing a dictionary with arguments to the constructor is okay-ish, but I would rather expect named arguments for more nameing (and potentially type) safety.

I don't find that very appealing. But as long as this just resides in adapter code, I don't care about that too much.

Implementing an adapter

I plan to add a CCXT marketplace to the marketplaces that I already have and expose that to the user. This way I don't have to add each marketplace explicitly, but I can just forward this. This of course breaks encapsulation, but in this regard I might not even want to encapsulate CCXT.

This is what the marketplace looks like at the moment:

I will add a CCXTMarketplace and a CCXTConfig. This configuration will only contain the name of the exchange and an opaque dict with parameters which I pass onto the CCXT library. This is what it looks like in diagram form:

I won't do validation and just let the user use all the flexibility of CCXT. This makes it an extemely narrow API for me, which is great. It forces a burdon on the user to specify the secrets in just the right way, though. This is not such a big deal, as we can quote a few of the examples from the CCXT documentation in ours.

In the new marketplace.ccxt_adapter module, I implement the configuration as a simple data class:

@dataclasses.dataclass()
class CCXTConfig:
    exchange: str
    parameters: dict

And then I start with the CCXTMarketplace. With one precision type hint one can also get the proper code completion in the IDE:

class CCXTMarketplace(Marketplace):
    def __init__(self, config: CCXTConfig):
        exchange_type: Type[ccxt.Exchange] = getattr(ccxt, config.exchange)
        self.exchange = exchange_type(config.parameters)

Now I can implement the other methods that I need to fulfil the Marketplace API. In the documentation the first private one is account balance, so I will try that. It was straightforward to implement into my design:

    def get_balance(self) -> dict:
        response = self.exchange.fetch_balance()
        return response["total"]

But one could do a bit better and write a function to parse the result. Here it is so trivial that I didn't do that.

Placing orders

The documentation on orders appears to be a bit more complicated. There is a currency pair needed, they call it “symbol”. Each exchanges denotes them differently. Examples given are “BTC/USD”. But in another part of the documentation they write that one should not try to parse them. Instead one should rather query them and use the CCXT library to parse that.

In CCXT terms, we have “exchanges” and “markets”. The exchange would for instance be Kraken. This has multiple markets, one of them for each currency pair. So we need to load the markets beforehand. Then we can filter the markets for the symbol. But then it will be easy to place orders:

    def place_order(self, asset_pair: AssetPair, volume_coin: float) -> None:
        self.exchange.create_market_order(
            symbol=get_symbol(self.markets, asset_pair), side="buy", amount=volume_coin
        )

I am not sure about the error handling there, this will come when I try it out. And indeed, it raises a ccxt.base.errors.InvalidOrder, where the exception from Kraken is passed right throuh. The text then reads kraken {"error":["EGeneral:Invalid arguments:volume"]}. I cannot further parse that in the adapter, I can just wrap it as my own exception such that it can be caught in the right location.

Getting prices

We also need to implement the spot price. This is done via price tickers. And it was very easy to wrap as well:

    def get_spot_price(self, asset_pair: AssetPair, now: datetime.datetime) -> Price:
        response: dict = self.exchange.fetch_ticker(
            get_symbol(self.markets, asset_pair)
        )
        result = Price(timestamp=now, last=response["last"], asset_pair=asset_pair)
        return result

Withdrawal

The withdrawal feature is a bit harder, it seems. I need to know the fees for withdrawal, otherwise I cannot decide whether it makes sense to withdraw assets. For the time being I will just leave that open.

Wrapping up

In order to get this new feature out to the users, I will have to update the documentation. I have also moved the --marketplace option from the command line into the config. This way there are no command line flags left, and everything is handled in the configuration file.

This new feature was really easy to implement because I already had a clear interface to implement. I just had to fill in the gaps and it was a matter of a few hours. This is the beauty of the “clean architecture”, where changes with a large impact to the user (now supporting over 100 exchanges) is done without changing much in the code.

This will soon be released in version 5.5.0.