SpiderOak One Performance Issues

As I wrote earlier, I have been using the SpiderOak One Backup service for a while now. It costs more than the usage of Duplicity with Backblaze B2, but it promises to work better. Also it creates backups continiously, so I don't have to wait for the full hour to have a backup. Using inotify it will automatically put files into the queue just as I save them. I really like this. My laptop could burst in flames and I would only lose a few minutes of work.

But the performance of the system is bad, again and again. I recall that I tried the service a few years ago and had just exactly the same issues with it. Often it would not utilize my upload bandwith (10 Mbit/s) fully but rather utilize one full CPU core, which seems excessive. I would restart the application and then it would stop this. Often this is triggered by using the GUI. I have the feeling that the GUI cannot really cope with ten thousand of files being listed, so it just freezes up as it tries to display it all.

Lately even restarting did not help. So I wrote to the support on 2020-05-23 about this issue. I got a reply telling me to move the ~/.config/SpiderOakONE directory somewhere else and sign back into the program. Then it would do the syndication again and it should work fine. At the time it seemed to work fine again after a few simple restarts, so I did not follow through on this at that time.

Contacting support

Then last week I had the issue again, it would just not upload anything. I asked the support on Twitter and they said that they had some issues with the servers, but it should be good again. On Monday I experienced troubles again, it would just not upload. So I chose the option to delete the upload queue in the program. After a restart of the software I saw the syndication screen.

This process took some time, and that is fine. It eventually got that it had to upload a bunch of files that where changed since the last backup. The progess was still super poor, and would only hog the CPU but not do any actual uploading. Then I tried the advice from the support and deleted the ~/.config/SpiderOakONE directory. After a restart I had the same syndication process, so apparently I had already done that using the built-in functionality.

After that went through, upload progress was super slow again. On the left you can see the queue of files that are to be uploaded. The program packs a few of them into an archive and sends that in one go. On the right you can see the four virtual CPU cores, on the bottom there is the network usage. There is a short break in the CPU load, then it uploads something for a few seconds and then stalls again, using the CPU. This pattern repeats itself again and again. Trying to quit the program will leave a CPU bound background process which does not react to the term signal. I have to send the kill signal to get rid of it.

I have watched this for a few hours, and you can see the time of day, remaining files to upload and remaining size to upload in the following table. One can see that some files did get uploaded, but changes to files are piling up faster than the software makes to trickle to the servers.

This cannot be blamed on my overall connection either. As you can see below in the readout from my modem, the upload is nowhere nearly saturated, and most of it comes from a video conference I was attending via my mobile phone.

At this rate the backup service is useless to me. I want a timely backup of my data, and I don't want to have a whole CPU core blocked from the software.

Contacting support again

On 2020-07-28 I have again contacted the support team and pointed them to a draft version of this blog post. They have replied saying that they don't see any other option than to delete all data and to register my laptop as a new device, deleting all old data on the server to free the quota.

This feels like a last-resort thing, and the support does not really seem to dig into the problem. There should be options to look at log files, view telemetry or something like that. They could store this on the device locally, this would not be a privacy issue. Just deleting everything, reinstalling does not seem like an intelligent measure, rather brute force.

I have still done it, I was short of canceling the service anyway. When I logged into the application again and registered my laptop under a new name, it would still download all the meta data from the old device. I have no idea what the point of this was, but it still took lots of time and CPU power.

After it has set itself up again, there were no files selected to be backed up. This is great, as the software should then have nothing to do. I first tasked it to delete my old files such that I would not have to download the meta data for them again. This took ages and also tons of CPU power, without any real usage of the network. I was not surprised at this point.

Then the old devices was removed, but the CPU utilization was still up. I have tried to close the application, but no avail. And then I killed it and all the other processes that it had spawned. During restart it would just not come back up, but still use a full CPU core.

I have then again deleted the ~/.config/SpiderOakONE directory in the hope that my old 91 GB of backups would have disappeared. I started the application once more, registered my laptop as a new device: mu-x220-3. It seemed to download a whole lot of data again, but it should have been all deleted!

After a while it was up again, yet it consumed a whole lot of CPU power for doing virtually nothing. I did not have any backup files selected for the new profile, still it was searching for files to back up.

At this point I was rather unhappy about the whole situation and was very close to canceling the service.

Uploading again … for a while

Then to my surprise the load started to go down again, back to a few percent. I selected the files I want to have backed up in order to give it another chance. It started to index these files and used a lot of CPU, but that was fine. And then it started to upload with full speed and using only some CPU resources.

A day later it is still uploading stuff, but already using more power and having less upload speed saturation. It is using around 80 % of a CPU core just to upload with less than 1 MB/s, which I find very frustrating. Perhaps the CPU load comes from compressing the files it uploads, that might explain the load; yet it feels way too much for that.

Apparently I got excited too quickly. The upload has stalled. The CPU load that one can see the in the following picture comes from other tasks, SpiderOak One does not use more than a few percent. The upload is virtually nonexistant.

Perhaps the software does not cope so well with suspending the laptop? Perhaps it gets stuck when the upload is interrupted in this way?

I have contacted support again, but it feels as they don't really have a clue about what is going on either. So I will likely see how it behaves and then just cancel the subscription.

Just for fun I have restarted the application again. After a while it has found all the new files and is now uploading again. The CPU usage is down again. It seems really eratic.

Bulk uploading finished

Over the past days the application has finished uploading all the bulk of my data. Now there are just incremental updates and there have not been any CPU load surges that I had before. I really hope that it stays this way and I can continue to use this service!