Looks like it was some sort of DDoS protection kicking in. But that's a good tip anyway. I've seen very slow SCP transfers in the past on weak CPUs, where changing the cipher sped things up massively.
Yeah, I use SSHFS for letting things see the storage of other things and disabled the encryption as much as possible cos why not on an internal network.
(Tried NFS but found it surprisingly flaky and *way* more complex to set up).
Only played with NFS for one share after SMB was being odd. Always read about using NFS rather than SMB in all scenarios since it's supposed to be much quicker.
Should definitely be better than Samba, yeah. And I liked it apart from it just flaking out every couple of days. Probably my fault but I couldn't be arsed spending much time sorting it out since I don't need 99% of its features - I just want super-easy access to files across the network and sshfs is kinda perfect for that.
How are you mounting stuff? Via fstab?
I remember years ago having various issues with network shares dropping out mounted using fstab and having to do "mount -a" far too often. Then I moved to autofs and it fixed all issues. If you're already using it then no idea, if you're not it might be worth playing with.
Setup was fine, it just needed to be restarted every few days which was a pain. Couldn't be arsed to dig into it since it was overkill for what I needed anyway.