Early this year, I started to setup a server at my family’s home, as well as switching our network setup to utilize a firewall, which is running OpnSense.
At that time, I think I spent the entire month, with numerous sleepless nights, to figure out why file transfers over network is so slow. I have tried Nextcloud, Owncloud, PVE, Debian, Arch, even installing Windows Server just to be sure, but it always drops to just 5 MB/s over WiFi, and over cable is just about 12-20 MB/s. Eventually, I gave up since transfers through cable could still reach 12 MB/s, which is still kind of acceptable, although inconvenient as I have to pull a very long LAN cable to connect to the firewall. I thought that it is because my server’s harddisk are not fast enough due to using RAID5, but crap, that’s extremely slow! I have since stick with SMB and arch linux on the server, and connect a very long LAN cable whenever I need to transfer something.
Today, I tried to switch to BBR as these few weeks I’ve been setting up cloud storage using VPS, and also hit with the same issue. People on LET have suggested me to switch to BBR, which fixed the issue on all my VPS. Guess what, after switching to BBR on my home server, I’m able to get 20MB/s SMB transfers over WiFi (limited by 2.4Ghz speeds) and 70MB/s over cable!
Am I being too noob as a new home lab user? (lol) Does all home lab user using Linux for storage, did switch to BBR, or there’s a problem with my network?
Nonetheless, I felt a bit sad because I invested a significant amount of money on VPS to setup my “homelab” on cloud recently, now just to find that I fixed my “local homelab” with just switching to BBR. I thought I could share this with those that face the similar performance issue.
Now I’m stuck with a perfectly working local homelab, and 2 yearly-commitment (promotion) VPS + 1 VPS. RIP me…
There’s some other problem, I have SMB shares saturating 10Gbps links over internally routed networks with no modifications. SMB does degrade quickly as latency goes up but it sounds like you’re saying it’s this slow on your internal network.