I’m technically just over 5 W/TB usable at present, so that’s what I checked. Raw is 3.3 W/TB; but most of it is 5/8ths usable to raw.
Setup is a small home ceph cluster, on older hardware. 4x Supermicro Fattwin nodes, with a single 2650V4 each (though they have a second socket that is empty), and up to 8 drives. These draw about 50W each, empty. 1X Supermicro 3U, with 2x2678v3’s, which draws about 150w empty, but holds 16 disks.
The biggest thing driving down the power per TB is that the machines are only half full with disks. I’ve got 4 disks per node in the fattwins, and about half full on the 3U. This is because Ceph lets me add disks as I go - every time storage gets over 85% utilized, I pick up whatever disk is cheapest; usually 2nd hand enterprise disks (since a failure and replacement doesn’t cost me data, and not usually much time.)
Fully loaded the power per TB would be easily twice as good. Newer hardware could also see some big gains, but the 3U is a machine I got almost 5 years ago, and the 4 nodes and the case they go in was such a deal that it’s still saving me money over other, less power-hungry options I could have tried.
Wattage also includes the 10g network switch that Ceph really prefers.
I’m technically just over 5 W/TB usable at present, so that’s what I checked. Raw is 3.3 W/TB; but most of it is 5/8ths usable to raw.
Setup is a small home ceph cluster, on older hardware. 4x Supermicro Fattwin nodes, with a single 2650V4 each (though they have a second socket that is empty), and up to 8 drives. These draw about 50W each, empty. 1X Supermicro 3U, with 2x2678v3’s, which draws about 150w empty, but holds 16 disks.
The biggest thing driving down the power per TB is that the machines are only half full with disks. I’ve got 4 disks per node in the fattwins, and about half full on the 3U. This is because Ceph lets me add disks as I go - every time storage gets over 85% utilized, I pick up whatever disk is cheapest; usually 2nd hand enterprise disks (since a failure and replacement doesn’t cost me data, and not usually much time.)
Fully loaded the power per TB would be easily twice as good. Newer hardware could also see some big gains, but the 3U is a machine I got almost 5 years ago, and the 4 nodes and the case they go in was such a deal that it’s still saving me money over other, less power-hungry options I could have tried.
Wattage also includes the 10g network switch that Ceph really prefers.