Since I retired Mediaserver 8.1 and Mediaserver 8.2 in favour of Mediaserver 8.3, I've had a spacious ThermalTake Core X9 case and AMD FX-8320 based motherboard sitting around. I also have a selection of hard drives ranging from 500GB to 4TB that are retired from my main unRAID system. I'd been hankering to turn all of this into a backup server and, in fact, had made a few attempts that never really made it past proof of concept. It was time to attack the project seriously....
These are installed in the case in ThermalTake drive cages, 5x 3.5" cages and even 6 hanging from the roof fans in handy dandy brackets of unknown origin. The Core X9 is huge, but this all fills it out quite well.
Initially I had a few issues getting this all to work. I had some extra hard drives and a PCIe SATA card installed, but things were unstable. The pair of LSIs seemingly didn't want to work together, or drives dropped offline or the system crashed.
I surmised that I was exceeding the capacity of the Corsair RM750X power supply. I backed off a few drives as well as the SATA expansion and all was well with the world. I didn't want to splash out on a new PSU as the idea here is to build from leftovers and redundant parts, so I'm happy to sacrifice a few TBs.
Next up, I installed StableBit Drivepool. I've had a licence for this for a while now, and used it in the few abortive attempts I made previously at this project. I could of course have just set up another unRAID server, but I really wanted to experiment with different approaches, and I really do like StableBit a lot. It's very straightforward to use, and has some of the features I like about unRAID such as support for different drive sizes in a pool as well as file and folder level duplication (or triplication if required).
I set up all my drives in a single pool with file duplication, which will give me an effective capacity of 17.5 TB. I did it like this as these are old drives and I see in the accompanying StableBit Drive Scanner that many have age related SMART errors.
My intention is to use this system to maintain a duplicate backup of some important shares on my unRAID server. Of course, a backup needs to run on reliable hardware, but I'm happy with this setup where I have my main copy of data parity protected on the unRAID server, and the backup copy duplicated on the Backup Server.
During my setup and tribulations with disks going offline, I had occasion to observe how StableBit DrivePool behaved when things go wrong. I was impressed with how transparent and straightforward it is, and that helped me build confidence that even in a worst case scenario, short of complete hardware destruction of both systems, I'd have a good chance of retrieving a substantial portion of my data. And that's good enough for my needs. I will, of course, run the two systems in separate buildings as an additional precaution.
Anyway, with my pool set up, I started transferring some sample data from unRAID to the drive pool. Both systems have 10G network adapters (ASUS XG-C100C) with latest firmware updates, routed through 10G ports on a Netgear GS110EMX switch.
Initial tests showed network speeds just clipping 1G and write speeds peaking at 120 MB/s, but often lower. This was a little disappointing, but I realised that the bottleneck seemed to be the spindle based drives on each side. I did an SSD transfer between the machines and that was noticeably faster. In the spirit of making do with what I had to had, though, I was happy to accept a degree of sluggishness on a backup system.
I wasn't too happy with conducting backups through windows file transfer, however, so I started casting about for something that could be automated, and hit upon Kopia. I'd first come across this a few weeks ago in the context of an unRAID docker container, and liked the sound of it. A scheduled backup to repositories that could exist on local storage or in the cloud. What's not to like?
I thought about setting that docker up, and having my unRAID essentially push data to my backup server. However, this backup system is power hungry, idling at just under 300W and hitting high 300s under load. I don't want it running 24/7. The idea is to schedule start it a couple of times a week, run Kopia to update backups, and then shut down. With that approach in mind, I though it best to install Kopia on the backup server itself, so I wouldn't need to deal with issues of an unRAID side docker having trouble finding a target location that may or may not have booted up, for some reason.
Kopia was relatively easy to set up via GUI. There was a bit of reading around terminology etc. Once I got going, it ran sweetly.
It works by setting up a repository, in my case on my local DrivePool, into which it saved snapshots. It's reminiscent of Apple TimeMachine in how it works - first take all the files, then just add difference over time. The files saved on the DrivePool are not really organised according to the original file structure, but instead into a series of directories that makes sense to Kopi, with all repository browsing and file retrieval done through the Kopia application.
And it's fast. It managed to index and backup 1,7 TB of data in the same time as a simple Windows file transfer copied across about 300GBs. Impressive.
In the first pass, it generated 110 errors, indicating network paths were not found. It was very easy to drill down into the folder structure to see where these problems were. Turned out, they were deep inside Final Cut project folders, themselves nested deep into a parent file structure. I figured there was something up with exceeding windows file path lengths. When I moved that folder to root level on the source share, thereby shortening the path, the files were picked up OK. And the great thing was Kopia figured out just the additional files it needed, so no long transfer times.
Overall, I'm happy with the performance of this, particularly at zero cost as everything here is on its second life. My next step is to configure the periodic boot of the system, have Kopia step in on start-up and run a snapshot, and then shutdown the server when done. As belt and braces, I might also set up a second repo in a cloud service and have it make additional copies of critical files there, just in case.
No comments:
Post a Comment