The What...
The used Scalar i2000 I ordered is almost here and the used Scalar i6000 is pending shipment.
The Why...
tl;dr I'd rather not have to change tapes out once a month and store+organize 400 LTO-6 cartridges.
Right now I have two tape libraries. One is a small 16 slot LTO-6 based SuperLoader 3 and the other is a 45 slot IBM TS3200 with one LTO-6 drive. The big one handles Linux ISO backups and the other handles all other tape backups. Plus Stor22 has a few TB dedicated to file based backups for some common, small files too.
Since I have over 400 LTO-6 cartridges right now I have to rotate them through every so often. That interval is usually around 1 month give or take two weeks. It's annoying and, let's face it, I'm not good about keeping up with it.
One of the bigger problems with so much tape is storing and organizing the tape that's not in one of the libraries. It's even harder when you factor in Bacula sometimes wanting an odd tape every so often rather than what's in the loader. This can happen for various reasons including me not loading ones that are all expired or not loading ones that expired the longest time ago. Pulling 60 tapes one by one from storage and keeping them all organized and safe sucks.
An Overview...
- Move into garage
- Unpack
- Test in garage on skid
- Split into two rack-sized halfs
- Move into house
- Move UPS & battery expansion module
- Move to final location
- Reassemble
- Retest
The Plan...Into The Garage...
The first major issue with both units is getting it here. Shipping on each unit, which is the size of two full racks side-by-side, and that weights 1,100 lbs to 1,500 lbs is a bit pricey. Plus the driver and myself will need to be able to get it into or near the garage using a pallet jack. This is hard since the driveway from the road to the garage is inclined.
It's gotta fit through the garage door too. I'm not positive if the door is up height enough for the skid+rack+packing on a pallet jack to fit under the garage door. Worst case it can sit right outside the garage door, I can remove it from the skid, and hopefully roll it into the garage. If that fails I might have to split the two "racks" apart and make a few quick emergency calls to friends to get enough hands to move the units inside.
The Plan...Unpacking...
Once it's in the garage it'll have to unpack it and get it off the skid. Hopefully it'll roll off as it's wheeled. At worst I'll probably try making a makeshift ramp outa some angle aluminum and plywood. Or maybe it'll stay on the skid until I'm done with initial testing. In that case some friends and I can move it from the skid into the house once it's split apart. Keeping it on the skid would make it easier to get inside as there is a 1' step from the garage floor to the bottom of the door frame. The downside here is that the skid would need to be by the door. So in all likelihood what'll actually happen is that it'll stay on the skid until some friends are over to help move it. At that point we can move each half off the skid and into the house. A ramp may be required. If so heavy plywood, 2x4s, and/or angle aluminum will be needed. Regardless of the exact details, getting these 600+ lbs "racks" into the house will be difficult.
The Plan...Initial Testing...
An initial round of testing will be done in the garage before splitting the control module and the expansion module. The goal is mainly to see how many slots are licensed, how functional the unit is, and in the case of the i2000 to see if the LTO-6 FC drive I got refurbished will work.
Depending on how things go with the i2000 it may be necessary to wait for the i6000 to arrive before splitting it and moving it inside. This is because, depending on space on the 3rd floor, I may opt to put one in one of the spare bedrooms. Although given the size, weight, and the fact that Jeff's using both, I'm leaning against this plan. The other reason to wait is so that the two units can be compared to see which will be a better "fit" for my use case. The better one will end up in my server area. The other may sit in the garage, in my workshop, get put on the 3rd floor, or get moved somewhere else.
As an interesting aside, my plan if I do need to move one of these units up/down stairs is to hire piano movers or the like. They are experienced with moving very heavy, large, semi-delicate items. This makes them a very good fit. I'm guessing the cost for my house (based on a friend who works for a piano company) would be between $400 and $800. Maybe as much as $1000 since there are two long flights of steps on top of it not being what they normally move. Assuming they even think they could move it safely. Where "safely" means both their safety and that of the tape library.
Power for testing may require a bit of effort. The standard PDUs for the units want L6-30s. Although the ones I purchased may have a different plug. According to the docs the i2000 will require less than 6A max at 120VAC. So I'll use my L6-30 to C19 adapters and a NEMA-5 to C20 plus an extension cord to get power to the i2000. The i6000 may require something more complex as the number of drives in it means that it may need more power than I'd want to run over a regular extension cord. Once they're in a permanent location proper power will be run.
The Plan...The Split...
The real interesting part of this whole operation will be splitting the unit in half. Both the i2000 and i6000 I ordered consist of a control module (size of one rack) joined with one expansion module (also the size of a rack). There isn't a lot of information on how to do this in the documentation that I've found. The docs do say how to disconnect the two X-axis bars from each other. That's probably the only non-obvious part. Although any wiring between the two does concern me a bit.
Once inside the house they'll need to be reassembled. Getting the X-Axis bars reconnected, any wiring, and the frames will be required. The difficulty level here is not known yet. Although splitting them should be a good indicator. On the plus side I'll have the other unit nearby for reference if I need.
The Plan...Electrical...
The required electrical for the units will be annoying in the short term and expensive in the long term. The short term plan will be to use a C19/C20 from two different PDUs in my existing racks to an L6-30. Assuming of course that the units have L6-30s on them. Since the i2000 and i6000 both require less than 10A @ 240VAC (probably under 5A @ 240VAC realistically) using the C19/C20s shouldn't be a problem. They're rated for 20A @ 240VAC after all.
The long term plan is to have my usual electricians, Absolute Electric, install two new outlets. Doing so will require a new breaker box for the UPS. Plus there is some other minor work I need done anyway such as wiring up some lights and a few 120v outlets in various spots in the workshop. All of the infra will need to go down for a day for this work as the UPS would need to be shutdown.
One advantage of having this done is that I might be able to have them put in an automatic transfer switch for the AC. That way it could run off of line power normally. If the power goes out for more than a few seconds it could switch over to UPS. That way I don't need to worry about keeping a second portable AC on standby.
Side note - Absolute Electric is awesome. They've been great about working with me over the years on my...unusual...electrical needs. Not just anyone would be willing and able to help install an APC SY16K (a rack sized 16kVA UPS). Plus all of the associated electrical that goes along with having a mini-data center. It's even all been county inspected. They're great folks to work with.
The Plan...Fibre Channel...
For many years I've been avoiding FC. The only real reason besides not needing it was that I already had 10GbE and 40Gbps Infiniband. But the addition of these units will require some basic FC infra. To that end a pair of C9148s with 32 port licenses each have been purchase along with a number of 4Gbps and 8Gbps quad port FC HBAs.
The general plan here is to connect one (or possibly both) tape libraries to both switches. Any LTO-5 and LTO-6 drives will be directly connected to one or both switches, depending on licensing (path failover is a seperate license) and the number of physical ports on the drives. Any drives LTO-4 and below will use the FC line cards that are part of the libraries. The libraries FC cards and management cards then being linked to both C9148s.
If licensing allows there will be full path failover for all drives. If not, then failover will only be on the host side and sorta on the C9148s - One of the C9148s will end up with the 'good' FC port for a given set of drives. Thus if that C9148 goes down then all of the drives off of it without path failover would become inaccessible.
The Plan...Configuration...
This will be in flux until the final hardware configuration and licensing is known. My current thinking is that if both libraries are up and running then I'll use one with a small pool of LTO-6 and SDLT (lol, yeah, really) for non-Linux-ISO backups and uber-critical file backups (financial docs, bacula database, etc) respectively. The other would have the majority of the LTO-6 and would handle Linux ISO backups.
If only one is in use then I'll need two (ideally three) LTO-6 drives in it. One for non-Linux-ISO backups, one for Linux-ISO backups, and the third optional drive for restores.
Thus far only one LTO-6 drive has been ordered. They're expensive and hard to find for i2000 and i6000 units. If there are to two libraries than each one will need at least one drive and, ideally, a second for restores. If there is one than the library will need two drives plus an optional third for restores.
I'd have three Bacula pools:
- [LTO-6] Linux-ISO
- [LTO-6] Everything else
- [SDLT] Uber critical files (optional)
This setup mirrors my current environment where I use an IBM TS3200 with an LTO-6 drive for Linux ISO backups and a SuperLoader 3 with an LTO-6 drive for non-Linux-ISO backups. By splitting up these backups I can more effectively recycle tapes. That's due to the "everything else" having a 2y retention period while the Linux ISO backup has a 9mo retention period. If jobs from both were written to one tape then the tape would effectively need to be kept for 2y even if 95% of the data could be trashed after 9 months.
The servers used for the bacula storage daemon (bacula-sd), the ones that will accept connections from the Bacula file daemons (aka clients) and write data from them to tape isn't know yet. I may use some of these old 2U boxes I have, I may use Stor15, or maybe Stor22 will continue to fill this role.
In The End...
In the end, no plan survives first contact with the enemy. I'm really looking forward to the challenge. And not having to change tapes out once a month.