Double Duty for Remote Sites Means Major Benefits

Contrary to prevailing wisdom, increased availability doesn’t necessarily mean increased costs. I’ve met with a number of companies recently that are combining two common technologies to improve their total availability without draining their total IT budget. The secret is to teach your old servers new tricks.

At the top of the availability food chain is complete site recovery — the ability to quickly resume operations at a geographically remote location. The concept is not new, just expensive. It requires a complete datacenter that closely mirrors the primary datacenter at a remote location. If a power outage, natural disaster or other event occurs, operations are quickly migrated to the standby datacenter.

When evaluating disaster recovery options, most companies fall into one of three groups:

1. They need a disaster recovery solution but can’t afford a dedicated standby site.
2. They operate a disaster recovery site, but it’s costing a bundle to maintain idle hardware.
3. They operate a secondary site for other purposes and would love to utilize it for a disaster recovery location.

No matter which of these groups you are a member of, there are ways to get the benefits of a disaster recovery site without doubling your IT budget.

Double Duty for Disaster Recovery Sites

Dedicated disaster recovery sites may someday rescue your company from the unthinkable, but traditional disaster recovery datacenters hum idly away, wasting processing power. As I mentioned earlier, several companies are using new technologies to make their existing servers perform double duty. Essentially, they are combining server provisioning and replication software to quickly change server personalities.

A server that once sat idle waiting for a disaster can now run non-critical applications such as off-host processing, test and development, quality assurance, or other tasks. In the event of a disaster at the primary datacenter, the server is quickly re-provisioned with the personality of a server at the primary datacenter (including the operating system, applications, patches and network settings).

For example, one company I met with recently has hundreds of production servers at its primary datacenter, and a similar hardware environment at a disaster recovery site several hundred miles away. Yet, the servers at this secondary site are not idle; they are currently running human resource applications as well as a test and development environment.

The company uses server provisioning software to capture images of server and application configuration settings from the primary site. These images are then replicated to the disaster recovery site. In the event of a disaster at the primary location, the company utilizes the same server provisioning software to re-image the disaster recovery servers with the applications and settings from the original production site. In couple of hours, this company can change their secondary location from running a few HR applications to a full production environment.

Making it Work

Whenever I propose this solution to other companies, it is usually met with three questions. First, does my secondary datacenter need to be an exact replica of my primary datacenter? Second, how do I get the server images (and the data) to the secondary location? Finally, how long does it take?

The answer to the first question is no, at least it should be no. There are several server provisioning solutions available that will allow you to take an image of a server and deploy it on another identical server. However, only a few have the ability to deploy that image on a server from a different manufacturer with a different configuration. To give yourself full flexibility, make sure you find a server provisioning solution that supports deploying images on dissimilar hardware.

In order to make a dual-use disaster recovery site, you must move the application data as well as the server personality images to the secondary location. How you do this is really up to you. Most customers choose replication technologies because they offer the fastest way to move the data. You can also use tape backup or any other method to move the data to the new site. Replication is typically more expensive, but there are ways to optimize what is replicated (and when) to minimize costs. What’s important is that it gets there.

Once the data and the images are at the secondary location, switching server personalities is a two-step process. First, the server provisioning software rebuilds the servers to look like clones of the production environment. This is where the real magic happens. The main benefit of server provisioning technology is its ability to quickly build dozens of servers simultaneously. The company mentioned above can rebuild 30 servers to look exactly like their production counterparts in less than 10 minutes. But remember that “building” a server is more than just configuring the operating system. Pick a server provisioning solution that is capable of building the operating system, installing applications and configuring network settings. Most server provisioning solutions allow you to automate the entire build process with a single click.

The second step of this process is hooking the servers up to their data, whether it was replicated or sent over on tapes. Usually this is not too painful, but it does require enough storage at the secondary site to accommodate the production data as well as the data that was already there. As you can see, the time it takes to rebuild the entire environment is heavily dependent on how big the environment is. To give you an idea, I’ve seen companies use this approach to re-provision more than 100 servers in about an hour. Your mileage may vary.

Finally, remember the tradeoffs with this approach. Companies that implement this kind of solution have recovery time objectives of several hours. This means that they can afford to be down for three or four hours while the servers at the secondary datacenter are being re-provisioned with the production applications and settings. Several hours doesn’t sound so bad when compared to how long it would take to recover without a disaster recovery site. Regional power outages can last for days (New York in 2003) and natural disasters can render an entire site useless for days or weeks.

Summarizing the Benefits

Disaster recovery doesn’t have to be expensive. If you don’t yet have a disaster recovery site, this approach can protect your company from site outages. If you already have a disaster recovery site, it may be time to put those idle servers to work — without sacrificing their primary role.