The High Price of Data

You’ve purchased servers, storage space, switches, cables, and countless other pieces of hardware. The Oracle licenses are bought and paid for, Enterprise Edition with a few add-ons. All told, you’ve spent a small fortune on this infrastructure. It’s finally time to start up your database and begin using it for projects…

…Data gathers, and now your cost begins.

A Game of Clones

DBAs and other IT professionals (and sometimes naive executives) can fall into the trap of thinking that the up front costs of application building are all you have to consider for an application. To be honest, a lot of IT professionals don’t even think that far–we need components, we buy them. But the costs pile up far beyond the initial purchase.

I’ve worked with hundreds of companies over the years and have seen how expenses related to the database–some hidden and some not so much–can plague a business over its lifetime. Storage, network bandwidth, inadequate server resources, we can all identify with those at some point or another, but they are the tip of the iceberg. The real problems involve data movement.

The Mountain That Rides

Sometimes trying to clone, migrate, or back up a database can feel like trying to hike across the seven kingdoms carrying 500lbs of cargo in the dark. Databases are big and cumbersome. Moving them has become easier and easier by way of SAN utilities; however, cloning, copying, and backing up are still as painful and time consuming as they have ever been. And at the very least, clones (and backups) take large swaths of disk space.

Many of you might be thinking that you never have to make clones so this doesn’t apply to you. Stop right there! While you may not have to clone databases frequently, every production database needs backups. And just as importantly, many businesses don’t ask for clones because they think it can’t be reasonably done. They know the outlook is dark and full of “no’s”.

Instead of thinking that you don’t need to worry about cloning because the business doesn’t ask for it, imagine what you could do if copying and cloning of data was not a problem–or even further, if it required barely any extra disk space or time. Think about where you’re missing out on opportunity due to a lack of ability to quickly provision databases.

Stark Territory

Data mobility creates opportunity. How many of you have suffered under a development database that hasn’t been refreshed in a couple years or tried to test performance improvements or parameter changes on a QA database that is only a subset of your production database? How many of you don’t have a development or test environment at all?

Imagine the kind of tax that puts on a development process. You can’t throw a rock nowadays without hitting some new app that a company (or individual) has released. With the traditional software medium still going strong and cloud apps spreading like wildfire it’s a huge deficiency for an organization not to be able to spin off databases for development, testing, and release as fast as humanly possible.

Slow processes mean a ton of wasted money for your business. If you’re backing up a database, copying the backup, then restoring in order to provision a development database you’re wasting disk space (cheap, but not without cost) and time. I’ve actually worked with many companies that can’t migrate or clone a database simply because they don’t have space on a filesystem to take and copy a backup. Many more companies can do it, but the process takes 3+ days (and in some cases over a week!). That’s days of lost time for development in a business where every hour counts and a missed schedule can mean missed revenue and market share. Let’s also not forget that you and the others doing the work cost money, further increasing the cost. It all adds up to a huge data tax that makes your initial hardware and license investment look like a pittance.

It’s no wonder many organizations have turned to DevOps, other database engines, data center migrations, and even outsourcing to fix the “problems” of overpriced resources and lost time. With data being positioned as a huge business asset in today’s world, a lack of data mobility is disastrous. Being able to easily move data decreases the total cost of data. This cost is monetary as described above, and even that can be dwarfed by a huge opportunity cost. And opportunity is worth more than a kingship these days.

Across the Narrow Sea

Over the last month or so I’ve had an opportunity to work with Delphix, an agile data platform made for data mobility. The software behind the platform sits inside of VMWare ESX Hypervisor and can be used across your data center (or in the cloud or across multiple data centers). My first use of it involved:

• A small(ish) blade server with four quad-core AMD processors and 64GB RAM
• An 8TB Oracle database (located on SAN)
• Delphix configured with disk space and minimum required setup

The 8TB database was added into Delphix as a dSource, which triggers a backup into the Delphix disk area. This is the only full backup that it ever had to take–it will forevermore use incremental snapshots from this point on. That full backup and the incremental snapshots are also compressed within Delphix for further space savings.

Once Delphix has linked up with a source database, you can start picking target systems (setup on these targets is fairly minimal and was done in about an hour) and provisioning. You can provision a clone of the source database from any time period that is retained in the Delphix disk area and make as many clones as the targets and infrastructure can handle. The target databases will not use any space after the clone except for changes made locally on that target. For instance, if you clone a 20TB database to a target system and make only 100MB in block changes, those changes are the only disk overhead required for the target (the block changes themselves are compressed and stored in Delphix).

Once we had set up the Delphix environment we proceeded to provision clones on the same server as the source. We were able to create two clones of this 8TB database in 17 minutes–one from the latest point in time and another of the source as it was 1.5 days prior. Provisioning to a different host yields the same results, as the cloning is done within Delphix so it doesn’t matter time-wise which host is used. Functional QA tests were run using the virtual databases and found no difference in functionality (100% pass) and no difference in performance compared to the source (in this case, your mileage may vary). Two clones of an 8TB Oracle database in 17 minutes…I honestly never thought it possible. One virtual database I refreshed (by reprovisioning the clone to current point in time) took slightly under 5 minutes.

Retaking the Iron Throne

Think of the possibilities with a platform like this. Development and QA databases spun off in minutes with minimal disk use. Reporting databases and datamarts created from any point in time faster than the time it takes to write the reporting queries. Version control that incorporates the source database as part of the branching and merging process. Even backups and disaster recovery can make use of the technology–imagine bringing a multi-terabyte database back online in minutes following a production disaster without having to keep a warm copy or perform a lengthy restore?

The cost of data at this point becomes obsolete. Opportunity abounds due to the ability to quickly adapt to any requirement no matter how big your database is, how many developers need access to the data, or how short the timeline is. I can honestly say it’s one of the first products I’ve ever worked with that perfectly aligns with the requirements of the business: fast access to resources, minimal time requirements, simple management, and no waste.

And it’s pretty fun being able to say “yes, I’ll have the database ready in 10 minutes” as well.

By the way…

If you’re headed to RMOUG 2014 be sure to check out my presentation with Kyle Hailey: Dueling Duplications – Physical vs. Virtual Data Cloning. We will be pitting RMAN against Delphix in an epic tourney, and may the most clones win!