Storage @ Work

Ignoring the crucial issue of the best mix of commoditization versus virtualization of the IS (Information Stack) for the moment:
1.Failsafes for running out of physical capacity?
What are some good Strategies for handling this?
If the "Thin Provisioning pool" is dry, what are the options and how are they accomplished?
If you wait until there is less "time to capacity" than "time to adding capacity", even reduced performance capacity, recovery is not
possible unless you can go offline.
How about Online Storage Services such as the one Don MacAskill of
SmugMug talks about on What a web business wants from storage -
StorageMojo.
2.Automated provisioning of new storage that is added to thinly provisioned volumes?
Everyone has entertained this wonderful idea and ignored it for a number of valid reasons.
Mainly they don't trust any part of the process.
Certainly not the Storage vendor.
The CRM has been less capable than it should be.
The ERP is dependent on the CRM which is actually a short-coming of the ERP.
The best is where Knowledge Management (KM) sits at the top of the "automated provisioning chain" and oversees the process.
There is a lot of IT infrastructure required to support the KM, ERP, CRM approach.
There needs to be a "lightweight KM (LKM) + LERP +LCRM" product.
Sort of a "Thin provisioning Storage Management".
3.Support for remote replication with thinly provisioned volumes?
If replication, remote or local, is required, how fast should it be?
Fast or very fast?
1 TB per hour?
10 TB per hour?
Faster?
Instantaneous?
Replication speed gets expensive very quickly.
Would it be cheaper to "Thick Provision"?
4.Performance that is consistent with normal volumes?
The SLA for "Thin Provisioned volumes" should have the minimum acceptable performance defined before attempting to Implement
"Thin Provisioned volumes".
Under all conditions.
The "worst case" is in Case 1. "Failsafe", the "Thin Provisioned volumes" are near or at capacity, is reached.
The point is you cannot wait until there is less "time to capacity" than "time to adding capacity".
Even reduced performance capacity.
You will never recover unless you can go offline.
Going offline may not be an option.
A good way to test this performance is to use an SFO (Search, Find and Obtain) query on ad hoc Information.
Run it repetitively and try to saturate the bandwidth to establish baselines for future reference.

Robert,
My posting attempted to capture comments made by David Scott in an article in Techworld. There was some distortion of his
comments made for the sake of brevity, but I think they were mostly accurate.
I don't agree that the mix of commoditization versus virtualization should be characterized as being "crucial". Please explain what
you mean by crucial - crucial for whom?
Please elaborate.
Failsafes for running out of capacity: We think its pretty important to give customers "cushions" from functions that could be
damaging. Thin provisioninjg is one of those. There are a lot of great things in our world that we deal with that are potentially
dangerous - such as most forms of transportation. There is no question that thinly provisioned volumes could fill faster than it is
possoible to add new storage. There are certainly sceanrios where this could happen, many of them would fall into the category of
administrative error or willful neglect. But does that mean its bad or wrong - or even undesirable. I don't think so. We can learn to
use technology responsibly and as a vendor we have implemented alert/warning thresholds and performance governors. At the end
of the day, the bad stuff that happens when a thinly provisioned volume runs out of space are not much different from the bad stuff
that occurs when a normal volume runs out of space. For thin provisioning, admins need to understand that their capacity might fill
faster than they think - and faster than file system or database systems might indicate (they are being spoofed). That's where the
cushions we provide come in.
Your reference to Amazon's online storage is another way to solve the problem of storage growth. There's nothing wrong with that,
but that kind of service is not really what most people want for their systems and applications. I don't want to suggest that Don
Macaskill or you are "wrong" - but most companies don't want to use an external storage service for their primary data stores.
Automated provisioning: All our customers do this today and integrating it with our thin provisioning is consistent with functionality
that our customers love. No kidding. Provisioning storage does not have to be a frustrating, time consuming exercise. Really. I
don't know what the architecture is under the covers and I don't know what functional components there are and whether or not they
approximate the pieces you believe are essential to acheiving this. I do know that talented engineers often find different ways of
doing things than what might be assumed.
Support for remote replication: I think you are comparing apples and oranges here. Replication performance is a separate
challenge than replication for thinly provisioned volumes. What do you mean by "thick provisioning"? - I'm trying to figure out if this is
a cynical question.
Performance consistency: I mostly agree with what you said , but its a bit of a rehash of the disk full scenarios discussed in point
one and our management tools for dealing with it (cushions). Again, normal volumes can run out of space just as quickly (for
practical purposes) - its a technology that can be reckoned with successfully.
regards - marc