It has now been a few years since virtualization technology, specifically x86 virtualization technology, has gone mainstream. What began in the lab is now increasingly in production. Those initial 15 machines have now grown into a full-fledged virtual infrastructure with more than 500 or 5000 virtual machines. One would expect the challenges of managing and growing virtual infrastructures to be addressed by now.  Surprisingly, customers continue to cite sprawl and lack of visibility as chief among their concerns, and configuration consuming hours of their time.

As more and more enterprises grow their virtual environments in line with corporate mandates in order  to reap the financial benefits of virtualization, they are echoing the same concerns that the ‘early adopters' voiced over a year ago. They are now realizing that the benefits of virtualization can rapidly be consumed by administrative nightmares and wasted sprawled resources, if VM management is not addressed.

Why have these challenges lingered? There are three primary reasons:

Without Knowledge, There is No Power. The information needed to manage a well-oiled virtual environment differs from its physical counterpart. Unfortunately, data about the innards of a VM aren't available alongside information about the container. New concepts like lineage, migration, dynamic usage and unintended dependencies are not well reported and understood. Information about storage isn't aligned with the VM either. Change information across the infrastructure is all but absent. To add to the woes, philosophical differences on the part of vendors - from hypervisor-only orientations to waiting-for-market timing - mean that traditional vendors have not been eager to resolve these issues.

It's "On the List." There are no underworked virtual administrators. Many firms still have manual provisioning processes and even more manual troubleshooting. Application owners are concerned that memory on their VM is too low, the VM is lagging, and the infrastructure is at fault. Hours are spent doing manual root cause analysis. In many cases, the virtual environment is perfectly fine - but the burden of proof remains on the virtual administrator. All of this builds a case for a management system that can diagnose and mitigate changes and potential failures. While startups are tackling this challenge, many administrators do not have the time to research these options. It is just another item on their rather lengthy to do list.

Things Aren't Getting Easier.  Environments are growing but they are also growing in complexity.  The hypervisor wars are heating up, so multi-platform environments are looking inevitable. VDI is bringing desktops into the datacenter. Multi-site implementations and cloud resources are providing early benefits but also increasing the management challenges.

However, in the coming year, the seeds of change are emerging, and there are some early indicators that the management challenges will be relieved.

Hooking It Up. When software first emerges, it is rarely the priority of the vendor to develop robust APIs. Some software packages, years after release, are still impossible to integrate. We're seeing talk of integration, standards and APIs from many of the major virtualization and cloud vendors. This means that vendors will be better able to provide visibility and management cross-platform, throughout the virtual environment. If they come to fruition soon, administrators should have the comfort of achieving the promised elasticity while maintaining manageability of their environments.

Efficiency Inflection Point. IT groups so far have been enjoying the benefits of virtualization. However, the economic magic of server consolidation has become the new baseline, and managers are looking to further optimize their environments. Application groups are increasingly aware that virtual resources are not free, and thus are more thoughtful in their consumption. When up to 30% of virtual machines are idle, and many are over-allocated, there is a renewed emphasis to cut costs and make better use of the existing resources.

New Developments from New Firms. While the traditional firms struggle with proprietary platforms and long development cycles, and the fundamental challenges of transformation in large companies, a group of startups are solving these management challenges already.  By creating management tools purpose-built for virtualization, aware of the elasticity of the environment, the mobility of the VMs and the challenge of monitoring and control, these solutions automate key elements of data collection, root cause analysis and simple management. With an emphasis on heterogeneity, an eye towards the real customer needs and innovative approaches, they will enable companies to continue to drive cost reduction and management best practices.

As virtual environments continue to grow, and the average environment is measured in hundreds of virtual machines, the challenges of sprawl and management are hitting a much broader set of customers across many industries. The universal nature of the problem has made its resolution more important. 2009 will bring with it some solutions which continue to drive virtualization and cloud computing, increasing efficiency, cutting costs, and easing the way to a more dynamic environment.

To read the full article please click here