It all began in early 2007, when our IT organisation was at the crossroads; the business was expanding, retail was going through change in the country, and competitors were whipping up euphoric expectations thereby putting artificial pressure on everyone.

Our server infrastructure was bursting at its seams, the data centre built almost five years back had already outlived its usefulness and servers could now be found spread across various offices in makeshift computer rooms, thus necessitating change and that too quickly.

The choice was between building a larger data centre which could hopefully last us the next decade, or outsource the data centre and adopt a hereto unknown buzzword – virtualisation. So the journey began with two teams going on different paths, one working on the business case to build a new data centre or outsource it, and the other to explore the world of virtualisation.

Our technology landscape was and continues to remain fairly simple with IBM i-series servers logically partitioned (read virtualised) for different business units running the core JDA retail ERP applications, Intel servers – standalone and blades – with Linux supporting Oracle e-Biz, PeopleSoft and Hyperion, and Windows with Microsoft SharePoint based applications respectively. We had just invested in Netezza, the data warehouse appliance, and many new applications were expected to be implemented in the next 12 months and thus new servers were to be procured and hosted in the data centre. The implementation of SAN and NAS with virtualised storage, a year earlier had improved the storage infrastructure, but data growth implied that it may not last the term that was originally anticipated and would require an upgrade much sooner.

The first team, led by the Head – Infrastructure, supplemented by a financial/purchase executive came back with budgetary estimates and time lines to build a new data centre and comparative analysis of outsourced options which effectively rendered the discussion one-sided in favour of outsourcing it. The team went off to discuss merits and capabilities of different service providers and negotiations but not before setting the other team with a target that whatever they did the number of racks should remain the same over the next 5 years to ensure that the business case did not go bust. A tall order claimed the second, which was the technical team lead by the Enterprise Architect and joined in by the application owners, Windows, SQL, and data centre specialists!

The starting point was the evaluation of available virtualisation stacks and it was a relatively easy choice with VMWare providing the best capabilities (as determined using an evaluation matrix). Other virtualisation technologies were as yet evolving and did not have the maturity or market expertise available. Initial candidates for the pilot were chosen from the test and development servers for non-critical applications running Windows operating environment. The first three months provided experience on how to configure, deploy, manage, scale up or down the configurations based on workload. All these worked well enough for the team to gain comfort with virtualised infrastructure.

Once the review confirmed the stability of the overall virtualised environment for the application stack proposed to run on it, I set the mandate that all new applications will be deployed only on virtualised servers while we work towards moving current applications in a phased manner. The die was cast in August 2007 with the data centre migrating away from dispersed locations into the outsourced service providers’ premises. The management was supportive, given a clear business case that promised savings within 2 years and increasing with each new application being added to the overall server farm.

The first pothole was hit quickly, unsettling the plans. The application in question behaved erratically as the number of users increased. The application provider tested the application in a controlled environment and confirmed the issue. The way forward at that point was to strike off that one application from the list and move on, but the apprehension remained about other applications from the same provider. All others worked without issues and thus were migrated and deployed on virtual servers.

In July 2008, a new application and technology stack was ready for deployment and the potholes gave way to roadblocks. Everything worked well until the load started increasing. One particular application provider accepted that they did not support virtual servers, but some customers had deployed them and it was working for them. They stipulated conditions under which the same would work on proprietary virtual servers but that was not aligned to the chosen standard. However, our tests on the defined technology stack indicated that the application once again was unable to scale up. We tried various alternatives including Microsoft Hypervisor, with no success. All applications from this technology vendor faced similar challenges and to date this remains unresolved and thus continues to run on physically separate servers representing about 25 per cent of the overall applications and workload.

Virtualisation brought new lessons to the team. Easy for provisioning new servers, someone also has to track the number of licenses required with every new server and track them. It’s a journey which is two steps forward, one step backward. Each new application if not certified explicitly requires rigorous testing beyond the conventional.

The starting point for us was slow with 8:1 virtualisation (8 partitions per physical server), moving now to 15:1 with a target in the coming year to get to 20:1 with new hardware offering multiple cores and CPUs. As anticipated, over the last 3 years, the data centre space has remained constant even though the number of servers has increased. Physical servers have reduced from around 100 to now 20. Utilisation and thereby efficiency is now around 40-50 per cent with some more room to grow from the earlier average of about 15 per cent.

The storage infrastructure was upgraded in 2009 increasing the capacity 3 times and connects well to the virtual environment. Now all applications with the exception of the incompatible and uncertified are operational on virtualised servers.

So what about Return on Investment? Well, we achieved break-even within 20 months and have saved more than 5 crore rupees in server investments in the last 2 years.

The future is already here with the formation of a Private Cloud using the overall server and storage infrastructure in a homogeneous interchangeable and on-demand scalable environment. In data centre resilience and disaster recovery is now enabled using virtualised servers. As we continue to work with our technology partners, over the next 12-18 months, we believe that the data centre will be able to offer on-demand compute and storage to the business at costs lower than what anyone could in a fully disparate physical world. Next step, Virtual Desktop Infrastructure!

By Arun Gupta