Choosing the services to compare

I’ve tried to select VPS and Cloud instances specifications that are broadly equivalent in terms of RAM, disk space, and bandwidth offered for this study. What’s not so obvious is the computational power offered, and I’d like to state that I’ve benchmarked nothing at all, so don’t take this as a scientific recommendation, just my take on a big topic.

The VPS baseline is comprised of some tried and tested, and trusted, services.

• A bottom of the range 512MB Memset miniserver VPS, which is a low cost baseline I’ve used for years.
• A comparable 512MB Slicehost VPS.
• A Brightbox Nano VPS, which is comparable to the Brightbox Cloud instances I’m trialling.

Things weren’t so simple in the Cloud.

Most VPS services come with a fixed amount of monthly bandwidth included in the price, but their Cloud equivalents exclude bandwidth. Cloud bandwidth is charged by unit of resource you use, per GB for example, and often at a different rate for incoming and outgoing traffic. These costs are largely independent of the type of instance, so happily for my analysis they don’t reduce my options.

Rackspace make the point that the AWS offerings are much higher specification than needed by most web applications, and I found that it’s a valid claim that ruled out all but the Micro instance.

So, in the end I chose these options for the Cloud team.

• AWS Micro On Demand instance
• AWS Micro Reserved instance, which is the same but incurs a booking fee upfront in exchange for lower hourly rates.
• A 512MB Rackspace instance that’s hopefully comparable to the Slicehost VPS above.

Unfortunately Amazon have complicated the economics of the Micro instance by removing all storage. This is provided by attached EBS – bless Amazon and their abbreviations – which is priced according to it’s own model based on volume size and the number of requests: another cost to consider alongside bandwidth.

My analysis

To simplify things, I’ve used the baseline Memset VPS service as the benchmark against which to assess the other options, but something to note is that some of these services were prices in USD and others in GBP. Attempting to make this as consistent I drew up a list of assumptions for this analysis.

1. Production has 8760 hours of annual use.
2. Staging has 1152 hours of annual use.
3. Exchange rate of 1.6 (USD/GBP).
4. I/O request rate of 1.3 requests per second.
5. Production traffic is split 90% outgoing and 10% incoming.
6. Staging bandwidth is split evenly between outgoing and incoming traffic.

I then tried running my model at under three traffic conditions that I consider to represent low, medium and high volume. These were 10GB, 50GB and 100GB per month respectively; trust me, I don’t need to worry about anything higher at the moment.

OK this makes Brightbox seem very expensive, and it is, but I should point out that their Nano VPS is backed by a separate MySQL cluster and includes a New Relic standard account. None of the other providers do any of this, but since they’re one of my current hosts I’ve added them anyway. After all, I did this analysis primarily for my own benefit.

Oh, and the step change you can see in the cost for Memset under high traffic was due to a one-off fee for increased bandwidth, the base allowance is 50GB/month. The other VPS services had sufficient allowance for all my test cases.

So how does the Cloud stack up in production?

It seems that these bottom end instances are relatively competitively priced against these VPS services, with only Memset offering me a better deal. The costs also scale directly producing a nice smooth progression in expenses as traffic increases.

As mentioned before, an AWS reserved instance requires an upfront commitment that’s a large proportion of the total annual cost. That’s fine if you’re sure you’re going to use it throughout the year, as a production server probably, but it’s not very Cloud-like. Still, under these conditions it does appear to offer overall savings relative to all the competition except Memset at intermediate loads.

If you can afford a production Cloud, testing is even better

The real advantages come if you’re in a position to use an identical platform for production and testing. If the cloud is competitive with a VPS at your routine levels of traffic, then it’s going to offer even better value as a pre-production testing environment.

I decided that the best way to investigate staging in the Cloud was to reduced the number of hours I’d need the instance for each month, scale the bandwidth proportionally, and rerun the original model. I chose an arbitrary 96 hours (4 days) each month for testing purposes, it could be more, but could quite easily be less depending on the type of project.

This chart shows the effect of reduced usage for each of the Cloud instances in the analysis above, and the advantages of the hourly Cloud contract are obvious as the cost scales directly.

If I’m using a VPS then I’m forced to use another VPS on a monthly contract if I want an identical staging platform, and this is much more expensive than using any of the Cloud alternatives presented here.

Conclusions

Will I be moving to the Cloud? I’m certainly tempted, especially with my own projects. It’s interesting how there seem to be cost sweet spots at both really low loads and higher ones.

Being able to use identical staging environments is something I’d love. I’ve tried local virtual machines, but they’re never really identical, and I dislike having a VPS that hardly does any work. Unfortunately to realise this dream I’ve got to move all my production to Cloud instances, and concerns about reliability – especially given the highly public problems at AWS – make me reluctant.

The key is to be able to quickly spin up new instances, and rapidly destroy them once the testing is done, so that I can keep costs down. Most services allow you to save custom images from which you can create servers, so may be I need to play with this in the Brightbox beta some more.

Long term, I think there’s no question that this’ll be the basis of my development workflow.

I wouldn’t argue with their observations on this providers reliability, but the main problem with Webfusion, or their parent Pipex, arises when you need any technical support. I short, there isn’t much and that isn’t very knowledgeable; possibly unless you’re a national magazine.

I contacted Webfusion about a problem that arose with my VPS; six weeks later and after much email, I moved to Memset without the matter being resolved. And as for Pipex, I’m still waiting for an answer to my questions about my broadband package after nearly a month.

Pipex services are low cost, reliable and performance is generally good; which would be great if their accounting and technical support was less dreadful. If your needs are mission critical this makes Pipex the biggest potential problem your business might have. The risk is yours, but it’s not one I’m willing to take any more.

I’m not happy with the web hosting package I currently use. I mean, does Fedora Core 2 sound good to you? OK, I know, it depends what I’m trying to do…

I’m trying to develop Java and Ruby on Rails applications, and just getting the thing configured with such an antiquated version of Fedora is an up hill battle! So I’ve decided to make alternative hosting arrangements before trying anything serious.

On the back of a new business enterprise — Google needn’t worry for the moment — I’m going to get a couple of new VPSs.

1. A development server; the RoR and Java machine with the appropriate server stacks.
2. Production hosting of Drupal 5 installations with LAMP for my clients, which are hard enough get faultlessly configured on the current system.

It also seems like a good idea to separate development and production; I’m bound to knock over the development server at some point.

Both these systems are going to be based in the UK. They’re also going to be much more expensive than my current VPS, but I’ve learned that you often get what you pay for and I’m prepared for the expense.

The new host? I’ve asked questions of several and the most responsive and impressive to date is MEMSET and its Miniserver Virtual Machine range.