Just to chip in some contribution here, sorry if am missing a point or not giving a direct (perhaps helpful?) answer to the original poster :) 

I recently had to deploy a site but because of budgetary constraints or rather what was available to us maybe at low cost (better economic sense) was the Amazon Web services (the EC^2 cloud). At first I was rather sceptical of the infrastructure since all your are given is access to one instance of whatever O.S you prefer (Linux, Windows, *BSD, Ms DOS :D) that you can log on to remotely (ssh, telnet, etc). 

What I found installed was the host language in which the site was going to run on, which is setup by default anyway. The rest of the components needed had to be figured out all from scratch. To cut a long story short, the native package manager (yum that is) on the distro proved to be more than a helpful tool in getting all the pieces together. I found the experience to be truly unique and highly enjoyable despite the fact that previously I have had no pain with deploying apps using hosts with pre-packaged services

Am just saying this simply because someone mentioned performing installations from source. And the O.P has problems un-installing components he does not need from his box :D

BTW: The Amazon Web services are rather more affordable than traditional hosting, really flexible and perhaps more fun to use SO LONG AS you are pretty comfortable with your platform of choice (Linux/UNIX, Apache, Engine-X (ngingx), PHP / PERL, etc). Try it!

Martin.