The only thing I can recommend is to try creating a separate subdomain and see if the problem persists…..or maybe add the same setup on a local install like mampp or xampp (if you know how to do that) and then see if it still happens. If it doesn’t and you think its your host, then I would move to a different host, especially the hosting companies that advertise speed and good performance as one of their features.
You can also try the “Yslow” plugin, if it works you may be able to see which files are causing your site to load slowly.
I create awesome sites for awesome people! Contact me if interested – email@example.com