Overtime, RAM goes away?

I wouldn’t make that generalization, no.

But the site I linked you to tells you about “top” which will help you–in your specific case–determine whether or not the memory is cached or whether it is free.

If it truly isn’t free, then starting a server will do no good. It could just as well be that your other server leaked memory and it is lost (and you need to restart your host). We can’t tell, merely with the information 170mb, so you’ll have to look it up a bit.