Minecraft, when allocated memory, doesn’t ever unload/free memory unless it needs to.
Regardless of whether there are 10 players on–or 2. Basically, when you allocate 10GB to Minecraft, it says “I will do my best to load all the active areas for any logged-in players. I won’t unload any areas (even if you leave them) unless I run low on free memory.”
This is an intended behavior, and something we should expect/be happy with. Minecraft holding 10GB of RAM doesn’t mean it runs slower than if it released down to 6GB…in fact the opposite; if it unloaded areas just for the sake of unloading them, then it would have to load it again later.
Free memory doesn’t help performance in any way; being out of memory is a different thing, but typically until you see your swap file increase, Minecraft being greedy is a good thing. (and if the swap file does end up getting used, it means you need to allocate less to Minecraft because it’s over-allocated.