Mineos-node 100% CPU with large server

I recently switched from mineos to mineos node and I noticed this when copying/importing my old servers:

As soon as I put the server files on disk in the new place (either through import or creating a blank server and copying the files) the minoes process cpu goes to 100% and the webUI stops responding.

Eventually I get round this by reducing the size of the server (and restarting mineos!) These servers have the dynmap mod installed which produces a web page map of the world (in 3D!). This can get very big - one “vanilla” server is now 3.5Gb in size because of this!

Removing the dynmap mod and the dynmap directory everything is ok.

Any idea what could be causing this? It seems to be related to the size of the server files?

My initial suspicion is that it is less the size of the files, as opposed to the file count. The webui attempts to report on the server status page the amount of space used by a given server. With dynmap and overviewer–utilities that create vast amounts of tiny files on the filesystem, mean that MineOS has to sum up the total disk space used by all the files.

But, a single 3.5 gb file and 1 million 3.5k files (as an example) have vastly different performance implications when attempting to calculate size on disk.

I’ll look into this–if it is too implausible to get a real size-on-disk count, I might just have the webui fail out after a few seconds rather than run your system to 100% for however long; I’ll try to reproduce this issue.

Hi,

I have seen the same behaviour. Starting MineOS on my server pegs one of the CPUs to 100% for a while. strace shows that it is aggressively scanning the /proc directory.

-Sage

Interesting find…if that is the case–then it’s something completely different than what I had theorized. Of course, now I’m even less certain why leaving out dynmap (from the OP) would have improved this 100% spike…

Just spent a bit more time investigating. Ran the webui with the --prof option, and analyzed with Webstorm. It’s tough to figure out, but it looks like, other than internals in node and libc, the biggest CPU consumption is used processing a Regexp: ^(\/?|)([\s\S]?)((?:\.{1,2}|[^\/]+?|)(\.[^.\/]|))(?:[\/]*)$ {1}

I have traced this to the splitPathRe function in object_hash_test.js

So it looks like it’s a problem with file processing after all.

1 Like

Have there been any more findings on this? For me, as soon as I put Dynmap files in the server folder (/var/games/minecraft/servers/‘world’/dynmap) MineUI stops responding. When I delete dynmap, everything returns to normal.

Mine does the same thing. I’ve started a new server on Node and tried installing DynMap but relocating the web folders outside of the server directory but DynMap won’t respond.

If someone has a solution to all of this it would be super helpful!

In the past i’ve had no luck with DynMap due to the HUGE amount of tiny files it has.

You have two easy solutions: One (very easy) inside the configuration.txt file, look for and change the storage type to sqlite, then delete the existing dynmap database (that the world images were generated within). Your second option, which I’m pretty sure many of you are less likely to use, is a MySQL Database.

I recommend just using the sqlite. Rather than a bunch of files being processed to calculate a size, you just have one big file/database with the images stored within, which should help being that it is only one file to process (thousands less to process). Back before MineOS Node, I had the problem with Python, and after switching the setting, the UI would only freeze for about 5 seconds then show the server directory size.

storage:
  # Filetree storage (standard tree of image files for maps)
  #type: filetree
  # SQLite db for map storage (uses dbfile as storage location)
  type: sqlite
  dbfile: map.db
  # MySQL DB for map storage (at 'hostname':'port' in database 'database' using user 'userid' password 'password' and table prefix 'prefix'
  #type: mysql
  #hostname: localhost
  #port: 3306
  #database: dynmap
  #userid: dynmap
  #password: dynmap
  #prefix: ""

If it still does freeze/crash after applying the new setting, then you are either stuck having to deal with the problem, or you’ll have to learn how to configure it to use MySQL efficiently. Good Luck!

Just so you know: all MineOS Turnkey installs come with MySQL pre-installed. Sure, you’ll still have to figure out creating users and passwords (that MySQL uses), but it is just as accessible an option for those willing to put in the effort.