Mineos-node 100% CPU with large server

Bug Web-UI
1

I recently switched from mineos to mineos node and I noticed this when copying/importing my old servers:

As soon as I put the server files on disk in the new place (either through import or creating a blank server and copying the files) the minoes process cpu goes to 100% and the webUI stops responding.

Eventually I get round this by reducing the size of the server (and restarting mineos!) These servers have the dynmap mod installed which produces a web page map of the world (in 3D!). This can get very big - one ā€œvanillaā€ server is now 3.5Gb in size because of this!

Removing the dynmap mod and the dynmap directory everything is ok.

Any idea what could be causing this? It seems to be related to the size of the server files?

2

My initial suspicion is that it is less the size of the files, as opposed to the file count. The webui attempts to report on the server status page the amount of space used by a given server. With dynmap and overviewerā€“utilities that create vast amounts of tiny files on the filesystem, mean that MineOS has to sum up the total disk space used by all the files.

But, a single 3.5 gb file and 1 million 3.5k files (as an example) have vastly different performance implications when attempting to calculate size on disk.

Iā€™ll look into thisā€“if it is too implausible to get a real size-on-disk count, I might just have the webui fail out after a few seconds rather than run your system to 100% for however long; Iā€™ll try to reproduce this issue.

3

Hi,

I have seen the same behaviour. Starting MineOS on my server pegs one of the CPUs to 100% for a while. strace shows that it is aggressively scanning the /proc directory.

-Sage

4

Interesting findā€¦if that is the caseā€“then itā€™s something completely different than what I had theorized. Of course, now Iā€™m even less certain why leaving out dynmap (from the OP) would have improved this 100% spikeā€¦

5

Just spent a bit more time investigating. Ran the webui with the --prof option, and analyzed with Webstorm. Itā€™s tough to figure out, but it looks like, other than internals in node and libc, the biggest CPU consumption is used processing a Regexp: ^(\/?|)([\s\S]?)((?:\.{1,2}|[^\/]+?|)(\.[^.\/]|))(?:[\/]*)$ {1}

I have traced this to the splitPathRe function in object_hash_test.js

So it looks like itā€™s a problem with file processing after all.

1 Like
7

Have there been any more findings on this? For me, as soon as I put Dynmap files in the server folder (/var/games/minecraft/servers/ā€˜worldā€™/dynmap) MineUI stops responding. When I delete dynmap, everything returns to normal.

8

Mine does the same thing. Iā€™ve started a new server on Node and tried installing DynMap but relocating the web folders outside of the server directory but DynMap wonā€™t respond.

If someone has a solution to all of this it would be super helpful!

9

In the past iā€™ve had no luck with DynMap due to the HUGE amount of tiny files it has.

10

You have two easy solutions: One (very easy) inside the configuration.txt file, look for and change the storage type to sqlite, then delete the existing dynmap database (that the world images were generated within). Your second option, which Iā€™m pretty sure many of you are less likely to use, is a MySQL Database.

I recommend just using the sqlite. Rather than a bunch of files being processed to calculate a size, you just have one big file/database with the images stored within, which should help being that it is only one file to process (thousands less to process). Back before MineOS Node, I had the problem with Python, and after switching the setting, the UI would only freeze for about 5 seconds then show the server directory size.

storage:
  # Filetree storage (standard tree of image files for maps)
  #type: filetree
  # SQLite db for map storage (uses dbfile as storage location)
  type: sqlite
  dbfile: map.db
  # MySQL DB for map storage (at 'hostname':'port' in database 'database' using user 'userid' password 'password' and table prefix 'prefix'
  #type: mysql
  #hostname: localhost
  #port: 3306
  #database: dynmap
  #userid: dynmap
  #password: dynmap
  #prefix: ""

If it still does freeze/crash after applying the new setting, then you are either stuck having to deal with the problem, or youā€™ll have to learn how to configure it to use MySQL efficiently. Good Luck!

11

Just so you know: all MineOS Turnkey installs come with MySQL pre-installed. Sure, youā€™ll still have to figure out creating users and passwords (that MySQL uses), but it is just as accessible an option for those willing to put in the effort.