Docker and file changes

Web-UI
1

I’m currently testing using MineOS in a Docker container (hexparrot/mineos:node-jessie). The image pulls fine and I an able to create a container and run MineOS. I have /var/games/minecraft mapped to a remote folder on my NAS rather than the Docker VM so that the data doesn’t disappear when he VM reboots (boot2docker). The issue I am running into is if I create a new world, it doesn’t show up in the GUI until I stop and restart the Docker container. Nor does the “Refresh Server List” link show the newly created server. Is there a way to get MineOS to “rescan” the servers folder, ether after creating a new server or via some manual button? I’m guessing this is already implemented, but may be relying on the underlying filesystem reporting changes (which doesn’t happen with remote filesystems).

Thanks.

2

Hrm, this is an interesting issue. Can you provide the /var/log/mineos.log file so I may look through it? Basically, “Refresh Server List” should already do this–it should refresh your webui entirely with all the available servers properly owned by the user you’re logged in on.

That means this is either breaking at:

a) detecting a new directory on a remote file system
or
b) the webui is detecting, but not displaying it in the webui.

The log might help me figure out which. Also, with your remote filesystem, how have you mounted it? (which FS type, is ownership identical remotely and locally, etc.?)

Any details might help me get there; I’m not able to set this up with a NAS, but if its just samba, or cifs, or nfs, maybe I can attempt to reproduce it all the same.

3

First, about my system: I am testing FreeNAS10. FreeNAS is based on FreeBSD which uses the bhyve virtual machine manager. There is a Boot2Docker VM running which hosts all the docker containers. I am using the hexparrot/mineos:node-jessie container and have /var/games/minecraft mapped to the FreeNAS host (not Boot2Docker) via 9pfs (Plan 9 Filesystem) which is what FreeNAS uses link the containers to the ZFS filesystem on FreeNAS. Ownership is set up identically on the container and FreeNAS.

Below is the mineos.log info:

{“level”:“info”,“message”:"[test] Discovered server",“timestamp”:“2017-01-06T19:33:36.023Z”}
{“level”:“info”,“message”:"[test] Created tail on logs/latest.log",“timestamp”:“2017-01-06T19:33:36.041Z”}
{“level”:“error”,“message”:"[test] Create tail on server.log failed",“timestamp”:“2017-01-06T19:33:36.045Z”}
{“level”:“info”,“message”:"[test] Watching for file generation: server.log",“timestamp”:“2017-01-06T19:33:36.045Z”}
{“level”:“error”,“message”:"[test] Create tail on proxy.log.0 failed",“timestamp”:“2017-01-06T19:33:36.281Z”}
{“level”:“info”,“message”:"[test] Watching for file generation: proxy.log.0",“timestamp”:“2017-01-06T19:33:36.281Z”}
{“level”:“error”,“message”:"[test] Create tail on logs/fml-server-latest.log failed",“timestamp”:“2017-01-06T19:33:36.284Z”}
{“level”:“info”,“message”:"[test] Watching for file generation: logs/fml-server-latest.log",“timestamp”:“2017-01-06T19:33:36.285Z”}
{“level”:“info”,“message”:"[test2] Discovered server",“timestamp”:“2017-01-06T19:33:36.432Z”}
{“level”:“info”,“message”:"[test2] Created tail on logs/latest.log",“timestamp”:“2017-01-06T19:33:36.438Z”}
{“level”:“error”,“message”:"[test2] Create tail on server.log failed",“timestamp”:“2017-01-06T19:33:36.439Z”}
{“level”:“info”,“message”:"[test2] Watching for file generation: server.log",“timestamp”:“2017-01-06T19:33:36.440Z”}
{“level”:“error”,“message”:"[test2] Create tail on proxy.log.0 failed",“timestamp”:“2017-01-06T19:33:36.441Z”}
{“level”:“info”,“message”:"[test2] Watching for file generation: proxy.log.0",“timestamp”:“2017-01-06T19:33:36.441Z”}
{“level”:“error”,“message”:"[test2] Create tail on logs/fml-server-latest.log failed",“timestamp”:“2017-01-06T19:33:36.442Z”}
{“level”:“info”,“message”:"[test2] Watching for file generation: logs/fml-server-latest.log",“timestamp”:“2017-01-06T19:33:36.443Z”}
{“level”:“info”,“message”:"[test3] Discovered server",“timestamp”:“2017-01-06T19:33:36.445Z”}
{“level”:“error”,“message”:"[test3] Create tail on logs/latest.log failed",“timestamp”:“2017-01-06T19:33:36.448Z”}
{“level”:“info”,“message”:"[test3] Watching for file generation: logs/latest.log",“timestamp”:“2017-01-06T19:33:36.448Z”}
{“level”:“error”,“message”:"[test3] Create tail on server.log failed",“timestamp”:“2017-01-06T19:33:36.450Z”}
{“level”:“info”,“message”:"[test3] Watching for file generation: server.log",“timestamp”:“2017-01-06T19:33:36.450Z”}
{“level”:“error”,“message”:"[test3] Create tail on proxy.log.0 failed",“timestamp”:“2017-01-06T19:33:36.452Z”}
{“level”:“info”,“message”:"[test3] Watching for file generation: proxy.log.0",“timestamp”:“2017-01-06T19:33:36.452Z”}
{“level”:“error”,“message”:"[test3] Create tail on logs/fml-server-latest.log failed",“timestamp”:“2017-01-06T19:33:36.473Z”}
{“level”:“info”,“message”:"[test3] Watching for file generation: logs/fml-server-latest.log",“timestamp”:“2017-01-06T19:33:36.491Z”}
{“level”:“info”,“message”:"[test] eula.txt detected: ACCEPTED (eula=true)",“timestamp”:“2017-01-06T19:33:39.332Z”}
{“level”:“info”,“message”:"[test2] eula.txt detected: ACCEPTED (eula=true)",“timestamp”:“2017-01-06T19:33:41.661Z”}
{“level”:“info”,“message”:"[test] autostart = false",“timestamp”:“2017-01-06T19:33:41.670Z”}
{“level”:“error”,“message”:"[test] Aborted server startup; condition not met: true",“timestamp”:“2017-01-06T19:33:41.676Z”}
{“level”:“info”,“message”:“Starting up server, using commit: d9a604b bumped express to 4.14.x\n”,“timestamp”:“2017-01-06T19:33:41.682Z”}
{“level”:“info”,“message”:"[test2] autostart = false",“timestamp”:“2017-01-06T19:33:41.851Z”}
{“level”:“error”,“message”:"[test2] Aborted server startup; condition not met: true",“timestamp”:“2017-01-06T19:33:41.877Z”}
{“level”:“info”,“message”:"[test3] autostart = false",“timestamp”:“2017-01-06T19:33:42.003Z”}
{“level”:“error”,“message”:"[test3] Aborted server startup; condition not met: true",“timestamp”:“2017-01-06T19:33:42.004Z”}
{“level”:“info”,“message”:"[WEBUI] admin connected from 172.16.100.201",“timestamp”:“2017-01-06T19:37:27.195Z”}
{“level”:“info”,“message”:"[WEBUI] Downloading official profiles.",“timestamp”:“2017-01-06T19:37:28.185Z”}
{“level”:“info”,“message”:"[test] admin (172.16.100.201) joined server namespace",“timestamp”:“2017-01-06T19:37:29.495Z”}
{“level”:“info”,“message”:"[test2] admin (172.16.100.201) joined server namespace",“timestamp”:“2017-01-06T19:37:29.539Z”}
{“level”:“info”,“message”:"[test3] admin (172.16.100.201) joined server namespace",“timestamp”:“2017-01-06T19:37:29.539Z”}
{“level”:“info”,“message”:"[test] transmittting existing file contents: logs/latest.log (821 bytes)",“timestamp”:“2017-01-06T19:37:32.416Z”}
{“level”:“info”,“message”:"[test2] transmittting existing file contents: logs/latest.log (1607 bytes)",“timestamp”:“2017-01-06T19:37:32.430Z”}
{“level”:“info”,“message”:"[test3] 172.16.100.201 issued command : “delete”",“timestamp”:“2017-01-06T19:37:59.095Z”}
{“level”:“info”,“message”:"[test3] received request “delete”",“timestamp”:“2017-01-06T19:37:59.118Z”}
{“command”:“refresh_server_list”,“level”:“info”,“message”:"[WEBUI] Received emit command from 172.16.100.201:admin",“timestamp”:“2017-01-06T19:38:23.216Z”}
{“level”:“info”,“message”:"[test2] transmittting existing file contents: logs/latest.log (1607 bytes)",“timestamp”:“2017-01-06T19:38:23.567Z”}
{“level”:“info”,“message”:"[test] transmittting existing file contents: logs/latest.log (821 bytes)",“timestamp”:“2017-01-06T19:38:23.705Z”}
{“command”:“create”,“server_name”:“test04”,“properties”:{“generate-structures”:true,“server-port”:“25567”,“level-name”:“test”,“level-seed”:“1001”,“difficulty”:“2”,“gamemode”:“0”},“level”:“info”,“message”:"[WEBUI] Received emit command from 172.16.100.201:admin",“timestamp”:“2017-01-06T19:39:00.825Z”}
{“level”:“info”,“message”:"[test04] Server created in filesystem.",“timestamp”:“2017-01-06T19:39:01.005Z”}
{“command”:“refresh_server_list”,“level”:“info”,“message”:"[WEBUI] Received emit command from 172.16.100.201:admin",“timestamp”:“2017-01-06T19:39:13.324Z”}
{“level”:“info”,“message”:"[test] transmittting existing file contents: logs/latest.log (821 bytes)",“timestamp”:“2017-01-06T19:39:13.530Z”}
{“level”:“info”,“message”:"[test2] transmittting existing file contents: logs/latest.log (1607 bytes)",“timestamp”:“2017-01-06T19:39:13.531Z”}
{“command”:“refresh_server_list”,“level”:“info”,“message”:"[WEBUI] Received emit command from 172.16.100.201:admin",“timestamp”:“2017-01-06T19:39:31.516Z”}
{“level”:“info”,“message”:"[test2] transmittting existing file contents: logs/latest.log (1607 bytes)",“timestamp”:“2017-01-06T19:39:31.725Z”}
{“level”:“info”,“message”:"[test] transmittting existing file contents: logs/latest.log (821 bytes)",“timestamp”:“2017-01-06T19:39:31.733Z”}
{“level”:“info”,“message”:"[test] 172.16.100.201 issued command : “backup”",“timestamp”:“2017-01-06T19:39:56.203Z”}
{“level”:“info”,“message”:"[test] received request “backup”",“timestamp”:“2017-01-06T19:39:56.205Z”}
{“level”:“info”,“message”:"[test] 172.16.100.201 issued command : “archive”",“timestamp”:“2017-01-06T19:40:02.354Z”}
{“level”:“info”,“message”:"[test] received request “archive”",“timestamp”:“2017-01-06T19:40:02.358Z”}
{“command”:“refresh_server_list”,“level”:“info”,“message”:"[WEBUI] Received emit command from 172.16.100.201:admin",“timestamp”:“2017-01-06T19:40:09.864Z”}
{“level”:“info”,“message”:"[test] transmittting existing file contents: logs/latest.log (821 bytes)",“timestamp”:“2017-01-06T19:40:10.838Z”}
{“level”:“info”,“message”:"[test2] transmittting existing file contents: logs/latest.log (1607 bytes)",“timestamp”:“2017-01-06T19:40:11.731Z”}

4

Hexparrot,

Have you had a chance to look at this? Is there any more information I could provide?

Thanks!

Ian

5

I still haven’t worked with FreeNAS, but retrying just Docker allowed it to detect all servers I created, as expected. It would appear that the issue here is beyond Docker, in the layer between Docker and the remote FS.

That is, whether I have the node-jessie image working with a docker container or even direct mounted to a local dir on the FS, it works. Thus, it seems like issue is that the directory watcher is not getting FS events when crossing over to the remote folder, just like you suspected in the initial post.

Is it possible that you have all your worlds inside the docker container, but instead relocate your archives and backups to your remote folder?

With the regular backup option (+use of save-all at a low interval), you can instead have everything running at local HDD speeds.

The downsides of this route are:
a) requires hdd space on the FreeNAS machine rather than on your remote mount
b) a crash would put you back to your last backup

but these issues can be mitigated by having a very frequent backup interval (5 minutes?). It could just be that 9pfs or ZFS just don’t work well with event reporting remotely. Sorry I don’t have better solutions available.