Managing Archives and Backups

So I haven’t been using this platform for very long. Had a few rough patches but things are operating pretty well. I have been having a huge issue with available disk space. I am running this in a VM on a decent desktop. I originally gave the system about 20 gigs and was running into issues daily with space due to doing backups every hour and archives every 24 hours. So I went in rebuilt the machine with 50 gigs and changed the backup interval to 2 hours. So now I run into problems every few weeks where it sucks down that space. How do I set this thing up to only maintain so many archives and so many backups so that it will manage the disk space on it own without me having to go in and delete stuff to free it up. When I forget to do this, the server crashes and corrupts and I have to go in a use a recovery point or sometimes an archive to fix it before the server will restart without crashing. Look forward to the advice and thanks.

-Phatboydw

  1. 2Tb HDD costs less than $90
  2. Write a cron script to purge old backups/archives. There are tons of this stuff already written (for example), so just update your crontab with one.

Siliconheart,

 Thanks for the links to the cron scripts, I will definitely play with this. After a quick review it looks like that will handle the archives very well as they are single files. The backups\recovery point system, I have not been able to full understand how it keeps its records on what files are for what in terms of the revisions. Up until now I have just been deleting the folders contents and running a new recovery point just to ensure I have a good one when I need to free space. Is there a more specific way to delete the older recovery points and only maintain so many?
 It just seems a bit odd to me that this type of functionality isn't built into this system as one would think that a continually growing list of copies is going to eat at a drive of any size eventually. Could I throw additional disk space or hard drives on to this system sure. But once cleaned up my minecraft server only sits at like 10gig why be so wasteful to dedicate 2Tb to the system? Thanks.

Have you tried using the web-ui’s built-in functionality to clear up unwanted restore points and archives?

Do you know how to use the web-ui to determine how much space is being used by your restore points, so you know if you should be clearing your restore points at all?

Hexparrot,

 I have been using the Webmin file manager tool to delete the files. On the MineOS main page, it displays the current percentage of the hard drive that is being used, also on the server status tab it shows the current space usage for both restore and archives. I have not seen anything on the MineOS side of things that offers any way of deleting the restore points which is definitely the bigger culprit of eating disk space.

There is a prune option by clicking on the heading I think (I am on my phone so not 100% sure)

That’s even easier than archives - there is a command like rdiff-backup --remove-older-than, so google for rdiff-backup for more info.

See the video here, which helps demonstrate how to use the web-ui for determining space used by restore points and how to remove them cleanly. Using the web-ui leverages the rdiff-backup --remove-older-than flag from the above post.

You’ll want to avoid deleting things manually in the future via a file manager (like webmin), for either of these approaches that use --remove-older-than (webui/command line).

Archives use the same interface (‘remove older than’ date) from the archives tab, and you can mass-delete or manually delete as necessary.

Hexarrot,

 I guess I am just a blind idiot. I have reviewed those pages several times and not once did I notice the text box ... at the top of the page ... stating to pick a date in order to prune. Thank you for pointing out the obvious and I'm sorry for wasting everyone's time. I will still look into some of the above methods for automating this process. Thank you all for your time.

-Phatboydw

Well, purging scheduling via web-ui would be an useful feature anyways. Maybe in next version, William?

Purge scheduling I suspect is really very easy to implement. However, like many features that would be unquestionably useful, I left this one out for liability concerns. When it comes to deleting data, I kind of want that responsibility passed down to the admin, because I get enough “I lost data” emails as-is, without giving people the method of deleting.

I don’t really like withholding features for this reason, but above all, I need to maintain my sanity and motivation!

1 Like

Hi, I know this is an old thread and there may be an answer to my question somewhere else but I haven’t found it.

Is there or can there be an implementation to either specify where the archives and backups are located (like if I’d like to put them on a secondary drive) or scheduling to move (instead of prune or delete) old files?

Thanks

Hello All.

I know this thread is super old, but I am new to Linux and had a diffluent time settings this up and feel like this could be helpful.

I am running the MineOS-Node and did the following to setup auto-pruning. Based on my settngs, any backups older than 5 days should be deleted at 6am, everyday.

1.Log into the turnkey webmin console as your “root” account. (ex. https://192.168.x.x:12321)

2.Click “Scheduled Cron Jobs”

3.Click “Create a new scheduled cron job.”

4.Fill in the information as need. I chose the following settings:

Job Details:
-Execute cron job as: root
-Active?: Yes
-Command: “rdiff-backup --force --remove-older-than 5D /var/games/minecraft/backup/Flat_Server” (Replace “Flat_Server” with your Minecraft server name.) (Also, drop the quotes)
-Input to command: (I left this blank)
-Description: Prunes all backups older than 5 days for “Flat_server”. Runs at 6am, everyday. (You don’t need to fill this out.)

When to Schedule: “Times and dates selected below…”
-Minutes: Selected… 0
-Hours: Selected… 6
-Days: All
-Months: All
-Weekdays: All

Date Range to execute: Run on any date

5.Click “Save”

I found the following site useful for learning about the “rdiff-backup” command.
https://tinyurl.com/js5molu

I hope this helps!

-TheHoFL

edit: Updated the webmin port.

1 Like