File system suddenly changed to read-only

Hello there!

I’ve successfully made a server over the last few weeks and everything had been running well, until suddenly I saw in my Discord server-log (I have the DiscordSRV plugin that streams the whole server log to a separate text channel on my Discord server) that the server suddenly couldn’t save chunks anymore, so I went to turn the server off, but found that the web interface wasn’t available anymore. I ried to run the stop command from the Discord, but that too, failed. I then had to log into the server and type /stop from there. The server stopped and I went into WinSCP to see if there was anything wrong with the files, it everything seemed in order, but when I tried to fiddle around a bit more I noticed that I couldn’t do anything. It gave an error code 4. I turned the whole computer off with PuTTY and had to do something else, thinking I’d fix it later.

Later I connected a screen and keyboard to the computer and turned it on. It booted a thing called “initramfs”. I googled the problem and found this ( answer, so I followed it, exchanging ‘reboot’ with ‘exit’, because ‘reboot’ didn’t do anything.
I finally booted into the standard blue screen again, so I exited out of that and landed on the login page. I logged in as root and tried to find out what the problem was. I finally found out that the filesystem was suddenly read-only!
That was not something I had done, so naturally I went and tried to Google a solution and found this ( so I tried it, but I couldn’t find the drive to remount. I tried to remount ‘/’ and that did something, but actually not. I can usually Google my way out of these kinds of problems, like I have done in the last few weeks trying to set everything up nicely and all, buy now I’m really stumped…
I did manage to get a backup on my main computer, so if everything goes very wrong I should be able to just reformat the drive, reinstall the OS and put everything back, but that would be quite the hassle so I really hope this can be fixed another way where I don’t lose my data.

Thanks for reading this whole thing, I know it’s a bit long. If there’s anything else I need to mention, just ask and I’ll try to deliver as much data as I can.
I really hope someone can help…

Did you run out of storage space?

To check: df -h

I can’t imagine! It’s a 220 GB HDD and I’ve only been using it for a few weeks. I thought it was like maybe 20% full at most. I’ll go check!

EDIT: I have now checked and I’ve taken a picture of the result:

It is curious why Linux would mount your / as read-only, certainly.

Here are the places we should check then:

  1. dmesg
  2. /var/log/messages
  3. mount -a

Perhaps filesystem corruption? We can run fsck:

To remount a filesystem as read-write, we can do:

# mount -o remount,rw /dev/sda2

That of course will only work if the mount that is read-only is /, which actually isn’t a given. Do you recall making /root a separate mount from / during the ISO installation?

Let’s see what /etc/fstab looks like, as well.

dmesg returns a lot of text, so I’ve copied it into a .txt file here:

Here are the four messages logs I can find there:

This command doesn’t seem to do anything. It delays for maybe half a second (though that might just be the delay between PuTTY and the sever)
root@SERVER-MINECRAFT ~# mount -a

I remember running fsck in that initramfs state, but I’ll try to run it from the normal terminal too.
I’ll follow that guide you sent about fsck in a little bit and when I have the results, I’ll edit them here.
EDIT: The filesystem has returned to being Read-only just as I feared. I cannot execute touch /forcefsk, so I need to find another way to do it.
EDIT2: I tried the recovery mode from the guide, but after selecting that I wanted to boot into recovery mode it put me back into that initramfs mode. I’ll try to run fsck from there, but I already did that twice.
Here’s a picture of what this mode looks like:

It seems like this also didn’t say anything, which made me think the drive is already in Read/Write mode??
root@SERVER-MINECRAFT ~# mount -o remount,rw /dev/sda2
I tested my hypothesis by copying a tile to the server with WinSCP and I got no errors.
So I guess the problem is gone for now? I really don’t trust this.

I don’t remember what I exactly set up in the partitions when I first installed it, but I do remember automatic mode failed and I had to quickly learn how to manually set partitions for Linux.
Maybe I didn’t set those up correctly (enough)? Maybe the fact that automatic mode failed should have been a warning? I don’t know.

Pretty much all knowledge of Linux I have at this moment I got from setting up this server business, so I’m kind of new to all of this. But I can Google really well, so that has helped an enormous lot, I think.

Here is the /etc/fstab file:

I had this same exact issue when I had installed ubuntu server. Do you remember when setting up your disk partitions as LVM?

I really don’t remember what settings I set when I set the partitions.
I remember having to allocate a bit of swap, so I did that. There was also some other thing it wanted and then I just allocated the rest to normal disk space.
I’m really not sure. I’d have to browse through the menu again to find what I did exactly, which I might do a little later as I don’t have MineOS Turnkey on a stick anymore, so I’d have to put it back on one.

I’m going through Ubuntu server installer and by default it’s set to select “Guided - use entire disk and set up LVM”.

Depending if you can search how to increase the LVM partition you may need to reinstall your OS. Just make sure you save anything on the drive.

When going through the instalation and get to partitioning your disk choose “Guided - use entire disk”
Select your disk
Choose yes or no if it looks good or you want to set your own sizes.

Also from my knowledge, you don’t necessarily need swap space if you have 8gb or more of ram. Especially if it’s an ssd your installing on to.

I wrote earlier that the automatic partitioning (I guess it’s actually called guided, the real name slipped my mind) didn’t work, so I partitioned it manually.

I have 3 GB of RAM and an old HDD…
I know it’s not ideal, but it’s been running a Paper 1.14.4 server for me and my friends just fine the past few weeks.

But what you’re saying are thing that I can only do when I reinstall the whole OS and I’d rather not resort to such drastic measures if I can help it.

1 Like

In the world of Linux, little things like that can break installations. It the ups and downs of using Linux that makes it such a great OS to use for basically anything.
I’m currently in near the end of installing Ubuntu server 18.04.3 lts with LVM to confirm what your running I to as it’s my day off and I don’t have anything much to do.

What kind of machine are you using if I may ask?

Also with setting up swap space, the rule of thumb is to have 2x the amount of ram you have.

I installed MineOS Turnkey on (according to that dmesg I did) Hewlett-Packard Compaq Presario CQ60 Notebook PC/303C, BIOS F.54 08/18/2009.
It used to be a laptop until the screen gave out, so my cousin ripped out all the parts and basically screwed them onto a plank. I’d been using that computer as my main one for quite a few years, when I finally got a new computer. It’s been sitting in the attic, collecting dust until a few weeks ago when my mates and I decided it’d be cool to make an adventure map together, so I decided to use that old computer to run a small server.
I’m planning on exchanging the HDD with an SSD when I’ve saved up enough and I’ve been trying to get my friends to help pay, so they too can profit, but to no avail… Ah well. Guess it’s mainly their problem; having a crappy server. (I don’t play on it myself that much)

Now I’ve seen the model number of the thing, I suddenly remembered I “inherited” another Presario CQ60, though that one it still completely functional (yet VERY slow).

I’ve been trying to do some more tests and stuff and u e found that sometimes it boots up just fine, just like it’s always done, but sometimes it boots into initramfs. If that happens I run fsck /dev/sda2 -y and when that is finished I run exit and then it boots. Just now when I did this it booted into read-only mode, but then I ran mount -o remount,rw /dev/sda2 and it just did it. I can now write again.

It seems to be completely random whether it decides to work well or not, but it seems like I at least now know how to fix it if it happens, but that seems like a really bad idea. I’d rather fix it at the root of the problem, than to just keep running commands manually to make it work for the next while. I’d also like to be able to just set the computer away again with no other inputs than power and internet, because it’s using one of my main screens right now and I’d like that back from the server, but if I need to keep running commands to keep it running I can’t do that.
I’m really quite stuck here…

Well you might need to just reinstall your OS to fix that.

After you installed your OS, install SSH

sudo apt-get install openssh-client

and FTP

sudo apt-get install vsftpd

The other thing I like to do personally is enable the root user and allow access with said root user through ssh and ftp. You will need to manually edit the config file and restart the service.

Just make sure that you make a good password for root if you care about security.

One last thing is to set a dedicated ip for your machine in your router settings if you haven’t done that already.

This way all you need to have is the power plug and an ethernet connection and you’re all set.

I already have root access with SFTP (WinSCP) and SSH (PuTTY), but I don’t think I can SSH into that initramfs state, so I’d still need the screen and keyboard.
I can even use my phone to access the filesystem through STFP if it’s really necessary. I also have an app that I can use to send SSH commands to it.

I also have a normal “Minecraft server”-ish ip that my players can use to connect to and it’s been working just fine for the last few years. I used to host servers on my main Windows computer, but just in the afternoons. Now I can (should be able to) have a server running 24/7. I’m using the same text IP as I used to and so far there have been no connectivity problems.

I probably will reinstall MineOS Turnkey if everyone is out of ideas.

I don’t think this is the solution you’re looking for. The issues you’re experiencing seem consistent with hardware related issues, not the installation.

[ 32.992912] blk_update_request: I/O error, dev sda, sector 263097464
[sda] Medium Error
[sda] Add. Sense: Unrecovered read error - auto reallocate failed

I/O error on sda isn’t something a new installation can fix, nor is UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.

Even at the bottom of the screenshot it says “[Hardware Error]:”

read-only is the systems way to protect your data from corruption, so while you can run fsck or reinstall, if something is broken about the HDD, expect it again with a reinstall of Turnkey–or any distro, for that matter.


Well, that is too bad…
Thank you for helping figure out what the actual problem is!

At least I know the actual problem now and I can fix it with a bit of money.
Do you think a Samsung 860 250 GB would be enough, or do you recommend some other storage device? (I don’t have an M.2 slot on the thing, of course, seeing as it’s actually a really old laptop)
I automatically make restore points every whole hour between 08:00 and 22:00 and as of yet, it hasn’t grown to be too big, so I think (hope) that 250 GB would be enough.
I have a cloud service I can access through WebDAV where I’m planning on uploading a full archive every day.