Then, to get rid of the desktop environment (if you want), take a look at -
USB issues have been a thing across several distros in the past based upon my experience (years later now I’ve yet to encounter that myself in comparison to years before, unless it is a super highly stripped down OS or something where I might sometimes at times encounter that issue), sometimes having issues based upon the port you plug it into, the BIOS/UEFI of the machine being problematic or having it’s own issues, the OS itself just not playing very well with USB storage devices, the selected drive storage file system format, and/or even at times a mixture of multiple and etc.
On AMD FX based systems/platforms, one workaround I know is setting iommu to soft (software) or off, and/or having acpi/apic set to off, in order to get things to work properly, you can try any mixture of that and/or any other Linux kernel boot flags; your mileage will vary.