From what I can tell, there is no way to remove external storage once it has been setup. Or perhaps, I am missing something. I do not see a "delete" button anywhere.
If you are 100% sure you are not using it, you could just delete it from the database. I've done it multiple times. But yeah, a delete storage button would be way easier 😀