Move Local files to External

[ashkir]

💡Describe your idea

My image host grew to 1.6 million images. But, here's the problem. I got the external servers when my main server was almost full. I didn't expect t oget so big, with zero advertising! So I have a few storage servers now. However, as we continued to grow an additional million images, this is a million and a half new rows for the MariaDB SQL database. This obviously means a bigger database, which needs more support and disk space.

Now my disk space is near full. I have to purchase more for the main server.

But, what if I could move the oldest images, and maintain old URL mapping to an external server?! Even if I could move them in batches, like 1 user's images on the local storage at a time to not overwhelm the server processes.

👏Where did you saw this?

N/A

🔥Interest outside our community

N/A

 

[DeCysos]

You could "COPY" the local images on the external server.

Then you go to your site and create an external save there, if not already done.

Then log into phpMyAdmin, look in the table chv_storages out the ID of the corresponding external memory.
Go to the chv_images table and for each image change the values in the "image_storage_mode to datefolder" and "image_storage_id" columns to the value (ID) of the external memory.

When copying the images, you should make sure that the old folder structure is maintained.

------------
I did not test it, but that's the way it should work.
As always, I recommend to backup the database beforehand.
------------
But it would be nice of course if there is a button in the dashboard for the transfer (at least the database configuration).

 

[Rodolfo]

dashboard/settings/tools


1. Create a new external storage for the target content, take note on its ID.
2. Actually move the files from the server to the target storage server. You will have to either use command line, SFTP, etc.
3. Go to dashboard/settings/tools and migrate to the storage ID from step 1.

If the mapping is correct you swapped the storage successfully.

 

[DeCysos]

However, he was not concerned with a second external memory.
So it was not about moving from Ext1 to Ext2, but rather from LOCAL to Ext1.

Is that synonymous with the method described?
Local files do not have StorageID "NULL"

 

[Rodolfo]

Use zero (0) for local storage.

The tool simply updates the storage_id and refresh the storage stats. You use zero (0) to indicate local storage.

 

[ashkir]

thank you! this should be interesting. Going to need to find a way to migrate 400,000 images. 😀

 

[tomsit]

rsync everything, thank me later 😂

 

[faker.king]

dashboard/settings/tools

View attachment 2550
1. Create a new external storage for the target content, take note on its ID.
2. Actually move the files from the server to the target storage server. You will have to either use command line, SFTP, etc.
3. Go to dashboard/settings/tools and migrate to the storage ID from step 1.

If the mapping is correct you swapped the storage successfully.

How would the URL rewriting work. Usually we use a subdomain for S3 storage. The original URL for the local storage without the subdomain, will it still work ?

 

[Rodolfo]

How would the URL rewriting work.

How the URL resolves for external storage is not in our layer. Chevereto maps the URL, it is not in charge of its rewriting.

 

[siddharth]

How would the URL rewriting work. Usually we use a subdomain for S3 storage. The original URL for the local storage without the subdomain, will it still work ?

It won't work because the main storage URL is without a subdomain. So you have to do a URL rewrite to your new server location with its subdomain and so the links and images won't get broken.