• Welcome to the Chevereto user community!

    Here users from all over the world gather around to learn the latest about Chevereto and contribute with ideas to improve the software.

    Please keep in mind:

    • 😌 This community is user driven. Be polite with other users.
    • 👉 Is required to purchase a Chevereto license to participate in this community (doesn't apply to Pre-sales).
    • 💸 Purchase a Pro Subscription to get access to active software support and faster ticket response times.

Is it possible to migrate images in local storage to external storage?

Status
Not open for further replies.
B

bangsters

Chevereto Member
Website URL
https://www.pixelsbin.com

Chevereto version
3.10.18

Description of the issue
Hey Rodolfo. Pixelsbin has been growing and we will be making some changes on environment. We have plenty of space on our local server for images, but we will be switching over to cloud for the images. The reason is we will be deploying redundant/failover servers including database. We experienced some downtime a while back due to fibercut hence we are making the change.


We will be switching to cloud (google, amazon, or ftp). Question is how can we migrate the local images to cloud without breaking the links or associations? Also, if down the road, if our cloud subscription needs to be migrated to a different cloud or ftp, is that possible as well without breaking associations to database or links?

Please advise.


And thanks again for this awesome platform.
 
It is possible but there's no tool that will do that for you automatically, you have to do it all manually. Basically you need to perform 3 actions:

1. Upload local to remote

For FTP/SFTP you can use the rsync command, for S3 is very easy by following this guide: https://chevereto.com/community/threads/migrate-your-existing-images-to-s3.5388/ No idea if Google, OpenStack or others support a similar sync feature. Key here is to sync (local) /images/* to (remote) /images/* using whatever you can for that matter.

2. Rewrite/redirect old URLS

In your local /images/ folder you will need to place a rule (server based) that will rewrite/redirect your old local URLs to the new remote URL. If you run Apache then you will need to write a rewrite rule in an .htaccess file on the root of your /images/ folder, if you run Nginx you will need to write the server rules for that sub-folder. If you run other kind of server then you will have to find how that server handles server side URL rewriting.

3. Update the database

You must run this query once you add the external storage from your dashboard:
SQL: 
UPDATE chv_images SET image_storage_id = <external storage id> WHERE image_storage_id = NULL
You also need to update the stats for that external storage, you only need to set the size (bytes) used in total by all the stuff that you uploaded to the external server.

Hope it helps,
Rodolfo.
 
Thanks. Looking for this in a long time. I thought it was going to be very hard updating the DB urls, but looking at it now it's only a single command away.
 
Hey @Rodolfo. Just want to let you know that all is working great, I updated the table for the images to reflect the chv_storage_id of the external storage. I did not need to make any redirect urls as they did not change. I migrated all images to cloud via gsutil, and all images are in cloud now. Much much faster with multi regional storage.

However, I did notice that in the settings for external storage, it shows that External Storage is 0KB, and Local is the xxxGB. That is incorrect, as all storage is in external now. If I click search link for local storage it opens external storage, and vice versa it I open search link external storage it opens the local storage
 
Last edited:
Like I said, you need to update the stats. Since you moved local to remote, you need to reflect that change on the stats table.
 
Thanks I missed that part. How do I update the stats? Particular table?

Also, any new images rhat get uploaded still reflect as local storage eventhough it is saved in external
 
Last edited:
chv_storages
1533917035976.png
chv_stats
SQL: 
SELECT * from chv_stats where stat_type = 'total'

Column "stat_disk_used"
 
bangsters said:
cname to storage.googleapis.com
Click to expand...

You can do that - but if your site uses SSL you will get mixed content. With Google and OVH storages it's not possible to get CNames working with SSL ... when I'm remember right, only with AWS it will work.
 
Ahh ok thanks for pointing that out. If I do decide to try it, how can I ge the embed codes to reflect my subdomain that is a cname to storage.googleapis.com?
 
You can map the external storage to any URL, but you must do all the DNS configuration on your own.

Keep in mind that Chevereto doesn't handle your domain DNS settings. You must refer to Google cloud documentation in how use your own cname for your bucket.
 
Thank you @Rodolfo!! I figured out a way around it. You are 100% correct that Google Cloud Storage does not do HTTPS when it is accessed via CNAME. But I managed to work around it, with redirect and some modification on the route file.

Thanks again.
 
Status
Not open for further replies.
Back
Top