Migrating FROM S3

[ubmm]

Hello all -

From the start I decided to use S3 thinking I could avoid the headache of later trying to migrate. Now, I've learned the hard way that S3 bandwidth prices are quite heavy. As a result, I actually want to move to local storage.

I can definitely activate a new storage container for new photo uploads, but I was wondering how do I migrate what is currently on S3 to my new host?

Do I just a) create a new storage that is local for new photo uploads and then b) inactivate s3 in chereveto and then c) copy all the files as-is (same directories etc) to the new host?

Thanks

 

[Lastopp]

Where are you moving? I also used S3 for a while.

I started by disabling uploads for both guests and users, then moved all my images to another storage provider and changed my settings in external storage. Also DNS records. It's very easy and straight forward.

 

[ubmm]

Thanks @Lastopp -

So I can just move the files from S3 to another storage platform? That is, I thought maybe the names/directories would be different or something, but it sounds like I could just copy them all (and thanks for the tip about disabling uploads during the process).

 

[Rodolfo]

I thought maybe the names/directories would be different or something

In your given external storage yes, of course that the system path won't be the same but Chevereto doesn't see that. Chevereto maps to <URL>/file.jpg so if the URL resolves to the same filepath than before then all files will work for you right away.

 

[ubmm]

Ok update but still an issue.

I moved everything from S3 to local. I deactivated external storage in the settings. New uploads are coming in properly to local storage. My question is related to images before deactivating external storage:

I used to have an alias in my DNS (I renamed it to mysite here for privacy)

CNAME i is an alias of i.mysite.com.s3-website-us-east-1.amazonaws.com

And my images would exist at the url, for example:

http://i.mysite.com/2016/06/28/05aaefc2083ce4174ab2f6f8aac7c6ab.jpg

What should I change my DNS settings to so that I can stop it pointing to S3 now but that old references (eg http://i.mysite.com/2016/06/28/05aaefc2083ce4174ab2f6f8aac7c6ab.jpg) will resolve to the file that was copied from S3 to local.

Thanks for the help

 

[Rodolfo]

To move from external to local you have to point that CNAME to your server (usually via subdomain) and map that to your given local folder.

 

[ubmm]

@Rodolfo Thanks - 2 specific questions if I may:

1. Pointing the CNAME to my server is done, but can you explain what you mean by mapping to a local folder?

2. I understand the link to the actual image, but can you explain how the links to the images in the "explore" mode (and the page they go to) are generated? That will help me understand the changes I have to make.
For example,

In explore mode, I see an image with the URL of (when hovering over):

http://mysite.com/image/uV7VG

That page contains the actual image which points to the jpg file.

How are those pages generated? After I change my cname the system cannot find the page if clicking on it.

Thanks

 

[Rodolfo]

1. Pointing the CNAME to my server is done, but can you explain what you mean by mapping to a local folder?

When you point a record to a given hostname or IP what you are doing is that you are pointing that record to your server DNS zone so is your server which handles how that record will be mapped. Basically, the DNS zone allows you to own that record and your web host hooks on that and resolves a given public path to that record.

2. I understand the link to the actual image, but can you explain how the links to the images in the "explore" mode (and the page they go to) are generated? Where does the mapping between http://mysite.com/image/uV7VG to the actual .jpg file happen?

It happens in getSrcTargetSingle at app/class/class.image.php but you won't get anything by knowing that. The thing that you must understand is that the URL at your external storage settings is placed before the path resolution so the only thing that you must do is:

1. Move the files from external server to a local server (not the same path at your main website, this is a must)
2. Point the subdomain (cname, whatever) to your local server and use the path used in the previous step
3. Deactivate the original external storage server

If you use cpanel this is trivial as create a sub-domain, map it to /sub-domain and put the files right there. Which files or folder structure? If you used S3 then the contents inside the bucket must be right inside /sub-domain

 

[ubmm]

Thanks ... I think I have it working. I originally transferred the s3 bucket into the same images folder used by local storage. However, I then mapped the subdomain i.mysite.com to have the doc root as the /images folder and all appears to be working. I need to make sure DNS propagation has happened but I think I'm in good shape now.

 

[Rodolfo]

The reason for use a different folder is because there could be overrides but at this point that doesn't matter, you already moved the files meaning that you didn't faced any file override.