Using Chevereto to view existing collection of pictures + bulk import

[f.d.roscamabbing]

Hi,
I am trying to set up Chevereto as a front-end for viewing a large collection of family pictures. There are some things I'd like to clarify before completing setup and potentially proceeding with something that does not work.

My situation: I have a file server at home with 10's of GBs of family pictures. These are already sorted in a folder tree structure.
I also have an external website at a commercial web hosting service (with limited storage space, so I cannot store all pics there) where I host Chevereto V3 (licensed version).

My goal is to have Chevereto index and serve all the pictures on my home server ("external" relative to the website). Ideally it would store all thumbnails and metadata locally and serve the pictures from the home server upon request. Is this at all possible using Chevereto?

I set up my home file server to serve SFTP access to the pictures, and I setup Chevereto to connect to that SFTP as external storage. To make Chevereto actually index the pictures on the home server, I have to import them I presume. I looked at the bulk importer documentation and example (which is a bit minimalistic I must say) and I am guessing that I must now re-organize the folder structure of the pictures to be in accordance with the example structures given. I am further guessing that then I run the bulk importer php script. it will look in all storages for importable folder strucures, find them in the external storage and start importing (i.e. for each picture it finds, it adds it to the database and creates a thumbnail. Some questions:

• Is this indeed how bulk importing works?
• Where are the thumbnails stored?
• After importing files, are they copied and stored "locally" or do the files link to the original "external" location?

In general: will what I describe above work at all or is there a better way to achieve my goal?

Thanks for your help!
Folkert

 

[f.d.roscamabbing]

Just managed to get 3.20.2 installed as a fresh install after upgrading unfortunately failed. I hadn't really started using it yet luckily. Now I see the bulk importer tab on the dashboard, this clarifies things a bit more: the importer looks for files in the "importing" folder on the web root.

Is it also possible to import from external storage somehow?

I am considering if it's maybe possible to make a symlink or something to make the local folder appear as if it's the external folder, though that would be a workaround with some caveats I fear.

It would take a really long time to upload everything and store it again on the external storage.

 

[Rodolfo]

Bulk content importer | V3 Docs

Documentation for Chevereto image hosting software (V3)

v3-docs.chevereto.com

The docs tell you that the system works as an importer, it takes files from one path and "imports" it into the system. You have to realize that we distribute uploads among N external storage servers, this wasn't made for just one server that's why the bulk importer will take those files and distribute it towards all your configured external storage servers. We do that because that is cheaper when you get millions of images, the concept of a sync utility is too limited for how we distribute storage. For us, sync should be just one special type of storage which doesn't distribute (I now, I'm boring you from this point).

What you could do is to RSYNC a copy from server to server, or lets say you create the copy in the external server and then you mount that ready-to-import directory into /importing using fuse or any network based filesystem. Kindly note that network will add an horrible I/O to the process, ideally you want to handle this on site but at the end it all narrows down to how many pictures are.

 

[f.d.roscamabbing]

Thanks for the quick reply.

So if I understand it correctly, "importing" here means that it adds the picture to the database, creates a thumbnail and it stores a copy of the file in the external storage if that is defined (or in one of them if multiple are defined).
In any case, for each file a round trip to and from the web server will be needed (now that I think of it, it would be needed in any case to create a thumbnail).

Am I correct to presume that after importing is complete, the files can be removed from importing directory and will have been stored elsewhere (i.e. in external storage in this case)?

 

[Rodolfo]

Yes, Chevereto actually removes all the files from the importing path and store the files in the external storage servers. It doesn't store the images in the database, just their mappings (the image src is on the storages).