• Welcome to the Chevereto user community!

    Here users from all over the world gather around to learn the latest about Chevereto and contribute with ideas to improve the software.

    Please keep in mind:

    • This community is user driven. Be polite with other users.
    • We recommend purchasing a Chevereto license to participate in this community.
    • Purchase a Community Subscription to get even faster ticket response times.
  • Chevereto Support CLST

    Support response

    Support checklist

    • Got a Something went wrong message? Read this guide and provide the actual error. Do not skip this.
    • Confirm that the server meets the System Requirements
    • Check for any available Hotfix - your issue could be already reported/fixed
    • Read documentation - It will be required to Debug and understand Errors for a faster support response

External storage images not getting deleted

M

mkerala

👽 Chevereto Freak
Recently noticed that direct links of images are still accessible even after deleting the images. Checked the storage location and files are still there despite image being deleted from the site and database. Tested by uploading a new image and deleting it. The file still remains in the external storage.

To confirm external storage account has the correct permission, logged into storage as the same user and was able to delete the file manually via SFTP. Tested on demo, but could not reproduce as demo is not using external storage.

▶🚶‍Reproduction steps
  1. Login as a user to a Chevereto site with external storage.
  2. Upload a any image file.
  3. Copy the direct image link.
  4. Delete the image and try accessing the image via direct link.
😢Unexpected result

The image is still active even after being deleted from the site and database.

📃Error log message

Enabled SFTP logs on the server and monitored all activity during upload and delete. Logs show activity for file being uploaded. But no activity when file is being deleted. Not even a connection attempt.
 
If the image is gone from the db is because it got an OK from the the external removal step. Does it happens with all storages? If not, then the issue is your external storage server.
 
I got two sftp file servers which has not had any changes being made for the last one year. Everything was working perfectly fine before I updated to 3.17.1 two days ago. I logged in via Filezilla as the same user account Chevereto uses and deleted the files. So there is no server side errors or permission issues.

Below is the full SFTP log. As you can see logs is showing all activity when the file was added. But nothing, not even a connection attempt when the image was deleted.

Code: 
Dec 30 13:28:15 storage sftp-server[2987675]: session opened for local user chevftp from [94.130.xxx.xx]
Dec 30 13:28:15 storage sftp-server[2987675]: received client version 3
Dec 30 13:28:15 storage sftp-server[2987675]: realpath "."
Dec 30 13:28:15 storage sftp-server[2987675]: opendir "/var/www/html/s2/images"
Dec 30 13:28:15 storage sftp-server[2987675]: closedir "/var/www/html/s2/images"
Dec 30 13:28:15 storage sftp-server[2987675]: open "/var/www/html/s2/images/DyWSDqsdQSDqsdqsdHG.png" flags WRITE,CREATE,TRUNCATE mode 0666
Dec 30 13:28:15 storage sftp-server[2987675]: close "/var/www/html/s2/images/DyWSDqsdQSDqsdqsdHG.png" bytes read 0 written 175389
Dec 30 13:28:15 storage sftp-server[2987675]: open "/var/www/html/s2/images/DyWSDqsdQSDqsdqsdHG.md.png" flags WRITE,CREATE,TRUNCATE mode 0666
Dec 30 13:28:15 storage sftp-server[2987675]: close "/var/www/html/s2/images/DyWSDqsdQSDqsdqsdHG.md.png" bytes read 0 written 457652
Dec 30 13:28:15 storage sftp-server[2987675]: open "/var/www/html/s2/images/DyWSDqsdQSDqsdqsdHG.th.png" flags WRITE,CREATE,TRUNCATE mode 0666
Dec 30 13:28:15 storage sftp-server[2987675]: close "/var/www/html/s2/images/DyWSDqsdQSDqsdqsdHG.th.png" bytes read 0 written 34684
Dec 30 13:28:15 storage sftp-server[2987675]: session closed for local user chevftp from [94.130.xxx.xx]
Dec 30 13:28:46 storage sftp-server[2987588]: realpath "/var/log/sftp.log"
Dec 30 13:28:46 storage sftp-server[2987588]: stat name "/var/log/sftp.log"
Dec 30 13:28:47 storage sftp-server[2987588]: open "/var/log/sftp.log" flags READ mode 0666
 
I got another Chevereto installation running v3.15.2. Tested the same external storage with the same settings(same URL too) and images are getting deleted.
 
I was able to test on a Chevereto 3.17.0 install of a friend. Added the same external storage and was able to replicate the issue.

Also found auto expire image is not working as well.
  • Affects only SFTP external storage.
  • Affects version 3.17 or higher
  • Will break the auto expire
I will wait for others using external storage to test this.
 
I tested this my own site v3.17.1 as I also use SFTP external storage, and I found once the image is deleted it takes 1 minute 15 seconds to be removed from the SFTP server, I have tested this 3 times now, hope this helps.
 
HenrysCat said:
I tested this my own site v3.17.1 as I also use SFTP external storage, and I found once the image is deleted it takes 1 minute 15 seconds to be removed from the SFTP server, I have tested this 3 times now, hope this helps.
Click to expand...
Thanks for testing. In my case the file stays there forever.

Did some further testing by adding a Hostgator SFTP as external storage. Same thing, file stays there for ever.

Another issue I noticed is the stats are not getting updated as well. If I click on search content, there is no image, yet shows 3.14 MB used.

1609395788278.png
 
The Active box should be ticked right? also what are the permissions and owner of the storage folder?

Permissions should be 755 and owner/group should the user with the highest permissions in /etc/passwd
 
Try adding the storage folder in a directory that is owned the SFTP user and not the default /var/www/html/
 
Last edited:
It's not, your logs posted above show /var/www/html/s2/images/ where /var/www/html/ is the default web folder, a user folder will be something along the lines of /home/user/web/public_html or similar.
 
HenrysCat said:
It's not, your logs posted above show /var/www/html/s2/images/ where /var/www/html/ is the default web folder, a user folder will be something along the lines of /home/user/web/public_html or similar.
Click to expand...
I can't use home folder as it is not mapped to Nginx and image won't load. /var/www/html/s2/images/ this folder is owned by the sftp user.

I even tried with Hostgator sftp and got the same results. So I don't think its an issue with the server. Besides the same folder and user works fine with my 3.15 Chevereto install.
 
mkerala2 said:
I can't use home folder as it is not mapped to Nginx and image won't load. /var/www/html/s2/images/ this folder is owned by the sftp user.
Click to expand...
Use a Symlink and don't forget to set disable_symlinks off; in nginx.conf

Do you use a control panel?
 
Hello,

I think you have the same problem as me, check your DB, go to "chv_queues" are cron jobs in "pending" status?
My images are removed from website "system" but not physically from external storage.

Cron job errors

Hi, I am having problems with cron jobs, all jobs are in "pending" status and images are not physically deleted from the server errors from cron testing: PHP Warning: Undefined array key "session.save_path" in /home/www/public_html/i/lib/G/G.php on line 86 PHP Stack trace: PHP 1. {main}()...
chevereto.com chevereto.com
 
evil said:
Hello,

I think you have the same problem as me, check your DB, go to "chv_queues" are cron jobs in "pending" status?
My images are removed from website "system" but not physically from external storage.

Cron job errors

Hi, I am having problems with cron jobs, all jobs are in "pending" status and images are not physically deleted from the server errors from cron testing: PHP Warning: Undefined array key "session.save_path" in /home/www/public_html/i/lib/G/G.php on line 86 PHP Stack trace: PHP 1. {main}()...
chevereto.com chevereto.com
Click to expand...
He already mentioned that the database is being updated, is just the actual files the problem. Unless he did a wrong check and the queues contain the failed attempts.
 
I just tested Amazon S3 cron removal and it works. Like I said before, try using other external storage (api) to determine where the issue is. If you keep testing SFTP you don't bring anything different to the process and you won't be able to tell any difference.

Perhaps the issue is the implentation, certificates, the cron user, whatever. Bottom line is do some change and report back, otherwise we will here forever running in circles.
 
evil said:
Hello,

I think you have the same problem as me, check your DB, go to "chv_queues" are cron jobs in "pending" status?
My images are removed from website "system" but not physically from external storage.

Cron job errors

Hi, I am having problems with cron jobs, all jobs are in "pending" status and images are not physically deleted from the server errors from cron testing: PHP Warning: Undefined array key "session.save_path" in /home/www/public_html/i/lib/G/G.php on line 86 PHP Stack trace: PHP 1. {main}()...
chevereto.com chevereto.com
Click to expand...
Thanks. This pointed me in the right direction. Indeed the jobs were pending for storage delete.

Chevreto 3.16 introduced a new requirement for a cron job https://v3-docs.chevereto.com/setup/requirements.html#cron. Even though the image get removed from DB, storage delete are queued for Cron job. This explains why it takes up to 2 mins for the actual file to be deleted.

Added cron job as root and queue started clearing. Took a while as there was one week of pending delete jobs.
 
Last edited:
You must log in or register to reply here.
Back
Top