• Welcome to the Chevereto user community!

    Here users from all over the world gather around to learn the latest about Chevereto and contribute with ideas to improve the software.

    Please keep in mind:

    • 😌 This community is user driven. Be polite with other users.
    • 👉 Is required to purchase a Chevereto license to participate in this community (doesn't apply to Pre-sales).
    • 💸 Purchase a Pro Subscription to get access to active software support and faster ticket response times.

Same images appearing on at least two accounts, other is overwritten

Status
Not open for further replies.
H

Hujasmaa

Chevereto Member
Website URL
<private>

Chevereto version
3.10.7

Description of the issue
Last year we had our Chevereto self hosted on our own vps and all the images stored at S3. In front of the site we had CloudFlare because we used it for https and to reduce the amount of data transfer from S3.

Last fall there occurred several cases when users reported that their images appeared on some other users account and vice versa. Also the preview images could be other than the real image.

I opened a tech support ticket here. The explanation was that it might be a server-related flaw so it wasn't covered with this support. Also it was thought the cache was guilty of false preview images.

I couldn't continue to let users to upload any more images so I closed the upload. I figured out I should move our site to TMD Hosting to be sure the server misconfiguration or such can be ruled out.

TMD Hosting refused to ingest our images (10,5 GB) so we set up a new SFTP vps and copied all the images from S3 there. It was set up to be in the same subdomain url as the S3 image storage earlier. S3 was switched off and the new vps on in the Chevereto's settings for external storage.

TMD Hosting transferred the site to their servers. They installed a Let'sEncrypt ssl so the CloudFlare was not necessary.

The new site functioned ok, all the images could be browsed, etc. and there was not any new mixed account images, just the old ones.

I asked them to set the correct email settings and update Chevereto to the latest revision, because they market the hosting as "fully managed Chevereto approved hosting". They made the update.

Today I decided to test everything is working ok. I made a new album in my own account and uploaded 381 images to it. The upload went fine and all the images appeared in my new album.

However, they also appeared on at least one other users albums (several albums) and they seem to have overwritten some content there. I purged the image cache of Google Chrome, but it didn't change anything. The images can still be seen on the both accounts.
 
I forgot, here's a quote from TMD Hosting's tech support:

"We would like to inform you that we have reviewed your case but the issue does not seem to be related to the functionality of your hosting account with us as it is properly set and configured.

Therefore, what we could suggest you is to contact your website's developer or any local web developer and ask them to review your website so that you could be sure that all your website's settings and paths are defined correctly. "
 
In any case, please note that any server issue prior 3.10.6 (false positive images) can't be fixed. These problems are related to how the id based filenaming used to work and I just changed that in 3.10.6. Anything prior that can't be fixed.

If you still have issues please share all the details, image links, everything. I won't dig this stuff, you have to show me everything. You just throw me two listing links. I need each individual image (at least).
 
All the image links and login credentials were written in the last field when writing the ticket.

I can live with the old faults, but there should be some kind of way to fix what is happening now.
 
You sent me two links of two listings, not the conflicting images. I won't dig any of your listings trying to find those images.
 
Ok, I will continue with the image links tomorrow, thanks. Now I just have to sleep as it is nighttime here.
 
I got your links but I don't know which image of those image pairs is the "correct" image.

Is "borg" the actual image or is "DJEzgo84" the actual image?
 
The actual image is uploaded by user borg as you can see from the hidden field which was filled when creating the ticket.

I don't mean to be difficult. If something is missing, please help me to clarify (as you do have done).

Thanks!
 
Ok, so the issue is that old images are getting replaced by new images? Like uploads from "borg" user are overwriting old images from "DJEzgo84"?
 
Yes. They seem to appear on other users account as well when I upload them to my own account.

This has happened last fall in smaller scale, but now there are tens or even hundreds of my images duplicated to another user's albums.
 
Most likely this is dragged from old IDs. Basically, the old ID system worked awfully in your website so there are tons of orphans (bad ID pairs) and when the system fulfill an actual ID, old (bad IDs) cause this issue.

Here is the proof:

/image/5HevP from user "borg" returns all its URIs containing 5HevP (which is the encoded ID). That's OK and is an image uploaded like one day ago. No issues there.

/image/5Ybg8 from "DJEzgo84" returns URIs containing 5HevP like 5HevP.jpg which is wrong but this happened months ago. Somehow, this image grabbed "5HevP" then when "borg" claimed this ID, it altered the image tied to this record.

The most awkward stuff here is that there is a gap of 287 IDs from 5Ybg8 to 5HevP (don't worry, I won't disclose the actual numeric IDs) and this tells me how big was the gap caused by the old ID reservation system in your server. It seems that the system just has a bunch of garbage in between caused by the ID reservation system that wasn't working as expected back then on your server.

Thing is that the encoded ID is sequential, so if an image of 4 months ago has this issue, there could be hundreds or more images susceptible to this issue. It is very hard to tell how big this could be and most likely it will stop happening when the N gap gets filled.

I have applied a fix for you that will get rid of this issue. I changed your chv_images AUTO_INCRIMENT from 9457 to 19000 so basically I'm adding a gap equals to the number of images you already have so there won't be any chance of collisions with old IDs. By doing this, the system is reserving IDs ~9500 numbers away from the possible messed IDs in the old system. I know that is way too much but I want to be sure that this won't trouble you any longer.

The old ID reservation system is gone and I will write an email informing users with this issue so they can apply this workaround to avoid future issues. It only applies to those that experienced collisions with the old ID reservation system.
 
So. If I understood correctly, the problem won't occur any more.

However, we have to manually correct the old mishaps. Every user has to delete the images which don't belong to them? That's not too overwhelming as I shut down the upload when the flaw occurred repeatedly.

There are backups available prior to my test upload batch of 381 images, which unfortunately messed up fellow user DjEzgo84:s albums. Would it be possible to do this:

I request backup restores from TMDHosting and my image storage provider. After that - could you make the same fix again?
 
Hujasmaa said:
If I understood correctly, the problem won't occur any more.
Click to expand...
That's right. If you want to be 100000% sure you can switch to the "original" or "mixed" file naming option under Dashboard > Settings > Image upload.

Hujasmaa said:
Every user has to delete the images which don't belong to them?
Click to expand...
Your database isn't that big, you could dump the ID+names and then run a script to pair ID+NAME with encodedID(). To be honest, that will need a larger explanation but if you are a coder/programmer I can give you these instructions.

Hujasmaa said:
After that - could you make the same fix again?
Click to expand...
Sure, you can get a backup so the old stuff won't be messed then you only need to run:
Code: 
ALTER TABLE chv_images AUTO_INCREMENT = 19000
Via the query console or using PHPMyAdmin (select table, operations, then fill the auto-increment value in the form).
 
Rodolfo said:
Your database isn't that big, you could dump the ID+names and then run a script to pair ID+NAME with encodedID(). To be honest, that will need a larger explanation but if you are a coder/programmer I can give you these instructions.
Click to expand...
I don't have the skills but I have a very capable server admin who has 10+ years of SQL experience. I think he might be able to do that if you provide the steps.

I can also ask him to do that table altering.

Rodolfo said:
That's right. If you want to be 100000% sure you can switch to the "original" or "mixed" file naming option under Dashboard > Settings > Image upload.
Click to expand...
That can be done any time with no problems?
 
The backup + auto increment workaround is way more straightforward. The other stuff needs to code two iterators.
 
Status
Not open for further replies.
Back
Top