• Welcome to the Chevereto User Community!

    Here, users from all over the world come together to learn, share, and collaborate on everything related to Chevereto. It's a place to exchange ideas, ask questions, and help improve the software.

    Please keep in mind:

    • This community is user-driven. Always be polite and respectful to others.
    • Support development by purchasing a Chevereto license, which also gives you priority support.
    • Go further by joining the Community Subscription for even faster response times and to help sustain this space
  • Chevereto Support CLST

    Support response

    Support checklist

    • Got a Something went wrong message? Read this guide and provide the actual error. Do not skip this.
    • Confirm that the server meets the System Requirements
    • Check for any available Hotfix - your issue could be already reported/fixed
    • Read documentation - It will be required to Debug and understand Errors for a faster support response

Duplicate uploads when using remote upload

Status
Not open for further replies.
demontaihack

demontaihack

Chevereto Member
i use remote upload with multiple URLs, When choosing the images to be uploaded i remove some unwanted images and here's the list that i used:

Code: 
https://www.goodfreephotos.com/albums/other-landscapes/time-lapse-landscape-at-dusk-with-water-sand-and-tree.jpg
https://www.goodfreephotos.com/albums/other-landscapes/sunset-behind-trees-in-the-river-landscape.jpg
https://www.goodfreephotos.com/albums/other-landscapes/forest-and-hills-landscape-with-road.jpg
https://www.goodfreephotos.com/albums/other-landscapes/girl-looking-at-mountains-landscape.jpg
https://www.goodfreephotos.com/albums/other-landscapes/sky-and-night-landscape-with-moon-and-birds.jpg
https://www.goodfreephotos.com/albums/other-landscapes/herd-and-pasture-with-sheep-and-mountains-landscape.jpg
https://www.goodfreephotos.com/albums/other-landscapes/rainbow-behind-the-mountain-landscape.jpg
https://www.goodfreephotos.com/albums/other-landscapes/rock-formation-landscape-and-scenery.jpg
https://www.goodfreephotos.com/albums/other-landscapes/dusk-skies-and-landscape.jpg
https://www.goodfreephotos.com/albums/other-landscapes/mountains-with-lake-majestic-landscape.jpg
https://www.goodfreephotos.com/albums/other-landscapes/red-sunset-over-the-lake.jpg
https://www.goodfreephotos.com/albums/other-landscapes/futuristic-cityscape-at-night.jpg
https://www.goodfreephotos.com/albums/other-landscapes/cowboys-in-the-landscape-with-light-exploding-from-trees.jpg
https://www.goodfreephotos.com/albums/other-landscapes/landscape-of-the-lake-with-canoe-sky.jpg
https://www.goodfreephotos.com/albums/other-landscapes/aerial-view-bridge-and-trees.jpg
https://www.goodfreephotos.com/albums/other-landscapes/landscape-with-mountains-sea-and-bay.jpg
https://www.goodfreephotos.com/albums/other-landscapes/rover-and-landscape-scenery.jpg
https://www.goodfreephotos.com/albums/other-landscapes/snowy-ranch-landscape.jpg
https://www.goodfreephotos.com/albums/other-landscapes/clouds-over-the-forest-landscape.jpg
https://www.goodfreephotos.com/albums/other-landscapes/high-waterfalls-and-landscape-in-nature.jpg

Code: 
https://www.goodfreephotos.com/albu...andscape-at-dusk-with-water-sand-and-tree.jpg
https://www.goodfreephotos.com/albu...unset-behind-trees-in-the-river-landscape.jpg
https://www.goodfreephotos.com/albums/other-landscapes/forest-and-hills-landscape-with-road.jpg
https://www.goodfreephotos.com/albums/other-landscapes/girl-looking-at-mountains-landscape.jpg
https://www.goodfreephotos.com/albu...y-and-night-landscape-with-moon-and-birds.jpg
https://www.goodfreephotos.com/albu...asture-with-sheep-and-mountains-landscape.jpg
https://www.goodfreephotos.com/albums/other-landscapes/rainbow-behind-the-mountain-landscape.jpg
https://www.goodfreephotos.com/albums/other-landscapes/rock-formation-landscape-and-scenery.jpg
https://www.goodfreephotos.com/albums/other-landscapes/dusk-skies-and-landscape.jpg
https://www.goodfreephotos.com/albums/other-landscapes/mountains-with-lake-majestic-landscape.jpg
https://www.goodfreephotos.com/albums/other-landscapes/red-sunset-over-the-lake.jpg
https://www.goodfreephotos.com/albums/other-landscapes/futuristic-cityscape-at-night.jpg
https://www.goodfreephotos.com/albu...landscape-with-light-exploding-from-trees.jpg
https://www.goodfreephotos.com/albums/other-landscapes/landscape-of-the-lake-with-canoe-sky.jpg
https://www.goodfreephotos.com/albums/other-landscapes/aerial-view-bridge-and-trees.jpg
https://www.goodfreephotos.com/albums/other-landscapes/landscape-with-mountains-sea-and-bay.jpg
https://www.goodfreephotos.com/albums/other-landscapes/rover-and-landscape-scenery.jpg
https://www.goodfreephotos.com/albums/other-landscapes/snowy-ranch-landscape.jpg
https://www.goodfreephotos.com/albums/other-landscapes/clouds-over-the-forest-landscape.jpg
https://www.goodfreephotos.com/albums/other-landscapes/high-waterfalls-and-landscape-in-nature.jpg

The thing is looking at the results you will see there are some images that are the same but with different titles. Please look at me screenshot and see my point:

https://demo.chevereto.com/i/ORc

Besides, get back to original site, i didn't see any duplicate images. So i 'd say there's something wrong with the code. I am using . Chevereto version3.8.13

Thanks.
 
There are several bugs causing this issue.

At first, there is a bug in the PHP function that checks for image integrity, so all those pasted URLs that look like this (notice the ... dots):
Code: 
https://www.goodfreephotos.com/albu...y-and-night-landscape-with-moon-and-birds.jpg
Pass the validation, causing a E_WARNING (division by zero). This is easy to fix, actually I've already patched the demo with it.

The other bugs are on the client side (JavaScript), like the wrong file queue count handling which causes the system to send request for all the pasted URLs, not just the applicable image files. This has been also already patched in the demo.

However, I'm still seeing weird stuff going because I've a 1:1 local/demo installation and in local I'm not seeing anything wrong, most likely there is some nasty bug causing it. Maybe is server related (Apache vs Nginx), at this point I'm still rumbling around.

Anyway, thanks for report this. I'm working on it.

Cheers,
Rodolfo.
 
Found it.

Bug happens when you use ID based file-naming and the request gets a connection error or something like that (in demo I'm getting "Connection reset by peer" issue, still figuring out why). Thing is that for some reason my nginx is failing which is causing a connection reset error and it seems that this happens just after Chevereto reserved the ID, so the next request is tied to a ID already reserved before, system doesn't know that the previous request died and that the reservation must be deleted.

This was though to narrow down, and I'm still working on it. If you want to avoid this issue, just don't use ID based file-naming.
 
Dear Rodolfo,

Even when I change file-naming to mix original +id there're still some errors (show a loading icon and never finish loading) with another list of URLs and the same on demo.chevereto.com:

  1. https://negativespace.co/wp-content...ve-space-underground-subway-train-station.jpg
  2. https://negativespace.co/wp-content/uploads/2017/04/negative-space-waiting-underground-station.jpg
  3. https://negativespace.co/wp-content...commuter-escalator-indoors-train-station.jpeg
  4. https://negativespace.co/wp-content...pace-train-tunnel-train-station-platform.jpeg
Meanwhile, i will test with the old list of URLs and get back to you soon.
Many thanks.
 
There are several bugs related to this, try to test on the demo where I'm applying these patches. At least at my end I'm not having the duplicate issue.

By the way, are you running nginx?
 
Found it, is a memory issue. Log was at kern.log

Code: 
Apr 23 11:40:05 demo kernel: Out of memory: Kill process 23164 (php-fpm7.1) score 426 or sacrifice child
Apr 23 11:40:05 demo kernel: Killed process 23164 (php-fpm7.1) total-vm:694728kB, anon-rss:211604kB, file-rss:1608kB
Apr 23 11:40:05 demo kernel: php-fpm7.1: page allocation failure: order:0, mode:0x201da

To summarize:
  • ID file-naming is very sensitive to the system reliability. Regarding if the thing die for any server issue, next request shouldn't be affected for that (still working on that).
  • The memory issue was caused by a huge load (the max. image size option does an extra resize every time a big image gets uploaded)
  • Increasing ram from 512MB to 1GB solved the issue for me. I'm pretty sure that files 2X bigger will cause same issues.
  • Remote uploader had a bug related to how it handles deletions from queue (fixed on demo)
  • PHP image integrity check had a bug (fixed on demo)
What I will do here is a file lock system to avoid ID conflicts, most likely that will get rid of the duplicate stuff.

I will keep working on this.
 
Fixed it, I end up using file based locks instead of DB locks for ID reservations. Patch is live on the demo.
 
I am using LiteSpeed server (Hawkhost). I have just tested with Negative links above on your demo and it's not working, nothing happens.
 
What is a "negative" link?

Anyway, works for me so it must be cache at the your end.
 
I mean it's not working with the links above ( from Negative Space site):

https://negativespace.co/wp-content...ve-space-underground-subway-train-station.jpg
https://negativespace.co/wp-content/uploads/2017/04/negative-space-waiting-underground-station.jpg
https://negativespace.co/wp-content...commuter-escalator-indoors-train-station.jpeg
https://negativespace.co/wp-content...pace-train-tunnel-train-station-platform.jpeg

Did you try with these links? i have tested many times with your demo and after submitting those links, nothing happened and i tried with the private mode of chrome.
 
Status
Not open for further replies.
Back
Top