Different upload behavior after upgrading from version 3.18.3 to 3.20.X [With included Workaroud]

zwame · Jul 5, 2021

▶ Reproduction steps

Go to /upload and upload a X image

😢 Unexpected result

📃 Error log message

We upgraded our Chevereto from version 3.18.3 to 3.20.8, and we noticed that the upload behavior changed, causing an error when someone uploads an image.
Just a note. I'm not sure if this is a Chevereto bug, a php bug, a GD bug or a FreeBSD bug.

Chevereto error:

Code:

Error report
AMD2.jpg -  errorId #9fd5c581ccb3d8a5

Php-Fpm error:

Code:

** errorId #9fd5c581ccb3d8a5 **
>> ErrorException [0]: rename(/tmp/chvtempiVFl0z,/var/www/img/images/2021/07/05/AMD2.jpg): Operation not permitted
At /app/lib/classes/class.upload.php:197

Stack trace:
#0 unknown file(unknown line): G\errorsAsExceptions(2, 'rename(/tmp/chvtempiVFl0z,/var/www/img/images/2021/07/05/AMD2.jpg): Operation not permitted', '/app/lib/classes/class.upload.php', 197, Array)
#1 /app/lib/classes/class.upload.php(197): rename('/tmp/chvtempiVFl0z', '/images/2021/07/05/AMD2.jpg')
#2 /app/lib/classes/class.image.php(504): CHV\Upload->exec()
#3 /app/lib/classes/class.image.php(647): CHV\Image::upload(Array, '/images/2021/07/05/', NULL, Array, NULL, true)
#4 /app/routes/route.json.php(88): CHV\Image::uploadToWebsite(Array, Array, Array)
#5 /lib/G/classes/class.handler.php(230): G\Handler->{closure}(G\Handler)
#6 /lib/G/classes/class.handler.php(130): G\Handler->processRequest()
#7 /app/web.php(411): G\Handler->__construct(Array)
#8 /app/loader.php(230): require_once('/app/web.php')
#9 /index.php(20): include_once('/app/loader.php')

There’s no Nginx error.

We run Nginx with the user “www” and the Chevereto php-fpm pool runs with user “img”. The user “img” belongs to the “www” group.

The images go to a folder inside the Chevereto directory site named “images”, that have read, write and execute permissions to the users “www” and “img”, recursively.

Code:

felix@webprojectos:~ % doas ls -lah /var/www/img/
total 456
drwxrwx---   9 img  www      16B Jun 25 23:47 .
drwxrwx--x  12 www  wheel    12B Oct 23  2019 ..
drwxrwx---   5 img  www       5B Jan  5 12:32 images

The /tmp folder has all permissions to all users:

Code:

felix@webprojectos:~ % doas ls -lah /
total 2345
drwxr-xr-x  19 root  wheel    25B Jun  9 01:02 .
drwxr-xr-x  19 root  wheel    25B Jun  9 01:02 ..
drwxrwxrwt   7 root  wheel   1.0K Jul  5 17:47 tmp

When an image was uploaded with Chevereto Version 3.18.3, there was no error, and the user “img” had read and write permissions, user “www” read permission, and “other” read permissions. Also note that chevereto created the "thumbnail" version of the image.

Code:

felix@webprojectos:~ % doas ls -lah /var/www/img/images/2021/06/07
total 54
drwxrwx---  2 img  www     6B Jun  7 16:17 .
drwxrwx---  7 img  www     7B Jun 28 17:03 ..
-rw-r--r--  1 img  www   3.2K Jun  7 16:17 2021_06-Apple_WWDC_2021.png
-rw-r--r--  1 img  www   4.0K Jun  7 16:17 2021_06-Apple_WWDC_2021.th.png
-rw-r--r--  1 img  www   454B Jun  7 16:16 amazonprime.png
-rw-r--r--  1 img  www   3.7K Jun  7 16:16 amazonprime.th.png

With Chevereto Version 3.20.8, Chevereto gives an error, but the image is uploaded. The difference is that only the php-fpm user, “img” has read and write permissions. “www” and “other” users don’t have any permission. Also, the "thumbnail" and "medium" version are not created.
This behavior happens uploading in all Browsers.

Code:

felix@webprojectos:~ % doas ls -lah /var/www/img/images/2021/07/05
total 2428
drwxrwx---  2 img  www     4B Jul  5 17:42 .
drwxrwx---  3 img  www     3B Jul  5 15:53 ..
-rw-------  1 img  www   752K Jul  5 17:42 AMD2.jpg

WORKAROUND
If I add the fallowing configuration to the Chevereto php-fpm pool, the issue is resolved:

Code:

env[TMP] = /var/www/img/temp
env[TMPDIR] = /var/www/img/temp
env[TEMP] = /var/www/img/temp

Here is the configuration of our Chevereto php-fpm pool, with the workaround:

Code:

[img]
user = img
group = img
listen = /var/run/php-fpm-img.sock
listen.owner = www
listen.group = www
php_admin_value[disable_functions] = exec,passthru,shell_exec,system
php_admin_flag[allow_url_fopen] = off
env[TMP] = /var/www/img/temp
env[TMPDIR] = /var/www/img/temp
env[TEMP] = /var/www/img/temp
pm = dynamic
pm.max_children = 200
pm.start_servers = 10
pm.min_spare_servers = 5
pm.max_spare_servers = 35
pm.max_requests = 100
request_terminate_timeout = 0
catch_workers_output = yes
chdir = /
php_admin_value[error_log] = /var/log/php-fpm/img.error.log
php_admin_value[upload_max_filesize] = 50M
php_admin_value[post_max_size] = 50M
php_admin_value[max_execution_time] = 30
php_admin_value[memory_limit] = 512M

With this workaround, the error disappears, and the image, "thumbnail" and "medium" image are created without problems and with the usual permissions:

Code:

felix@webprojectos:~ % doas ls -lah /var/www/img/images/2021/07/05
total 4651
drwxrwx---  2 img  www    11B Jul  5 23:21 .
drwxrwx---  3 img  www     3B Jul  5 15:53 ..
-rw-------  1 img  www   752K Jul  5 17:42 AMD2.jpg
-rw-------  1 img  www   249K Jul  5 21:12 EX2-ALL.jpg
-rw-------  1 img  www    96K Jul  5 18:09 F618R2-RTPT__rear_lines.jpg
-rw-r--r--  1 img  www   379K Jul  5 23:21 Supermicro-X7SPA-HF-Atom-D510-ICH9R-6xSATA-RAID-2xSODIMM-miniITX-i-id13473-i-img_1orig-1288182340768.jpg
-rw-r--r--  1 img  www   134K Jul  5 23:21 Supermicro-X7SPA-HF-Atom-D510-ICH9R-6xSATA-RAID-2xSODIMM-miniITX-i-id13473-i-img_1orig-1288182340768.md.jpg
-rw-r--r--  1 img  www    19K Jul  5 23:21 Supermicro-X7SPA-HF-Atom-D510-ICH9R-6xSATA-RAID-2xSODIMM-miniITX-i-id13473-i-img_1orig-1288182340768.th.jpg
-rw-------  1 img  www   427K Jul  5 15:53 VIA_EPIA_M-Series_Mini-ITX_board_image_top.jpg
-rw-------  1 img  www    64K Jul  5 21:39 fit-pc.jpg
-rw-------  1 img  www    68K Jul  5 21:30 sapphire.jpg

The difference is that the temporary folder is now located in the same directory and ZFS dataset of the Chevereto website.
I just want to reinforce the the "/tmp" folder has all permissions to all users, and i saw that Chevereto could create temporary and session files in the system tmp folder.

We are running Chevereto inside a FreeBSD 12.2 machine, with nginx 1.18, php 7.4.20 and MariaDB 10.4.19.

As I said initially, I'm not certain this is a Chevereto bug, but I thought it was good to report it here.

Thanks.

Rodolfo · Jul 5, 2021

Thanks for a very complete report.

In V3.20.8 there's no wrapper for that failure. Note: https://www.php.net/manual/en/function.rename.php#117590

UNIX-like operating systems, filesystems may be mounted with an explicit uid and/or gid (for example, with mount options "uid=someuser,gid=somegroup"). Attempting to call rename() with such a destination filesystem will cause an "Operation not permitted" warning, even though the file is indeed renamed and rename() returns (bool) true.

app/lib/classes/class.upload.php

Change this:

PHP:

$uploaded = rename($this->downstream, $this->uploaded_file);

To this:

PHP:

        try {
            $uploaded = rename($this->downstream, $this->uploaded_file);
        } catch(Throwable $e) {
            $uploaded = file_exists($this->uploaded_file);
        }

Does it works? It should work without any server modification.

zwame · Jul 6, 2021

Thanks for the very complete report.

In V3.20.8 there's no wrapper for that failure. Note: https://www.php.net/manual/en/function.rename.php#117590

UNIX-like operating systems, filesystems may be mounted with an explicit uid and/or gid (for example, with mount options "uid=someuser,gid=somegroup"). Attempting to call rename() with such a destination filesystem will cause an "Operation not permitted" warning, even though the file is indeed renamed and rename() returns (bool) true.

Click to expand...

Thank you for your answer.
Ok, that makes sense, because “/tmp” and “/var/www/img” are two separate ZFS datasets and each one has its own mount point.

Code:

felix@webprojectos:~ % doas mount
zroot/tmp on /tmp (zfs, local, noatime, nosuid, nfsv4acls)
zroot/var/www/img on /var/www/img (zfs, local, noatime, nfsv4acls)

app/lib/classes/class.upload.php

Change this:
PHP:
$uploaded = rename($this->downstream, $this->uploaded_file);

To this:
PHP:
try {
$uploaded = rename($this->downstream, $this->uploaded_file);
} catch(Throwable $e) {
$uploaded = file_exists($this->uploaded_file);
}

Does it works? It should work without any server modification.

Thanks. Yes, it worked perfectly.

An example:
https://img.zwame.pt/images/2021/07/06/sapphire.jpg
https://img.zwame.pt/images/2021/07/06/sapphire.th.jpg
https://img.zwame.pt/images/2021/07/06/sapphire.md.jpg

I removed the “temp env” definitions from php-fpm conf and Cheveretto didn’t gave an error when I uploaded the image and created the “medium” and “thumbnail” images.
The permissions of the files are also ok

Code:

felix@webprojectos:~ % doas ls -lah /var/www/img/images/2021/07/06
total 245
drwxrwx---  2 img  www     5B Jul  6 14:47 .
drwxrwx---  4 img  www     4B Jul  6 14:47 ..
-rw-r--r--  1 img  www    68K Jul  6 14:47 sapphire.jpg
-rw-r--r--  1 img  www    17K Jul  6 14:47 sapphire.md.jpg
-rw-r--r--  1 img  www   5.0K Jul  6 14:47 sapphire.th.jpg

The comment in the php page referred that this is not a php bug, but gave that workaround. Can this be considered a Chevereto bug?
Is it possible to implement that change in future releases, without affecting other Chevereto Sites or can I maintain that change in our Chevereto Site, without fear that future releases modify that part of the code?

Thank you one more time for your kind answer.

Rodolfo · Jul 6, 2021

zwame said:
Can this be considered a Chevereto bug?

Sorry for not making that more clear, of course that this is a bug.

It happened because in V3.20 we abandoned the old PHP warning system in favour of Exceptions. This got in the middle, we made a mistake there that we couldn't anticipate earlier. I'm very sorry about it.

The change has been already committed for V3.20.9 which will be available within this week so this won't be an issue for anybody else.

Again, many thanks for reporting this.

Different upload behavior after upgrading from version 3.18.3 to 3.20.X [With included Workaroud]

zwame

Chevereto Member

Rodolfo

👑 Chevereto Godlike

zwame

Chevereto Member

Rodolfo

👑 Chevereto Godlike