Bulk importer

[SunnyBlueSkies]

Since we're at it, it's hard for me to replicate but for some reason the bulk importer doesn't seem to function anymore after this update.

I'm simply using the parase-user portion but the system glitches or continues to say it's working on importing the files but with no results.

 

[Rodolfo]

Try flushing it, at Dashboard > Bulk importer

 

[SunnyBlueSkies]

From the error log:

[672cf313c2caf] ErrorException thrown in /home/sunnyblu/public_html/app/legacy/routes/importer-jobs.php:0

# Message [Code #1]
Allowed memory size of 536870912 bytes exhausted (tried to allocate 1120473088 bytes)

# URI
/importer-jobs/2/errors

# Method
GET

# Time
2024-11-07T17:04:19+00:00 [1730999059]

# Incident 672cf313c2caf

# Backtrace
------------------------------------------------------------
0 /home/sunnyblu/public_html/app/legacy/routes/importer-jobs.php:0
{main}()
------------------------------------------------------------
1
Chevere\ThrowableHandler\shutdownErrorAsException()
------------------------------------------------------------

[672cf32952f4a] ErrorException thrown in /home/sunnyblu/public_html/app/legacy/routes/importer-jobs.php:0

# Message [Code #1]
Allowed memory size of 536870912 bytes exhausted (tried to allocate 2147483648 bytes)

# URI
/importer-jobs/2/process

# Method
GET

# Time
2024-11-07T17:04:41+00:00 [1730999081]

# Incident 672cf32952f4a

# Backtrace
------------------------------------------------------------
0 /home/sunnyblu/public_html/app/legacy/routes/importer-jobs.php:0
{main}()
------------------------------------------------------------
1
Chevere\ThrowableHandler\shutdownErrorAsException()
------------------------------------------------------------

Exception thrown in /home/sunnyblu/public_html/app/src/Legacy/Classes/Storage.php:852

# Message
Storage id 8 doesn't exists

# Time
2024-11-07T19:10:21+00:00 [1731006621]

# Incident 672d109d3e369

# Backtrace
------------------------------------------------------------
0 /home/sunnyblu/public_html/app/src/Legacy/Classes/Storage.php:852
{main}()
------------------------------------------------------------
1 /home/sunnyblu/public_html/app/legacy/routes/json.php:1644
Chevereto\Legacy\Classes\Storage::migrateStorage()
------------------------------------------------------------
2 /home/sunnyblu/public_html/app/src/Legacy/G/Handler.php:455
Chevereto\Legacy\G\Handler->{closure}()
------------------------------------------------------------
3 /home/sunnyblu/public_html/app/src/Legacy/G/Handler.php:179
Chevereto\Legacy\G\Handler->processRequest()
------------------------------------------------------------
4 /home/sunnyblu/public_html/app/legacy/load/web.php:662
Chevereto\Legacy\G\Handler->__construct()
------------------------------------------------------------
5 /home/sunnyblu/public_html/app/legacy/entrypoints/index.php:44
require_once(string(length=50))
------------------------------------------------------------
6 /home/sunnyblu/public_html/index.php:12
require_once(string(length=59))
------------------------------------------------------------

 

[Rodolfo]

Allowed memory size of 536870912 bytes exhausted (tried to allocate 1120473088 bytes)

It's running out of memory, how big are the files you are importing?

Storage id 8 doesn't exists

Could you check at the database?

 

[Barry]

I ran into this issue also. In my case, the local disk of my server was over %95 full. Worth checking

 

[nsfwupload]

Ive just checked out this and concur that it does not work. I uploaded a 1000 images to no_parse and failed at 27 files

I then uploaded just 50 images to no-parse and again failed after just 27 files...

Log:

1731082396 - [Thread #0] No implicit categoryId property found, about to check category metadata object
1731082397 - [Thread #0] Image ID 554 inserted
1731082397 - [Thread #0] Inserted content, items++
1731082397 - [Thread #0] --------------------------------------------------------------------------------
1731082397 - [Thread #0] About to run out of time, breaking iteration now
1731082397 - [Thread #0] Chunked process ended (killed)
1731082397 - [Thread #0] ================================================================================
--
[OK] Automatic importing looped 2 times ~ /dashboard/bulk for stats

The imported images also cant be deleted, I've had to delete them via phpMyadmin

 

[anhthoai]

Same here. I see my php info set max_execution_time=3600 but chevereto dashboard always show only 30s. That maybe make importing run only 30s everytime. Could you please check it @Rodolfo

 

[Rodolfo]

@anhthoai

Thank you for your message. There was a bug causing the CLI to be restricted to 30 seconds. This issue has been fixed in Chevereto v4.2.4.