• Welcome to the Chevereto User Community!

    Here, users from all over the world come together to learn, share, and collaborate on everything related to Chevereto. It's a place to exchange ideas, ask questions, and help improve the software.

    Please keep in mind:

    • This community is user-driven. Always be polite and respectful to others.
    • Support development by purchasing a Chevereto license, which also gives you priority support.
    • Go further by joining the Community Subscription for even faster response times and to help sustain this space

Deprecate MD5 file checksum in favor of XXH64

Rodolfo

Rodolfo

👑 Chevereto Godlike
Chevereto Staff
Administrator
Describe your feature request
The MD5 file checksum hashing is O(n) and when we started to support big files you can notice a very abysmal performance. A 5GB file can easy take up to 20s to compute the hash. This RFC is for updating the file hashing function, which is used only to detect duplicates.

The xxh64 algo is way faster and collision resistant, it is natively supported by PHP and we can also easily add it to JavaScript (wasm) which is really fast in the browser.

1745846012182.png

^ The XXH64 algo is 16X faster than MD5.

Where did you see this?
I was lurking at the source code and I noticed this performance bootle neck.

Community interest
Anyone having long waits after the media has been uploaded is likely due to this. File md5 hashing is slow as the bigger the file it gets and we should update in this.
 
Last edited:
Seems that it will be xxh64, I only found a wasm module capable of deliver fast for big files.
 
You must log in or register to reply here.
Back
Top