• Hey Guest, don't forget to VOTE on each RFC topic. Your voting determine Chevereto development! No votes, no development.
  • Welcome to the Chevereto user community!

    Here users from all over the world gather around to learn the latest about Chevereto and contribute with ideas to improve the software.

    Please keep in mind:

    • 😌 This community is user driven. Be polite with other users.
    • 👉 Is required to purchase a Chevereto license to participate in this community (doesn't apply to Pre-sales).
    • 💸 Purchase a Pro Subscription to get access to active software support and faster ticket response times.

Block image based on MD5 Hash ban list

mkerala

👽 Chevereto Freak
💡Describe your suggestion

Block abusive image which are repeatedly uploaded to the site by adding MD5 hash to block list similar to IP ban.

IP ban is less effective given that most ISPs assign dynamic IPs and some users keep uploading same abusive image again and again.

👏Where did you saw this?

Chevereto already make use of MD5 for blocking duplicate images, same could be used for blocking offensive images.

🔥Interest outside our community

Background: Recently someone contacted me about their private pic uploaded to my site. I immediately removed the pic and banned uploader IP. However, this pic gets uploaded again and again from different IPs everyday and being shared on other sites using the image link. This person have to contact me everyday with a new link to take down and I felt really powerless to prevent this personal abuse.
 
Same scenario here a couple of times.
I removed the images and exactly the same ones keept beeing uploaded from all over the World (random IP's) like someone is using a VPN service.
 
+1

Same here. This can be really useful combating the predator who is uploading the same CP to my site from new IP's every other day.
 
Agreed a cool idea, I have started banning ranges 178.4.*.* to stop the same uploads day after day.
 
The system already uses MD5 for duplicated detection, adding a blacklist shouldn't be that hard but I don't think that such measure will really make a difference as you only need to alter one pixel and the hash will be completely different.

It is well known that inexperienced attackers will use just one image, but generate a bunch of the "same image" with different hashes is extremely easy and I won't be surprised if they do that right away this gets implemented. Applying this MD5 ban will mean +5 minutes or so for generating a bunch of same images with different hash.

Rather than a file/string hash, we need an image content hash (also known as image fingerprint) with a DB storing that data. This is not just use another hash function, it needs more infrastructure and to make it perform fast the DB will be huge. Just huge.

I think that ask the application to filter this kind of thing is a good idea, but implementing a firewall in your server will certainly be helpful as the desired blocker is not a trivial feature to add.
 
An MD5 blacklist may not stop duplicate uploads it will certainly reduce them, bit like the spam folder on our email accounts, won't catch all spam but we all still use it 😉
 
Firewall is ineffective against these guys as they are not bots. There are so many free VPN providers out there and it is impossible to block them all.

I agree that MD5 block can be easily bypassed. But, we can put at least an additional roadblock to make it harder so they just find another site.
 
Firewall is ineffective against these guys as they are not bots. There are so many free VPN providers out there and it is impossible to block them all.

I agree that MD5 block can be easily bypassed. But, we can put at least an additional roadblock to make it harder so they just find another site.
I agreed this idea. But its not block completely this.

Also you can use ASN numbers for block like https://www.spamhaus.org/drop/asndrop.txt

you can find VPN's ASN numbers list for block.
 
Show me reports or something that tells me that this could be an effective measure. Sorry to ask about this, but I don't think that is wise to waste time in developing something just to find out that attackers will easily bypass it.
 
The only reason you ask is you know no one can present such evidence, the 'Enable Duplicate Uploads' feature has already been developed by your good self, we are just asking for the option to make it permanent, rather than just 24 hours.
 
If attackers are using several IPs to fool the dupe/flood protection, why you think that something more easier to fool (by getting a bunch of attacking images with different signature) will really make a difference?

I think that there's an issue here, but I don't share your thoughts on the best solution for it.
 
Show me reports or something that tells me that this could be an effective measure. Sorry to ask about this, but I don't think that is wise to waste time in developing something just to find out that attackers will easily bypass it.

These are old articles, but still relevant.

https://www.pcmag.com/news/336791/hash-list-to-help-google-facebook-more-remove-child-porn
https://www.thorn.org/blog/eliminating-child-sexual-abuse-material-hash-values/
https://nakedsecurity.sophos.com/20...email-for-child-porn-images-leads-to-arrest‏/
 
Maybe a way to store the image that has been banned/blocked. If the same image gets uploaded regardless of which ip the image will fail to upload?
 
Hope you noticed the ImageDNA thing. That's basically a fingerprint system, the thing that I've mentioned like 10 post before.

2288

I must say that I wasn't aware of this MS cloud service and the good thing is that it simplifies the implementation to a merely bunch of API calls. I'm reading that is free (if you are eligible) which is the best thing that we could ask.

I just sent the form where you ask for access, for anyone interested, here the service terms:

PhotoDNA Cloud Service Terms of Use
Last updated: May 2018

1. Terms of Use
Your use of the PhotoDNA Cloud Service is governed by the Microsoft Online Subscription Agreement (https://azure.microsoft.com/en-us/support/legal/subscription-agreement/) which incorporates the Online Services Terms (https://www.microsoft.com/en-us/licensing/product-licensing/products.aspx), and the terms set out below. The terms set out below control to the extent there is a conflict with other applicable terms, notwithstanding the entire agreement provision of the Microsoft Online Subscription Agreement.

2. Acknowledgement of Relevant Legal Requirements
In connection with your obligation to comply with all laws and regulations applicable to your use of Online Services as set out in the Online Services Terms, you acknowledge that the content of the customer data you process using the PhotoDNA Cloud Service may be subject to specific legal requirements which may include, but are not limited to, laws requiring the reporting of any facts or circumstances from which you obtain actual knowledge of an apparent violation of child pornography laws to the National Center for Missing and Exploited Children (NCMEC) and/or a government agency in your jurisdiction. Any information that Microsoft provides to you regarding the use of the PhotoDNA Cloud Service is not intended as legal advice and is not a substitute for the advice of your own legal counsel.
3. Purpose limitation, Your instructions to Audit and report to NCMEC
The purpose of the PhotoDNA Cloud Service is to prevent the spread of child sexual abuse content, and to support investigations targeted to stopping the distribution and possession of child sexual abuse content ("Purpose"). You may use the PhotoDNA Cloud Service solely for the Purpose, and you must not use it for any other purposes. To achieve the Purpose, it is important for you and Microsoft to work together to maintain the integrity of the service.

Accordingly:
(a) You hereby authorize Microsoft to take steps to monitor and audit your usage of the PhotoDNA Cloud Service to help ensure that the service is used solely for the Purpose, and otherwise in accordance with these terms.
(b) You hereby authorize Microsoft to provide aggregate reports to NCMEC that summarize the number of images you uploaded on the PhotoDNA Cloud Service that match the signatures of known child pornography images.You hereby instruct Microsoft to identify you in these reports. You understand that such reports do not relieve you of any legal requirements that might arise from your use of the PhotoDNA Cloud Service, including, but not limited to, any obligation you have to file NCMEC reports.

4. Eligibility
Access to the PhotoDNA Cloud Service is subject to Microsoft's sole discretion based on our eligibility criteria and vetting process. Microsoft reserves the right to re-verify eligibility at any time and suspend access to the PhotoDNA Cloud Service at any time. To apply for and use the PhotoDNA Cloud Service, you must provide current, complete, and accurate information in the registration form and any re-verification requests from Microsoft.

You are solely responsible for:
(a) keeping your password and account confidential; and
(b) any and all activities that occur under your account.

You must promptly notify Microsoft of any unauthorized use of your account or any other breach of security. Microsoft will not be liable for any loss that you incur if someone else uses your password or account, either with or without your knowledge. However, you could be held liable for losses incurred by Microsoft or another party if someone else uses your account or password.

5. Offer Details for the PhotoDNA Cloud Service
This page is the Portal for the purposes of the PhotoDNA Cloud Service Subscription and the applicable Offer Details are as follows:
(a) The PhotoDNA Cloud Service is provided free of charge (with limited transactions per month) or charged as Content Moderator transactions (if used with Content Moderator as required for high volume usage).
(b) The PhotoDNA Cloud Service is a Limited Offering.
(c) The Term of a PhotoDNA Cloud Service Subscription is 30 days. Microsoft may limit or throttle your PhotoDNA Cloud Service transactions, including as provided for in service documentation.

6. Internal Use Only
You will use the PhotoDNA Cloud Service solely for your internal use. You may not use the PhotoDNA Cloud Service to provide a Managed Service Solution as defined in the Microsoft Online Subscription Agreement.

7. No Support or SLA
The PhotoDNA Cloud Service is not covered by customer support and does not have an SLA. The PhotoDNA Cloud Service is not a Microsoft Azure Service, including when used in conjunction with Content Moderator or other Online Services.
 
I also read about this PhotoDNA, maybe you can have this switch box in control panel where owner of site can pick which service to use, one you already got added or PhotoDNA.

Also you could add another one to list https://www.meldpunt-kinderporno.nl/over-ons/hash-database/ so that way site owner can pick which to use and then assign API from ones he can get it from.

Last one i think is free and that would be great thing to add into drop box like you do to some extra services/tools so when you select it, the proper form box comes like a API box and then you assign api key you get and save changes so it can start working on checks when some 1 uploads images.
 
I have been using https://www.cybertip.ca/app/en/projects-arachnid and it works perfectly. They do fuzzy image matching to detect images that have been slightly modified when you post to the crawl endpoint. This detect a much larger amount of CSAM than a cryptographic hash check.

I created a custom script to send image list every hour. I get email alert if there is any flagged image.
 
I have been using https://www.cybertip.ca/app/en/projects-arachnid and it works perfectly. They do fuzzy image matching to detect images that have been slightly modified when you post to the crawl endpoint. This detect a much larger amount of CSAM than a cryptographic hash check.

I created a custom script to send image list every hour. I get email alert if there is any flagged image.
is there possible for you to share copy of that script? So others here may be able to use it.
 
Back
Top