Facebook bots hitting login page too much.

[mkerala]

For the past one week Facebook bots are hitting my login page multiple times every min. My database server is going crazy with all these requests.



These are not users redirected from FB, but real FB crawlers as the access log has all FB IP like this one.

66.220.149.18 IP Address Details - IPInfo.io

Full IP address details for 66.220.149.18 (AS32934 Facebook, Inc.) including geolocation and map, hostname, and API details.

ipinfo.io

To add, I am not seeing any increase in any genuine traffic or increase in user registration. Serverfault has few report of similar incidents for other websites as well. No idea why FB is hitting my login page so much.

Does anyone else experiencing similar issue?

 

[june]

Hey, I haven't had that issue yet but I'm sure you could get past it by simply adding /login to the disallow to your robots.txt.

[CODE highlight="2"]User-agent: *
Disallow: /login/[/CODE]

On a side note, maybe they are real people and the ip you are getting is facebook's redirection page?
the hostname for the ip you mentioned above is fwdproxy-prn-018.fbsv.net.
I'll just assume that means forwarding proxy server

 

[mkerala]

Hey, I haven't had that issue yet but I'm sure you could get past it by simply adding /login to the disallow to your robots.txt.

[CODE highlight="2"]User-agent: *
Disallow: /login/[/CODE]

On a side note, maybe they are real people and the ip you are getting is facebook's redirection page?
the hostname for the ip you mentioned above is fwdproxy-prn-018.fbsv.net.
I'll just assume that means forwarding proxy server

Sorry, forgot to add this was the first thing I did. It worked for like 2 days and then back again.

If it was genuine user, why are all they going to the login page?

 

[mkerala]

A closer look at access logs suggest these may be genuine users using Facebook App or something. But still can't figure out why they are getting redirected to login page.

"31.13.127.4 - - [27/Jan/2021:00:00:33 +0530] "GET /login HTTP/2.0" 200 10499 "https://gifyu.com/image/nwAS" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.2 Mobile/15E148 Safari/604.1"

 

[june]

Can you check if there are other users from *.fbsv.net hostname?
Maybe those people are visiting login page themselves I see no trace of any redirect

 

[mkerala]

Can you check if there are other users from *.fbsv.net hostname?
Maybe those people are visiting login page themselves I see no trace of any redirect

I can see lot of hits to login page and all has a FB IP with hostname fbsv.net.

173.252.127.7 - - [27/Jan/2021:00:51:55 +0530] "GET /login HTTP/2.0" 200 10501 "https://gifyu.com/image/lGtE" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.2 Mobile/15E148 Safari/604.1"
173.252.127.38 - - [27/Jan/2021:00:51:54 +0530] "GET /login HTTP/2.0" 200 10500 "https://gifyu.com/image/4Abb" "Mozilla/5.0 (iPhone; CPU iPhone OS 12_4_9 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.1.2 Mobile/15E148 Safari/604.1"
173.252.95.19 - - [27/Jan/2021:00:51:53 +0530] "GET /content/images/system/home_cover_1580027946249_c7ddaa.jpg HTTP/2.0" 200 528869 "https://gifyu.com/login" "Mozilla/5.0 (iPhone; CPU iPhone OS 14_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.2 Mobile/15E148 Safari/604.1"
173.252.83.120 - - [27/Jan/2021:00:51:53 +0530] "GET /image/6jiP HTTP/1.1" 200 16014 "-" "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)"

 

[mkerala]

Found this on https://www.facebook.com/externalhit_uatext.php

Why does Facebook appear in my server logs?​

Facebook allows its users to send links to interesting web content to other Facebook users. Part of how this works on the Facebook system involves the temporary display of certain images or details related to the web content, such as the title of the webpage or the embed tag of a video. Our system retrieves this information only after a user provides us with a link. You may have found this page because a Facebook user sent a link from your website to other Facebook users. If you have any questions or concerns about any links or content sent by one of our users, please contact us at legal@facebook.com.

 

[june]

hits to login page

Sorry, i meant to say if there are more hits from that hostname but on other pages as well

 

[june]

Found this on https://www.facebook.com/externalhit_uatext.php

Why does Facebook appear in my server logs?​

Facebook allows its users to send links to interesting web content to other Facebook users. Part of how this works on the Facebook system involves the temporary display of certain images or details related to the web content, such as the title of the webpage or the embed tag of a video. Our system retrieves this information only after a user provides us with a link. You may have found this page because a Facebook user sent a link from your website to other Facebook users. If you have any questions or concerns about any links or content sent by one of our users, please contact us at legal@facebook.com.

This just looks like like they are talking about referrer in analytics.

 

[mkerala]

Sorry, i meant to say if there are more hits from that hostname but on other pages as well

Yeah, I can see the same IP accessed few image page as well. But what's concerning me is how much hit login page is taking compared to other pages.