Interesting stats...

A section to chat about whatever takes your fancy about the team in general. For queries about the UBT Forum, please post a message here. Likewise, any suggestions for making the forum better...
Post Reply
UBT - Timbo
UBT Forum Admin
Posts: 9673
Joined: Mon Mar 13, 2006 12:00 am
Location: NW Midlands
Contact:

Interesting stats...

Post by UBT - Timbo »

Hi all

Just seen this on the front page of the forum:
Most users ever online was 828 between Thu 29. Jun 2017 and Mon 3. Jul 2017
Sadly, most of these "visitors" are spider-bots, from search engines and the like...so, one wonders what they are doing with all the data they are spidering from our forum ;-)

I'm half tempted to ban most of them, but then I stop myself as I'm thinking that some people might find our forum interesting - although they would need to be a member to see anything !!

I think it's time to see how "exposed" our forum is, on the likes of Google, Baidu, Yahoo and Bing.

regards
Tim
chriscambridge
Active UBT Contributor 1+ yr
Posts: 2178
Joined: Mon Aug 08, 2016 1:56 pm
Location: UK

Re: Interesting stats...

Post by chriscambridge »

Well interesting you should say that, earlier the site was offline due to lack of SQL connections; a known method of doing a simple DOS attack.

Image

Have you checked the IPs, even though IP spoofing is not that difficult, to check that these are spider bots and not something else?
Woodles
UBT Contributor
Posts: 11757
Joined: Thu Dec 20, 2007 12:00 am
Location: Cambridgeshire

Re: Interesting stats...

Post by Woodles »

Same again this morning, just managed to get through after about 30 minutes of trying. SQL errors like Chris got last night and "504 Gateway timeout" errors.

All seems well now though?

[Edit]Spoke too soon, just spent 20 minutes trying to open this post up again to edit it[/edit]

Mark
UBT - Timbo
UBT Forum Admin
Posts: 9673
Joined: Mon Mar 13, 2006 12:00 am
Location: NW Midlands
Contact:

Re: Interesting stats...

Post by UBT - Timbo »

Hi both

I've seen the 504 error a few times.

I had suspected that it was because the webhosting is on one server and the SQL database is on another (which is shared).

Hence any of the customers sharing the SQL server could easily slow down it's response and hence cause an error.

I will check the logs if I can and see what might be the issue...but for now, I will suspend all the spider-bots (bar Google) and see if this makes any difference.

regards and apologies for any inconvenience caused
Tim
chriscambridge
Active UBT Contributor 1+ yr
Posts: 2178
Joined: Mon Aug 08, 2016 1:56 pm
Location: UK

Re: Interesting stats...

Post by chriscambridge »

Hi Guys,

Tim what this PHPBB/MySQL error message is saying is that this DBUser: F****r3 has opened too many concurrent DB connections, which I think is [1203].

It should not make any difference if the webserver and sqlserver are on different physical servers, this is todo with DB resources.

--

A quick fix would be just to increase the number of connections allowed, normally within the MySQL config file - however, over 1200 connections within the session period, perhaps an hour or less/more, is obviously a very high number.

--

Normally all different software you would run on your server (PHPBB, Control Panel, any other software, etc) would be under a different DB User, so normally this would only be talking about this forum. Obviously if you used the same DB user across different software installs then that is going to seriously affect your DB resources availability, but I doubt you have done that.

--


In case it is not obvious, I have smudged/starred the DBUser for security reasons! :?

--

What I would definitely do to help track down what is going on, if you have not done already, is integrate Google Analytics as this will give you an exact detail of what search engines are accessing the site, as well as what countries/cities/OS/browsers are being served. You should then be able to easily see "what" is taking all those DB handles. (connections).

--

What I would also check is that any add-ons, such as the portal, blocks, etc - are actually using the 'normal DB user': F****r3, and not another one(s), as that would double+ your used DB handles.

--

I have seen these errors previously going back a few months.. Hence, another thing to check, even though you have a dedicated server, is that the hosting provider has not set a different constraint on DB connections to what has been done in MySQL.

--

One thing you could do to limit un-necessary access is to block any access from outside the UK! However it *may block Mobile broadband and some VPNs, and obviously anyone on holiday!

I have written the code for you, and put a copy of it on my server so you can see/test it! I just ran it through AVAST VPN (UK) and that was okay; I also ran it through AVAST VPN (GERMANY) and it detected it was outside the UK which is obviously what you want.

Feel free for people to test it with mobile data; I think perhaps it will say you 'are outside the UK', which is normal as satellites are not registered as ISP networks.

You can view more information about GEO Locations, and this specific one, http://www.geoplugin.com.

--

In terms of actually using it on this forum, if you wanted - What you would want to do is put everything within index.php page in root, ABOVE everything else with NO white space (eg empty lines) above this. If you are using Google Analytics then you will want that above this code.

Code: Select all

<?php
	$geoplugin = unserialize( file_get_contents('http://www.geoplugin.net/php.gp?ip=' . $_SERVER['REMOTE_ADDR']) );
	if ($geoplugin['geoplugin_countryCode'] != 'GB') {
		$msg='<h3>UK Boinc Team forum can only be used by UK visitors!</h3><h3>If you are based in the UK and you are seeing this then you may be using a VPN and/or Mobile data. Instead use a local based ISP to access this forum. Thank you!</h3>';
		exit($msg);
	}
	//delete the else statement below; it is only here to test on my site
	else{
		echo '<h3>Welcome Visitor!</h3><h3>You are using a UK based internet provider.</h3>';
  	}
  	//delete the else statement above; do not delete the php closing tag below!
?>

You can see this code running live on my website, at:

http://www.mywebsite.com/ubt/

Obviously you will need to change mywebsite.com to the correct URL!
UBT - Timbo
UBT Forum Admin
Posts: 9673
Joined: Mon Mar 13, 2006 12:00 am
Location: NW Midlands
Contact:

Re: Interesting stats...

Post by UBT - Timbo »

Hi Chris

Thanks for the insight :-)

There is only the one DB on this domain and on the SQL server, although there is a second (different) database, that links to the forum and is a maillist program - so that I could potentially email any named members. I tried this a while ago and got into trouble with the hosting company as they were seeing too many bounce backs onto their sever, from email addresses that were no longer valid (ie past UBT members who've given up BOINCing and have since changed broadband or email providers).

I'm in a state of flux for the next 2-3 weeks, moving house, so if I get a chance I will look into it, when I have some downtime (and assuming this doesn't become a "major" issue).

regards
Tim
chriscambridge
Active UBT Contributor 1+ yr
Posts: 2178
Joined: Mon Aug 08, 2016 1:56 pm
Location: UK

Re: Interesting stats...

Post by chriscambridge »

Hi Tim,

Yeah no worries, I just thought I would give you a heads up on a few possibilities, most of which you probably already know. I have only noticed these errors every once in a while so I guess its not critical.

I only did the GEO location code as it was something that I wanted to do years ago and back then it was sooo dificult, where-as now, as you can see its so simply, just a line or two of code.

Just to add, if you were to go down that route then you would also have to look at, if I remember rightly, the user_agent so you could let non UK server based search engines through, like Google.co.uk - Its not that much more difficult as long as you know the user_agent, which will be in your server logs.

What I will do, as the code is on this post, is pull that copy offline as it was just to show you that it actually works. Any queries if necessary just PM me.
UBT - Timbo
UBT Forum Admin
Posts: 9673
Joined: Mon Mar 13, 2006 12:00 am
Location: NW Midlands
Contact:

Re: Interesting stats...

Post by UBT - Timbo »

Hi Chris

I think the GEO code might be going a bit far, as we do have some members of the forum who are non-UK residents...so, stopping them have access might be a step too much, esp as some are admins for various projects, plus we have a number of ex-pats, who are resident in USA, Australia and other places, who might want to visit the forum, once in a while.

But I do appreciate the effort you've gone to on this :-)

regards
Tim
chriscambridge
Active UBT Contributor 1+ yr
Posts: 2178
Joined: Mon Aug 08, 2016 1:56 pm
Location: UK

Re: Interesting stats...

Post by chriscambridge »

:)
Post Reply