Majestic-12
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
Mine is set at 28 works, with 25 buckets, it won't always use the full amount of workers.Darren wrote:I started again yesterday but I can not get any speed out of it.
We had a great setup before but I just can seem to get the setting right.
I have told it to use 15 buckets but only 4 start.
Anyone still got our old settings?
Try those settings and see what happens, I'm getting quite a bit done on these settings, its more trial & error with this one. Just keep an the out on the errors text bottom right on main page of the window, if that alters from normal to high or v.high then you need to alter the settings back down, I'm slowly pushing mine up to find my ideal settings.
Don't forget to restart the client if you alter the numbers of workers
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
I never had a problem running it before with that or DIMES way back then.Rockinfroggi wrote:The client has shut down a few times due to an error but doesn't say what the error is, will slowly read up in the forum and work out what the hell I'm doing
Does it run OK with DepSpid as both use bandwidth?.
Gary.
PS: You can alter your flag to English if you wish rather than been a UK flag
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
UBT - Halifax--lad wrote:
I never had a problem running it before with that or DIMES way back then.
PS: You can alter your flag to English if you wish rather than been a UK flag
It could be something in my setup or settings so I'll have a read up and a fiddle until I break something.
I used the UK flag because I thought we were a UK team? and as they don't have the flag for my planet 'Alpha Stupidita' :dontknow: would you rather I had the English flag (does it affect the stats?)
Gary.
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
If you can't find the problem just put a help request in the forum over there, usually get a prompt replyRockinfroggi wrote:It could be something in my setup or settings so I'll have a read up and a fiddle until I break something.
I used the UK flag because I thought we were a UK team? and as they don't have the flag for my planet 'Alpha Stupidita' :dontknow: would you rather I had the English flag (does it affect the stats?)
No flags don't affect the stats at all, its down to personal preference I suppose, notice we have a welsh dragon amongst us :lol:
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
I also just spotted this
I checked it out how it tells you to check and I was getting lots of those errors so I used this to patch over Microsoft's settings it was very easy, just open the .exe file thats downloaded, it opens up a DOS window, just follow the screen settings, mine is now set to 50. As soon as this is changed windows brings up a warning saying it wants the CD Windows was on, just cancel this as it is Windows trying to alter the settings back
More info can be found hereIf you run Windows XP SP2 (Service Pack 2 -- other operating systems are not affected) then you might want to check Events Viewer -> Application. You may notice some of the following warnings:
"TCP/IP has reached the security limit imposed on the number of concurrent TCP connect attempts."
This is so-called Microsoft security kicking in if they detect more than X failed outgoing connections, which can happen with crawler because it can come across with sites that are down.
This "security" slows down crawling and possibly responsible for more timeouts than necessary, so I am searching (still using Google, but hope not for long) for ways to workaround this POS of a feature
I checked it out how it tells you to check and I was getting lots of those errors so I used this to patch over Microsoft's settings it was very easy, just open the .exe file thats downloaded, it opens up a DOS window, just follow the screen settings, mine is now set to 50. As soon as this is changed windows brings up a warning saying it wants the CD Windows was on, just cancel this as it is Windows trying to alter the settings back
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
Stick in a request for help over there thenm with any luck it can be solved, I recall mine used to shut down every so often but it was quite rare for me. I don't think you would see the other warnings unless you increase the settings for workers, I have so I think thats why mine was showing the warningsRockinfroggi wrote:Just checked my event logs and under security there is not one error but under Applications I have a bunch of Net Runtime 2.0 Error Reportings and they all tie in with when the client shuts down.
Gary.
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
Touch wood it hasn't shut down for the best part of the day but I increased the number of buckets and workers as I only had 6 buckets running at any one time and instead of getting more it's gone down to 3 buckets?
Yes I did restart the client after changing. No biggie, just thought I'd mention it while I see what else I can break.
Edit, Just a thought, I take it there is nothing to be gained by adding this to all my PC's as they would be sharing the same bandwidth, but would I be better off putting it on one of my dedicated Boinc systems rather than my main workhorse PC?
Bam Bam.
Yes I did restart the client after changing. No biggie, just thought I'd mention it while I see what else I can break.
Edit, Just a thought, I take it there is nothing to be gained by adding this to all my PC's as they would be sharing the same bandwidth, but would I be better off putting it on one of my dedicated Boinc systems rather than my main workhorse PC?
Bam Bam.
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
Just in case anyone else has the problem I had, here's the reply I got from their forum.
Gary.
Hi there!
I think I know what it is, doh - I forgot to add config file to .NET 2.0 build, you can workaround this as follows:
create text file in node's directory called MJ12node.exe.config with the following text in it:
Code:
<configuration>
<runtime>
<legacyUnhandledExceptionPolicy enabled="1" />
<legacyNullReferenceExceptionPolicy enabled = "1"/>
</runtime>
<system.net>
<settings>
<httpWebRequest useUnsafeHeaderParsing="true" />
</settings>
</system.net>
</configuration>
I am going to update .NET 2.0 ZIP file now: you can alternatively unzip and overwrite all files from here: http://majestic12.co.uk/files/mj12node/ ... 2_v148.zip
Gary.
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
UBT - Halifax--lad wrote:Mine is set at 28 works, with 25 buckets, it won't always use the full amount of workers.Darren wrote:I started again yesterday but I can not get any speed out of it.
We had a great setup before but I just can seem to get the setting right.
I have told it to use 15 buckets but only 4 start.
Anyone still got our old settings?
Try those settings and see what happens, I'm getting quite a bit done on these settings, its more trial & error with this one. Just keep an the out on the errors text bottom right on main page of the window, if that alters from normal to high or v.high then you need to alter the settings back down, I'm slowly pushing mine up to find my ideal settings.
Don't forget to restart the client if you alter the numbers of workers
Like Darren I'm only showing 4 Mrs Bouquets at a time, how many are you showing in Activity David?
Long Frog Kick The Bucket.
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
Sure, no problem, I've only gone by things I've read as I don't know enough about all the settings but the changes I have made are:Darren wrote:Hey Froggi,
you seem to have some good settings thre & I'm too lazy to tinker with mine.
Care to share?
Prefer all TLDs apart from .COM/.NET/.ORG
Downstream 60%
Upstream 80%
Workers 40
Buckets 30
Archiving Delay 70
LZMA Word size 20
Those are the only changes I've made so far, hope It helps as it would be good if we could get into the daily top Ten instead of 11th and 12th.
Gary.
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
UBT - Halifax--lad wrote:When I get more time to tinker with the settings I may set MJ-12 back to crawling, I was getting too many errors
TBH I've pretty much ignored the errors as it will sometimes creep up to high or very high but within an hour or so settles back down to normal, my successes are usually between 70% to 90%.
Gary.
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
Yup, just looked, we are #9 at the mo :D
Just thought, not sure if my figures are better because I have 3 nodes running? I didn't think that would make any difference as they all have to share the same bandwidth.
Just as a test I'll shut down 2 of my nodes around midnight tonight and run on just one tomorrow to see if my rate drops down.
Gary.
Just thought, not sure if my figures are better because I have 3 nodes running? I didn't think that would make any difference as they all have to share the same bandwidth.
Just as a test I'll shut down 2 of my nodes around midnight tonight and run on just one tomorrow to see if my rate drops down.
Gary.
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
Put it this way shall I just to annoy you I used to be able to crawl 1.5 million URL's in a day when I got the settings perfect, only ever did it for the one day though, I slowed it down after that to lessRockinfroggi wrote: :evil: :angry4: Iza gonna be watchin you very closely and if ya don't share yer settings Darren will sit on you while we get some brave soul to give you a full body wax very very slowly.
Guantanamo Frogster.
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
Can I ask (in a French accent) why you have gone with the English flag as opposed to the Uk? If everyone had the UK flag as with all the other projects we would be #4 overall in the world, and if you share your super duper pooper scooper settings we may work our way to #1 in the world.
Just a Frog thinking out load, ignore as required
Crunchy Nut Frog (just add milk)
Just a Frog thinking out load, ignore as required
Crunchy Nut Frog (just add milk)
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
Well I am in England Not too bothered for the coutry stats as such, as its more our overall stats that will help push us upRockinfroggi wrote:Can I ask (in a French accent) why you have gone with the English flag as opposed to the Uk? If everyone had the UK flag as with all the other projects we would be #4 overall in the world, and if you share your super duper pooper scooper settings we may work our way to #1 in the world.
Just a Frog thinking out load, ignore as required
Crunchy Nut Frog (just add milk)
-
- Posts: 1630
- Joined: Mon Nov 06, 2006 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1630
- Joined: Mon Nov 06, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
Dette innebærer at våre publikasjoner formidles til bibliotek, universitet, høyskoler og andre institusjoner der bøkene gjøres tilgjengelige for offentligheten - og selvsagt også til interesserte privatpersoner. Når ikke annet er avtalt avleveres ett eksemplar til hver forespørrer. I tillegg kommer den vanlige pliktavleveringen til Nasjonalbiblioteket. Forlaget har så langt distribuert bøker til Norge :P
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
Rockinfroggi wrote:Dette innebærer at våre publikasjoner formidles til bibliotek, universitet, høyskoler og andre institusjoner der bøkene gjøres tilgjengelige for offentligheten - og selvsagt også til interesserte privatpersoner. Når ikke annet er avtalt avleveres ett eksemplar til hver forespørrer. I tillegg kommer den vanlige pliktavleveringen til Nasjonalbiblioteket. Forlaget har så langt distribuert bøker til Norge :P
Code: Select all
This involve that our pamphlets intermediary at directory , college høyskoler and other the institutions there books do accessible for public and obviously also at interested privatpersoner. As nothing but am agreed upon deliver a copy at apiece enquiry. Additional comes it common pliktavleveringen at Nasjonalbiblioteket. Forlaget has so far distributed books at Norway
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
UBT - Halifax--lad wrote:Up to 6th place for the day, my settings seem to be working fine, still need more time to play with them before I pass them out to all of you
I'm beating Iragi frog anyway
Here in Iraq we do the beating, so get down you western infidel.
All looks pretty level pegging to me but we will see what the rest of the day brings.
Sad Ham Insane.
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
UBT - Halifax--lad wrote:Don't forget though your using 3 machines against my 1, and I'm still kicking ure ass
Well there's team spirit for you, here I was in my own little world thinking we were out to beat other teams and it turns out we are trying to beat eackother
Not only that, I now have to protect my donkey from from Laddies boot
Not even looked at that yet but may have a play with it later today.Anyone brave enough to turn on the webserver bit yet?
UnderFrog
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
Time to show my settings then I think, it seems to be running fine now on my machine, I know I can push it some more as yet
So here is the main display
I am crawling all domains
Here is my connection settings, just realised from looking at screen shot I had not chosen my connection type, so done that now
My crawl settings, the workers are still quite low and I will push them up further later on during the course of the week. The workers are controlled by profiles. Notice I have the retry boxes ticked on all the DNS & other hard errors, this means before they are classed as an error they will be retried, cuts down the amount of fails.
The next two are profiles, these are great to use and should be used at all times. I can set MJ-12 to work harder when I'm in bed and not using the computer, when I'm at work, etc etc. On the 2nd image the "override number of workers to be" is in actual number of workers not as a percentage as the profile review would have you believe. To enable the override workers function to work correctly you have to ensure that you have set the amount of maximum workers correctly in the crawl options
Eg Max amount of workers in crawl options set to 50 you then cannot ask a profile to have 90 workers as the crawl option only allows 50
I also have an override profile which I can activate if I'm using computer not within my specified hours just in case the crawler makes the net slow, to activate that you have to inactivate the timed profiles you may have set up. To easily get to profiles once set up to activate or inactivate just right click on MJ-12 icon in sys-tray and go to profiles
Then rest of screens
So here is the main display
I am crawling all domains
Here is my connection settings, just realised from looking at screen shot I had not chosen my connection type, so done that now
My crawl settings, the workers are still quite low and I will push them up further later on during the course of the week. The workers are controlled by profiles. Notice I have the retry boxes ticked on all the DNS & other hard errors, this means before they are classed as an error they will be retried, cuts down the amount of fails.
The next two are profiles, these are great to use and should be used at all times. I can set MJ-12 to work harder when I'm in bed and not using the computer, when I'm at work, etc etc. On the 2nd image the "override number of workers to be" is in actual number of workers not as a percentage as the profile review would have you believe. To enable the override workers function to work correctly you have to ensure that you have set the amount of maximum workers correctly in the crawl options
Eg Max amount of workers in crawl options set to 50 you then cannot ask a profile to have 90 workers as the crawl option only allows 50
I also have an override profile which I can activate if I'm using computer not within my specified hours just in case the crawler makes the net slow, to activate that you have to inactivate the timed profiles you may have set up. To easily get to profiles once set up to activate or inactivate just right click on MJ-12 icon in sys-tray and go to profiles
Then rest of screens
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
I have got it crawling all the domains, thought there is no point missing any out, forgot about Martle, might be an idea to message him and see how he is doingDarren wrote:I have changed mine so it looks more like your settings then halifax.
Are there any TLD's you ain't got checked?
Also what happened to martle?
Is it worth me dropping him an email?
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
UBT - Halifax--lad wrote: Well I am in England Not too bothered for the coutry stats as such, as its more our overall stats that will help push us up
*Cough*
Here is me trying to get England in the top stats, then the numptys arrive and change flags Shocked
Make ya bloomin mind up, you're as fickle as a woman
All settings changed on 2 nodes, thought it might be an idea to leave one node with my original settings to see what improvement there is, if any.
The Quintessential English Frog.
Last edited by Rockinfroggi on Mon May 07, 2007 5:28 pm, edited 1 time in total.
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
Piggies are supposed to go Oink not Oy
Yes, all nodes are on the same connection, that's why I didn't think it would make that much difference plus looking at yours and David's settings you both have faster connections than me.
I think I would have done better today but one node was returning nothing for half the day until I delete the profiles.
Still, even with all our increased production we seem to have dropped a couple of places :shock:
Gary.
Yes, all nodes are on the same connection, that's why I didn't think it would make that much difference plus looking at yours and David's settings you both have faster connections than me.
I think I would have done better today but one node was returning nothing for half the day until I delete the profiles.
Still, even with all our increased production we seem to have dropped a couple of places :shock:
Gary.
-
- Posts: 1630
- Joined: Mon Nov 06, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
Darren wrote:I just got a warning from my ISP about download levels over the past couple weeks. No doubt this had something to do with it over the last few days.
So I am laying low for a couple weeks but may start it up again once the dust settles.
I can't really afford to be capped again!
I was just wondering whether capping might be a problem not that I've had a warning yet but I may ease off a bit.
Gary.
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 1630
- Joined: Mon Nov 06, 2006 12:00 am
You are not alone. I'm still going, though not as fast as some others :shock:Rockinfroggi wrote:Looks like I'm flying the flag on my own on this now.
Obviously there is no real benefit to the team stats so is there any point in me continuing or is it just a waste of time?
Gary.
I think it worthwhile, hope the others come back soon.
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 1630
- Joined: Mon Nov 06, 2006 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 8
- Joined: Wed May 16, 2007 1:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
Welcome to the team, forum & MJ-12, you can't really do it too wrong if you go back 2 - 3 pages and then start reading through the messages you will see hints & tips for settings.Active Tech wrote:well i joined up on this one has well hope i'm doing it right tho
Feel free to ask any questions. Do you crunch BOINC then
-
- Posts: 8
- Joined: Wed May 16, 2007 1:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
UBT - Halifax--lad wrote:Pushing my connection want to see how long it takes a cap to be introduced, should have happened 10 mins ago but still at full speed for now, got a program that monitors how much bandwidth I have used
You wanna be capped? and there was me thinking I was the forum idiot
Gary.
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
I can only be capped between 4pm & midnight and then all I get is a 4 hr speed restriction, so no download limit cap as such, just a speed restriction. On my package it would be capped to 2Mbps, so far I have gone 4 times over my limit but still at full speed
http://allyours.virginmedia.com/html/in ... affic.html
http://allyours.virginmedia.com/html/in ... affic.html
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Active UBT Contributor 15+ yrs
- Posts: 7515
- Joined: Wed May 03, 2006 1:00 am
- Contact:
"Sometimes we moderate the speeds", perhaps their reduced customer base has given them some more headroom..
Looks like if they do cap you are capped for 4 hours not just to to the end of peak time "This will last for 4 hours from when the traffic management policy is applied"
Still seems a quite reasonable and open "fair usage" policy
Looks like if they do cap you are capped for 4 hours not just to to the end of peak time "This will last for 4 hours from when the traffic management policy is applied"
Still seems a quite reasonable and open "fair usage" policy
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am