Majestic-12
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
Sure, no problem, I've only gone by things I've read as I don't know enough about all the settings but the changes I have made are:Darren wrote:Hey Froggi,
you seem to have some good settings thre & I'm too lazy to tinker with mine.
Care to share?
Prefer all TLDs apart from .COM/.NET/.ORG
Downstream 60%
Upstream 80%
Workers 40
Buckets 30
Archiving Delay 70
LZMA Word size 20
Those are the only changes I've made so far, hope It helps as it would be good if we could get into the daily top Ten instead of 11th and 12th.
Gary.
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
UBT - Halifax--lad wrote:When I get more time to tinker with the settings I may set MJ-12 back to crawling, I was getting too many errors
TBH I've pretty much ignored the errors as it will sometimes creep up to high or very high but within an hour or so settles back down to normal, my successes are usually between 70% to 90%.
Gary.
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
Yup, just looked, we are #9 at the mo :D
Just thought, not sure if my figures are better because I have 3 nodes running? I didn't think that would make any difference as they all have to share the same bandwidth.
Just as a test I'll shut down 2 of my nodes around midnight tonight and run on just one tomorrow to see if my rate drops down.
Gary.
Just thought, not sure if my figures are better because I have 3 nodes running? I didn't think that would make any difference as they all have to share the same bandwidth.
Just as a test I'll shut down 2 of my nodes around midnight tonight and run on just one tomorrow to see if my rate drops down.
Gary.
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
Put it this way shall I just to annoy you I used to be able to crawl 1.5 million URL's in a day when I got the settings perfect, only ever did it for the one day though, I slowed it down after that to lessRockinfroggi wrote: :evil: :angry4: Iza gonna be watchin you very closely and if ya don't share yer settings Darren will sit on you while we get some brave soul to give you a full body wax very very slowly.
Guantanamo Frogster.
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
Can I ask (in a French accent) why you have gone with the English flag as opposed to the Uk? If everyone had the UK flag as with all the other projects we would be #4 overall in the world, and if you share your super duper pooper scooper settings we may work our way to #1 in the world.
Just a Frog thinking out load, ignore as required
Crunchy Nut Frog (just add milk)
Just a Frog thinking out load, ignore as required
Crunchy Nut Frog (just add milk)
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
Well I am in England Not too bothered for the coutry stats as such, as its more our overall stats that will help push us upRockinfroggi wrote:Can I ask (in a French accent) why you have gone with the English flag as opposed to the Uk? If everyone had the UK flag as with all the other projects we would be #4 overall in the world, and if you share your super duper pooper scooper settings we may work our way to #1 in the world.
Just a Frog thinking out load, ignore as required
Crunchy Nut Frog (just add milk)
-
- Posts: 1630
- Joined: Mon Nov 06, 2006 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1630
- Joined: Mon Nov 06, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
Dette innebærer at våre publikasjoner formidles til bibliotek, universitet, høyskoler og andre institusjoner der bøkene gjøres tilgjengelige for offentligheten - og selvsagt også til interesserte privatpersoner. Når ikke annet er avtalt avleveres ett eksemplar til hver forespørrer. I tillegg kommer den vanlige pliktavleveringen til Nasjonalbiblioteket. Forlaget har så langt distribuert bøker til Norge :P
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
Rockinfroggi wrote:Dette innebærer at våre publikasjoner formidles til bibliotek, universitet, høyskoler og andre institusjoner der bøkene gjøres tilgjengelige for offentligheten - og selvsagt også til interesserte privatpersoner. Når ikke annet er avtalt avleveres ett eksemplar til hver forespørrer. I tillegg kommer den vanlige pliktavleveringen til Nasjonalbiblioteket. Forlaget har så langt distribuert bøker til Norge :P
Code: Select all
This involve that our pamphlets intermediary at directory , college høyskoler and other the institutions there books do accessible for public and obviously also at interested privatpersoner. As nothing but am agreed upon deliver a copy at apiece enquiry. Additional comes it common pliktavleveringen at Nasjonalbiblioteket. Forlaget has so far distributed books at Norway
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
UBT - Halifax--lad wrote:Up to 6th place for the day, my settings seem to be working fine, still need more time to play with them before I pass them out to all of you
I'm beating Iragi frog anyway
Here in Iraq we do the beating, so get down you western infidel.
All looks pretty level pegging to me but we will see what the rest of the day brings.
Sad Ham Insane.
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
UBT - Halifax--lad wrote:Don't forget though your using 3 machines against my 1, and I'm still kicking ure ass
Well there's team spirit for you, here I was in my own little world thinking we were out to beat other teams and it turns out we are trying to beat eackother
Not only that, I now have to protect my donkey from from Laddies boot
Not even looked at that yet but may have a play with it later today.Anyone brave enough to turn on the webserver bit yet?
UnderFrog
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
Time to show my settings then I think, it seems to be running fine now on my machine, I know I can push it some more as yet
So here is the main display
I am crawling all domains
Here is my connection settings, just realised from looking at screen shot I had not chosen my connection type, so done that now
My crawl settings, the workers are still quite low and I will push them up further later on during the course of the week. The workers are controlled by profiles. Notice I have the retry boxes ticked on all the DNS & other hard errors, this means before they are classed as an error they will be retried, cuts down the amount of fails.
The next two are profiles, these are great to use and should be used at all times. I can set MJ-12 to work harder when I'm in bed and not using the computer, when I'm at work, etc etc. On the 2nd image the "override number of workers to be" is in actual number of workers not as a percentage as the profile review would have you believe. To enable the override workers function to work correctly you have to ensure that you have set the amount of maximum workers correctly in the crawl options
Eg Max amount of workers in crawl options set to 50 you then cannot ask a profile to have 90 workers as the crawl option only allows 50
I also have an override profile which I can activate if I'm using computer not within my specified hours just in case the crawler makes the net slow, to activate that you have to inactivate the timed profiles you may have set up. To easily get to profiles once set up to activate or inactivate just right click on MJ-12 icon in sys-tray and go to profiles
Then rest of screens
So here is the main display
I am crawling all domains
Here is my connection settings, just realised from looking at screen shot I had not chosen my connection type, so done that now
My crawl settings, the workers are still quite low and I will push them up further later on during the course of the week. The workers are controlled by profiles. Notice I have the retry boxes ticked on all the DNS & other hard errors, this means before they are classed as an error they will be retried, cuts down the amount of fails.
The next two are profiles, these are great to use and should be used at all times. I can set MJ-12 to work harder when I'm in bed and not using the computer, when I'm at work, etc etc. On the 2nd image the "override number of workers to be" is in actual number of workers not as a percentage as the profile review would have you believe. To enable the override workers function to work correctly you have to ensure that you have set the amount of maximum workers correctly in the crawl options
Eg Max amount of workers in crawl options set to 50 you then cannot ask a profile to have 90 workers as the crawl option only allows 50
I also have an override profile which I can activate if I'm using computer not within my specified hours just in case the crawler makes the net slow, to activate that you have to inactivate the timed profiles you may have set up. To easily get to profiles once set up to activate or inactivate just right click on MJ-12 icon in sys-tray and go to profiles
Then rest of screens
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
I have got it crawling all the domains, thought there is no point missing any out, forgot about Martle, might be an idea to message him and see how he is doingDarren wrote:I have changed mine so it looks more like your settings then halifax.
Are there any TLD's you ain't got checked?
Also what happened to martle?
Is it worth me dropping him an email?
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
UBT - Halifax--lad wrote: Well I am in England Not too bothered for the coutry stats as such, as its more our overall stats that will help push us up
*Cough*
Here is me trying to get England in the top stats, then the numptys arrive and change flags Shocked
Make ya bloomin mind up, you're as fickle as a woman
All settings changed on 2 nodes, thought it might be an idea to leave one node with my original settings to see what improvement there is, if any.
The Quintessential English Frog.
Last edited by Rockinfroggi on Mon May 07, 2007 5:28 pm, edited 1 time in total.
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
Piggies are supposed to go Oink not Oy
Yes, all nodes are on the same connection, that's why I didn't think it would make that much difference plus looking at yours and David's settings you both have faster connections than me.
I think I would have done better today but one node was returning nothing for half the day until I delete the profiles.
Still, even with all our increased production we seem to have dropped a couple of places :shock:
Gary.
Yes, all nodes are on the same connection, that's why I didn't think it would make that much difference plus looking at yours and David's settings you both have faster connections than me.
I think I would have done better today but one node was returning nothing for half the day until I delete the profiles.
Still, even with all our increased production we seem to have dropped a couple of places :shock:
Gary.
-
- Posts: 1630
- Joined: Mon Nov 06, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
Darren wrote:I just got a warning from my ISP about download levels over the past couple weeks. No doubt this had something to do with it over the last few days.
So I am laying low for a couple weeks but may start it up again once the dust settles.
I can't really afford to be capped again!
I was just wondering whether capping might be a problem not that I've had a warning yet but I may ease off a bit.
Gary.
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 1630
- Joined: Mon Nov 06, 2006 12:00 am
You are not alone. I'm still going, though not as fast as some others :shock:Rockinfroggi wrote:Looks like I'm flying the flag on my own on this now.
Obviously there is no real benefit to the team stats so is there any point in me continuing or is it just a waste of time?
Gary.
I think it worthwhile, hope the others come back soon.
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 1630
- Joined: Mon Nov 06, 2006 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 8
- Joined: Wed May 16, 2007 1:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
Welcome to the team, forum & MJ-12, you can't really do it too wrong if you go back 2 - 3 pages and then start reading through the messages you will see hints & tips for settings.Active Tech wrote:well i joined up on this one has well hope i'm doing it right tho
Feel free to ask any questions. Do you crunch BOINC then
-
- Posts: 8
- Joined: Wed May 16, 2007 1:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
UBT - Halifax--lad wrote:Pushing my connection want to see how long it takes a cap to be introduced, should have happened 10 mins ago but still at full speed for now, got a program that monitors how much bandwidth I have used
You wanna be capped? and there was me thinking I was the forum idiot
Gary.
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
I can only be capped between 4pm & midnight and then all I get is a 4 hr speed restriction, so no download limit cap as such, just a speed restriction. On my package it would be capped to 2Mbps, so far I have gone 4 times over my limit but still at full speed
http://allyours.virginmedia.com/html/in ... affic.html
http://allyours.virginmedia.com/html/in ... affic.html
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Active UBT Contributor 15+ yrs
- Posts: 7515
- Joined: Wed May 03, 2006 1:00 am
- Contact:
"Sometimes we moderate the speeds", perhaps their reduced customer base has given them some more headroom..
Looks like if they do cap you are capped for 4 hours not just to to the end of peak time "This will last for 4 hours from when the traffic management policy is applied"
Still seems a quite reasonable and open "fair usage" policy
Looks like if they do cap you are capped for 4 hours not just to to the end of peak time "This will last for 4 hours from when the traffic management policy is applied"
Still seems a quite reasonable and open "fair usage" policy
-
- Posts: 3790
- Joined: Mon Mar 13, 2006 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 1434
- Joined: Tue Jan 09, 2007 12:00 am
-
- Posts: 8
- Joined: Wed May 16, 2007 1:00 am
-
- Posts: 8
- Joined: Wed May 16, 2007 1:00 am
-
- Active UBT Contributor 10+ yrs
- Posts: 3227
- Joined: Wed Aug 23, 2006 1:00 am