Page 1 of 1

2 GPU prob

Posted: Sat Nov 04, 2017 2:52 pm
by UBT - Mikee
Hi

Just installed a second GTX 1060.

Having a bit of confusion with BOINC though. Ran Moo earlier and was expecting to have 2 WUs running but only 1 was going though it was showing as using both cards. Run time is virtually the same apart from 1 which ran for 1300 seconds rather than 2000 seconds. Moo preferances are all set to 'autoselect'. Will try a few more later.

Thought I'd try Collatz - that is running 2 separate WUs.

Seems like 'autoselect' may be the problem - should I leave it there or select something like 'CUDA 4 pipe 256 thd'?

Mike

Re: 2 GPU prob

Posted: Sat Nov 04, 2017 3:31 pm
by UBT - Timbo
Hi Mike

Well done on the new acquisition :-)

I have 2x GTX 970's in one PC so I can offer some advice:

1) yes, Moo! will use both GPU's at the same time for each task as you've seen

2) yes, Collatz will use one GPU for each task and will process two tasks concurrently, one on each GPU

I would leave Moo! on autoselect for the time being...it will assign tasks that suit the GPU's and you'll be rewarded accordingly.

(In my experience, if the Moo! tasks were downloaded before you fitted the 2nd GPU, then it could be that the tasks in the cache were those suited for a single GPU...things should be different, once you have new tasks downloaded after you fitted the 2nd GPU).

regards
Tim

Re: 2 GPU prob

Posted: Sat Nov 04, 2017 6:42 pm
by UBT - Mikee
Hi

Thanks for that.

Collatz seems to be running slower than expected - compared to Tackleaways' it's running at the same speed but his is a HD7870 which has a passmark of 4350 compared to mine at 8636.
Moo's are also running at the same speed as before I added the extra GPU - further investigation required!

Mike

Re: 2 GPU prob

Posted: Sun Nov 05, 2017 12:50 am
by chriscambridge
Mike you probably already have, but just to double check, are you using the Collatz speed up code?

I am sure when Mark comes across this post here will list the speed up code for a GTX 1060 (if one is available).

--

Tim, do you know how many GPUs Moo can crunch across? And do they have to be exactly the same series (970, 1080, 1080Ti, etc)??

I do not run Moo but it would be interesting to find out..

Re: 2 GPU prob

Posted: Sun Nov 05, 2017 4:43 pm
by Woodles
Hi Mike,

As Tim said, Moo will use all the GPUs available to it, I've seen it running on four GPUs at the same time but I don't know what the limit is.

Each WU has a certain number of 'packets' inside it and it loads a subset of these onto each GPU. As a GPU finishes it's work, it's loaded with another set of packages until all the work is done. This means that different capability GPUs can work on the same WU but it also means that once the last packets have been loaded onto a GPU, any GPU that finishes it's work after that will sit idle. There's quite a few forum posts supposedly detailing how you can force one WU per GPU but it usually means running multiple copies of Boinc and limiting each to a specific GPU. Not tried any, I didn't think it was worth the effort for the slight gain.

All WUs downloaded when one GPU was installed will only run on one GPU, adding another GPU will cause it to sit idle as Moo doesn't seem to want to run more than one WU at a time (probably why you have to run multiple Boinc instances to get one WU per GPU) Same with any WUs downloaded when two GPUs are installed, Moo will refuse to run them on one GPU and won't download any more work until they're done.

Collatz only uses one GPU for each WU so no problems there.

As far as I can see, you're slightly faster than Tackleway on Collatz? I believe AMD GPUs are slightly faster than Nvidia on Collatz but I would more suspect that the passmarks are inaccurate.

If you really want to beat him then I would point you at viewtopic.php?f=93&t=6001&p=111354#p111354 Using the speed up files should drop the execution times from 20 - 30 minutes that you're presently getting to 10 - 13 minutes. Credit per WU goes down slightly but nowhere near enough to cancel the gains out :D

Mark

Re: 2 GPU prob

Posted: Sun Nov 05, 2017 5:12 pm
by UBT - Mikee
Hi

Thanks for the advice. I'm having a play with the Collatz app as I write. All the other GPU tasks I've tried seem to be playing right. Milkyway, Collatz, Primegrid are all about what I'd expect - it's just Moo that's not right. WUs are taking the same amount of time with 2 GPUs as with 1. My gut feeling is that the GPUs are doing the same work for each WU rather than sharing - sort of half each.

One thing that was weird was that I detached and redownloaded Moo - the first WU ran for 1093 secs - sorted thought I - however, each subsequent WU ran for around 2000 again...

Mike

[edit]
Collatz reduced from 1900 seconds to 1030 - result! Doesn't recognise 'Unrecognized command: kernels/reduction=48', just changed that to 'kernels_per_reduction=48' see what that does.

Re: 2 GPU prob

Posted: Sun Nov 05, 2017 6:52 pm
by Woodles
Hi Mike,

Sorry, kernels_per_reduction is the correct setting.

Mark

Re: 2 GPU prob

Posted: Mon Nov 06, 2017 1:01 am
by chriscambridge
Nearly a 50% speed up! Wow..

Re: 2 GPU prob

Posted: Mon Nov 06, 2017 8:32 am
by Woodles
Yes but credit per WU does drop by ~15% :D

Re: 2 GPU prob

Posted: Mon Nov 06, 2017 8:39 am
by Woodles
An even better improvement on a 1080ti:

Basic - https://boinc.thesonntags.com/collatz/r ... =139610213

'Improved' - https://boinc.thesonntags.com/collatz/r ... =139607582

68% reduction in time, 24% reduction in credit. Three times as many tasks, 2.4 times as much credit :)

Mark

Re: 2 GPU prob

Posted: Mon Nov 06, 2017 9:40 am
by UBT - Mikee
Hi

Me again. Collatz are now averageing around 950 seconds/WU. So this time last week I was getting through 2 WUs per hour @ 36000 each (72000). Today it's 8 WUs per hour @ 30000 each (240000) - theoretically that's 5.7 million a day. Not gonna happen though as it's gets a bit noisy with both cards going flat out. Silent mode only reduces GPU speed by 2% and makes naff all difference!

Still can't get Moo to play properly which is a shame as that's the one I wanted to get right, but I'm not wasting resources doing, effectively, the same WU twice.

Mike

Re: 2 GPU prob

Posted: Mon Nov 06, 2017 9:45 am
by Woodles
Very nice Mike, quite an improvement :)

Go on, you know you have to run flat out just to say you've earned 5.7 million credits in a day :D

Shame about Moo, I've never had any problems with it so can't help I'm afraid. Still, there's other projects available.

Mark

Re: 2 GPU prob

Posted: Mon Nov 06, 2017 8:13 pm
by UBT - Mikee
Hi

Moo is weird! Just done a few more and once again the first one went through at 1028 seconds, subsequent ones at around 2000 - time for a post to Moo!

Mike

Re: 2 GPU prob

Posted: Mon Nov 06, 2017 9:46 pm
by UBT - Timbo
UBT - Mikee wrote:Hi

Moo is weird! Just done a few more and once again the first one went through at 1028 seconds, subsequent ones at around 2000 - time for a post to Moo!

Mike
Hi Mike

Have you cleared the cache of Moo! tasks, since before you added the 2nd GPU, or do you still have some with earlier deadlines than tasks downloaded more recently.

Secondly, are both GPU's running at the same clockspeed and core voltage?

I would suggest you download "MSI Afterburner" and check the settings for both GPU's - as one might be running at a different clockspeed than the other...which could lead to different results.

https://www.msi.com/page/afterburner

You can also try downloading GPU-Z and checking what it finds on each GPU.

https://www.techpowerup.com/gpuz/

regards
Tim

Re: 2 GPU prob

Posted: Mon Nov 06, 2017 10:10 pm
by Woodles
Hi Mike,

What else are you running at the same time as Moo? Your CPU times seems a little excessive, do you have a spare CPU core just for the GPUs?

Mark

Re: 2 GPU prob

Posted: Mon Nov 06, 2017 10:17 pm
by UBT - Mikee
Hi

All suggestions welcome!

Cache well cleared and project removed and reinstalled.

Just done GPUZ and compared both cards side by side and, considering they're made by different manufacturers, the results are near identical - the only differences are a slight change in BIOS version, sub vendor and device ID - everything else is identical.

I'm using GPU Tweak to monitor. When Moo is running both cards show near identical graphs, can't check at the moment but whilst running Collatz the only difference is GPU Clock speed, the Zotec is at 1595MHz, the ASUS at 1734MHz but as I'm doing 2 WUs at different percentages of completion this may be normal - I'll see what happens when I download some Moos tomorrow - site seems to be offline at the mo and my cache is full. As you suggest it maybe different clockspeed making a difference but I don't see why!

Mike

Re: 2 GPU prob

Posted: Mon Nov 06, 2017 10:21 pm
by UBT - Mikee
Woodles wrote:Hi Mike,

What else are you running at the same time as Moo? Your CPU times seems a little excessive, do you have a spare CPU core just for the GPUs?

Mark
Hi

You know what I'm running at the same time!! :-)

I've tried reducing cores - I'll try that for a bit longer tomorrow, though I did notice today that BOINCManager automatically reduced core usage when both GPUs are running. Currently it's running 15 out of 16 threads as Collatz is using 0.896 each - thought it would have reduced to 14 but there you go.

Mike

Re: 2 GPU prob

Posted: Tue Nov 07, 2017 1:20 am
by UBT - Timbo
UBT - Mikee wrote:Hi

<snip>

I've tried reducing cores - I'll try that for a bit longer tomorrow, though I did notice today that BOINCManager automatically reduced core usage when both GPUs are running. Currently it's running 15 out of 16 threads as Collatz is using 0.896 each - thought it would have reduced to 14 but there you go.

Mike
Hi Mike

In the "good old days", one would use "BOINC Manager > Options > Computing Preferences"

and then set the "Computing > Usage limits" by a percentage amount, based on the number of CPU cores you had.

So, if using ONEx GPU, on Collatz (which uses, in your case 0.896 of a core), so a 16 core uses 100%, so 15.104 "cores" (16 minus 0.896) would mean you can use 94.4% of your CPU usage for CPU tasks. Call it 94%

If using 2x GPU, then we're talking about 14.208 "cores" which is a usage percentage of 88.8% - so say 88%.

This also assumes you want to use as many cores as possible for BOINC and you are not trying to run memory intensive tasks on the CPU, such as multiple tabs in your browser, or running very CPU intensive games etc.

Maybe you can adjust things manually?

I'm only using BOINC Manager v7.8.2 and that doesn't "auto manage" core usage (to the best of my knowledge :-( )

So, it's possible that one GPU is being starved of resources and hence is taking longer to crunch, than another ?

I think a bit more "tinkering" with your BOINC Manager settings might be called for.


In my case, running 2x GTX970's, Moo! uses 0.2 of a CPU and both GPU's....so, on my octo-core CPU, I have set my Usage percentage set at 97% and Moo! is pretty consistent, taking 1,850 seconds on average, but with a very small number taking as little as 1,700 secs and a similarly small number taking 1,973 secs....so, the min-max spread is approx +/-7% of the average

regards
Tim

Re: 2 GPU prob

Posted: Tue Nov 07, 2017 1:28 am
by UBT - Timbo
UBT - Mikee wrote:Hi

<snip>

I'm using GPU Tweak to monitor. When Moo is running both cards show near identical graphs, can't check at the moment but whilst running Collatz the only difference is GPU Clock speed, the Zotec is at 1595MHz, the ASUS at 1734MHz but as I'm doing 2 WUs at different percentages of completion this may be normal - I'll see what happens when I download some Moos tomorrow - site seems to be offline at the mo and my cache is full. As you suggest it maybe different clockspeed making a difference but I don't see why!

Mike
Check the settings for GPU Tweak and see if you can adjust both GPU's to run at the same clock speed.

I know that AfterBurner allows this...so, maybe you can set both GPU's to the same speed - try the lower speed first of course and see if this helps "normalise" your crunching time.

regards
Tim

Re: 2 GPU prob

Posted: Tue Nov 07, 2017 8:24 am
by Woodles
UBT - Mikee wrote:You know what I'm running at the same time!! :-)
Well there's your problem :twisted: :lol:
UBT - Mikee wrote:I've tried reducing cores - I'll try that for a bit longer tomorrow, though I did notice today that BOINCManager automatically reduced core usage when both GPUs are running. Currently it's running 15 out of 16 threads as Collatz is using 0.896 each - thought it would have reduced to 14 but there you go.

Mike
I believe Boinc Manager will see two lots of 0.896 as being more than one core so will reserve that, probably doesn't do floating point maths :) In which case, MOO probably doesn't get a core to itself with only requesting 0.2 of a core.

As a quick check, how many CPU tasks are running while MOO is? As a slightly longer test, try disabling all CPU projects completely and see how Moo works. I think you may need to manually reserve a core for the GPUs (Computing preferences / use at most 99% of the CPUs)

Mark

Re: 2 GPU prob

Posted: Tue Nov 07, 2017 9:21 am
by UBT - Mikee
Hi

Right sorted it!

Tasks ARE taking just 17 minutes.

Take a look at this WU https://moowrap.net/result.php?resultid=63919674
CPU and run time is 31 mins 38 seconds, but if you look at the start and finish times (8:57 to 9:14) it's 17 mins. I was watching the WU in the manager and couldn't understand why it was at 90% at around 10 mins.

So Moo seems to have a error at their side!

Mike

Re: 2 GPU prob

Posted: Tue Nov 07, 2017 9:43 am
by Woodles
Hi Mike,

Can you check the Boinc Manager messages to see when it thinks it started that task?

Seems to have been reported straight away so the finish times should be accurate.

Mark

Re: 2 GPU prob

Posted: Tue Nov 07, 2017 9:54 am
by UBT - Mikee
Hi

https://moowrap.net/result.php?resultid=63918168

According to the event log the last one started at 9:31 and finished at 9:49 = 18 minutes. STDERR txt agrees with start/finish times.

Mike

Re: 2 GPU prob

Posted: Tue Nov 07, 2017 9:55 am
by Woodles
Hi Mike,

Wonder why Boinc/Moo thinks it took 31 minutes then?

Mark

Re: 2 GPU prob

Posted: Tue Nov 07, 2017 10:15 am
by UBT - Timbo
Woodles wrote:Hi Mike,

Wonder why Boinc/Moo thinks it took 31 minutes then?

Mark
Hi Mark

Could it be that Windows is getting confused and seeing TWO GPU's but (presumably) only ONE monitor and hence BOINC isn't refreshing the screen at the right time, so Windows isn't updated and hence doesn't update the BOINC Manager logs in a timely fashion?

I do recall, in the past that if one has a multiple GPU rig, that a dummy VGA plug is sometimes required to force windows to think a second (and subsequent) monitor is connected - so, might that be related?

regards
Tim

Re: 2 GPU prob

Posted: Tue Nov 07, 2017 10:33 am
by UBT - Mikee
UBT - Timbo wrote:
I do recall, in the past that if one has a multiple GPU rig, that a dummy VGA plug is sometimes required to force windows to think a second (and subsequent) monitor is connected - so, might that be related?

regards
Tim
Hi
When I was putting in the new card I was looking for a link to connect the 2 cards together (Crossfire?) - I had a card in the original box that looked like a link but there's nowhere to fit them on either of the cards so I assume they don't bother with that anymore. I do recall that these cards were recommended NOT to link them for BOINC use. I remember about the dummy plug and was thinking about that - haven't got one but I do have a couple of smaller spare monitors - I'm going to connect one up and see what happens.
Mike

Re: 2 GPU prob

Posted: Tue Nov 07, 2017 11:01 am
by Woodles
Hi Tim,

I have a couple of systems with dual GPUs in and none of them have dummy monitor plugs (in fact, none of them has any monitor at all connected!) All run at the 'proper' times, roughly half of what a single GPU does.

Mikes results show both GPUs are being recognised and used, it's just by the time the output gets to Moo it thinks they've taken twice as long as they actually have, weird!

Hi Mike,

Crossfire is for AMD cards, SLi for Nvidia but the 1060 does neither (the 1070 and 1080 only do two way, Nvidia is seriously restricting it)

As far as Boinc is concerned, there's no advantage to linking them and some disadvantages so you're not missing anything.

Since the tasks are actually only taking 17 minutes each, does it really matter what the results page says? You're still doing more WUs, you just have a 46 hour day :D

Mark

Re: 2 GPU prob

Posted: Tue Nov 07, 2017 11:16 am
by UBT - Mikee
Woodles wrote:
Since the tasks are actually only taking 17 minutes each, does it really matter what the results page says? You're still doing more WUs, you just have a 46 hour day :D

Mark
Hi

You're right - just have to ignore it. Tried a 2nd monitor - didn't make a difference!

Mike