Project recently started using too much memory?

log in

Advanced search

Message boards : Questions/Problems/Bugs : Project recently started using too much memory?

1 · 2 · Next
Author Message
Greg
Project administrator
Send message
Joined: 26 Jun 08
Posts: 582
Credit: 223,912,432
RAC: 15,881
Message 165 - Posted: 24 Oct 2009, 2:45:12 UTC

I have released workunits for larger factorizations that use the lasievef application. This application uses up to 1 GB of memory. BOINC should not download or start these workunits if there is insufficient memory for them, but there are a few reports that it is. You can manually choose not to run these workunits by deselecting lasievef in NFS@Home preferences in your account. Workunits for the lasievee application will remain available.

And, as always, thanks for your contributions!

Profile denim
Avatar
Send message
Joined: 19 Sep 09
Posts: 17
Credit: 10,253,313
RAC: 5,298
Message 185 - Posted: 28 Oct 2009, 20:42:52 UTC
Last modified: 28 Oct 2009, 20:42:59 UTC

Thanks for the info.
____________

Profile wreck2002
Send message
Joined: 5 Sep 09
Posts: 16
Credit: 26,898
RAC: 0
Message 193 - Posted: 29 Oct 2009, 16:21:50 UTC - in response to Message 165.

Is there any way to reduce the memory that the programe lasievef use?
1G seems not acceptable for most of the computer I see,but as the number grow bigger,it will be more and more number need to use 16e.

Greg
Project administrator
Send message
Joined: 26 Jun 08
Posts: 582
Credit: 223,912,432
RAC: 15,881
Message 195 - Posted: 29 Oct 2009, 18:00:17 UTC - in response to Message 193.

Unfortunately no, not without significantly slowing the computation. I agree that 1GB / core is a lot for most of today's computers, but the good news for those without enough memory to currently run lasievef is that there are enough numbers in the lasievee range to keep the project busy for years. Hopefully in a few years, 1GB/core will be typical.

Profile denim
Avatar
Send message
Joined: 19 Sep 09
Posts: 17
Credit: 10,253,313
RAC: 5,298
Message 200 - Posted: 31 Oct 2009, 14:47:22 UTC

Is there a way to adjust which type of WU's go to which machine of ours? I ask because I have 2 machines that can handle the load of the big 1gig WU's, but 3 that really struggle.

Greg
Project administrator
Send message
Joined: 26 Jun 08
Posts: 582
Credit: 223,912,432
RAC: 15,881
Message 202 - Posted: 31 Oct 2009, 17:46:13 UTC - in response to Message 200.

Yes. BOINC allows you to designate profiles called home, school, and work. You can, say, call the computers without sufficient memory "school" computers by viewing the computers on your account, selecting details for the low memory computers, then changing location near the bottom. Then, in your NFS@Home preferences you can add separate preferences for school and disable lasievef in those preferences. Let me know if you have questions.

Profile denim
Avatar
Send message
Joined: 19 Sep 09
Posts: 17
Credit: 10,253,313
RAC: 5,298
Message 213 - Posted: 5 Nov 2009, 6:28:23 UTC

Thanks Greg. :)

bdodson*
Send message
Joined: 2 Oct 09
Posts: 50
Credit: 99,807,428
RAC: 20,874
Message 214 - Posted: 6 Nov 2009, 17:58:42 UTC - in response to Message 202.

... BOINC allows you to designate profiles called home, school, and work. You can, say, call the computers without sufficient memory "school" computers by viewing the computers on your account, selecting details for the low memory computers, then changing location near the bottom. Then, in your NFS@Home preferences you can add separate preferences for school and disable lasievef ...


On our Xeon cluster(s) the memory use for lasievef on the new
target R269 is nowhere near 1Gb:
19 708m 330m 724 R 100.1 0.7 4:51.67 lasievef_1.07_x 19 627m 322m 720 R 100.1 0.7 4:45.84 lasievee_1.07_x

which shows lowest priority 19, then virtual memory, then
RAM used. I'm switching these from "school" to "work", with
the latter profile _only_ accepting lasievef tasks. If
sufficiently many other users/computers are set like this,
Greg may have to monitor the available tasks (in the server
settings) to make sure that we don't run out of lasievef tasks.
This hasn't been a problem with few people selecting the
"large" or "huge" setting.

I'm especially interested in the factors of R269, a number
with larger "difficulty" (on the "Status of Numbers" page).
These numbers get more extensive ECM pretests, and keeping
numbers requiring the 16e siever feasible gives us a larger
pool of interesting candidates. -bdodson

Greg
Project administrator
Send message
Joined: 26 Jun 08
Posts: 582
Credit: 223,912,432
RAC: 15,881
Message 215 - Posted: 6 Nov 2009, 18:07:17 UTC

One option you can use is to select only lasievef to receive only those tasks, but then select "If no work for selected applications is available, accept work from other applications" so if lasievef does run out of work, then your computer won't be idle.

bdodson*
Send message
Joined: 2 Oct 09
Posts: 50
Credit: 99,807,428
RAC: 20,874
Message 216 - Posted: 6 Nov 2009, 19:36:19 UTC - in response to Message 215.

...select "If no work for selected applications is available, accept work from other applications" so if lasievef does run out of work, then your computer won't be idle.


Done, thanks. Just to be clear, I'm still banning the 16e siever
("lasievef") from linux machines with 1Gb/core or less memory, as
well as on the Windows machines. Task Manager reports 356,172K for
15e ("lasievee"), but there's no reason to risk having boinc being
in the way. -bdodson

k6xt
Send message
Joined: 16 Nov 09
Posts: 2
Credit: 159,976
RAC: 499
Message 268 - Posted: 5 Dec 2009, 23:15:58 UTC - in response to Message 165.
Last modified: 5 Dec 2009, 23:19:28 UTC

Wondering why one of my PCs was brought to its knees I just discovered the high memory use. I don't mind doing the work but need a way to stop just NFS processing but not the rest of the projects I'm running. So far I don't see a way to stop just NFS when I sit down and use the PC, without stopping all computation. Is there a way?

The undesirable alternative is to stop receiving these WU. Each of my 4 processors is taking a half GB of the 2GB available making the PC almost unusable. Takes me 5 minutes just to get into Boinc Manager to shut it down.

Thanks Art
____________

Greg
Project administrator
Send message
Joined: 26 Jun 08
Posts: 582
Credit: 223,912,432
RAC: 15,881
Message 272 - Posted: 6 Dec 2009, 22:33:10 UTC - in response to Message 268.

In the BOINC Manager, under Advanced, Preferences, in the Disk and Memory Usage tab, I suggest lowering the memory usage to 50% or lower when the computer is in use and 75% or 80% when the computer is idle. This should cause BOINC to run at most three NFS@Home tasks at a time. If you have another, low memory project also active, BOINC should run one of those in the fourth slot.

Profile Paul D. Buck
Send message
Joined: 17 Sep 09
Posts: 7
Credit: 540,914
RAC: 0
Message 274 - Posted: 7 Dec 2009, 10:03:42 UTC - in response to Message 272.

In the BOINC Manager, under Advanced, Preferences, in the Disk and Memory Usage tab, I suggest lowering the memory usage to 50% or lower when the computer is in use and 75% or 80% when the computer is idle. This should cause BOINC to run at most three NFS@Home tasks at a time. If you have another, low memory project also active, BOINC should run one of those in the fourth slot.

This is another place where a "throttle" on the number of simultaneous project tasks would be very good. Maureen (sp?) at CPDN also was supportive of the idea for their project but Dr. Anderson sadly is not. I think I ran into a problem the other day with too many tasks trying to run at the same time and so I got a whole slew of them that failed because the system was trying to run too many at once, then tried to increase swap memory and the tasks blocked and failed.

k6xt
Send message
Joined: 16 Nov 09
Posts: 2
Credit: 159,976
RAC: 499
Message 277 - Posted: 8 Dec 2009, 16:22:37 UTC - in response to Message 272.

In the BOINC Manager, under Advanced, Preferences, in the Disk and Memory Usage tab, I suggest lowering the memory usage to 50% or lower when the computer is in use and 75% or 80% when the computer is idle. This should cause BOINC to run at most three NFS@Home tasks at a time. If you have another, low memory project also active, BOINC should run one of those in the fourth slot.


When I started in on distributed computing (Sept 99) the explanation was that SETI would instantly release whatever resources were required when another application needed them. I took this to mean any application, not just user requests for CPU cycles and memory. Until lately that has seemed true. Implicit in your recommendation is that NFS will NOT release resources as needed by user or PC. Is that true, or have I read too much into this?

I took the 50% suggestion but set idle memory down from 100 to 95% try that for a time.

Regards
Art

____________

bdodson*
Send message
Joined: 2 Oct 09
Posts: 50
Credit: 99,807,428
RAC: 20,874
Message 368 - Posted: 19 Jan 2010, 0:26:55 UTC - in response to Message 216.

... Just to be clear, I'm still banning the 16e siever
("lasievef") from linux machines with 1Gb/core or less memory, as
well as on the Windows machines. ... there's no reason to risk
having boinc being in the way. -bdodson


Lots of other users seem to have taken this view as well, with
server status sometimes showing < 4000 "16e results in progress".
Recently this has been tending upward, in the 5000's. Today I'm
seeing 6,217. I know why I'm back to running these on the linux
machines with sufficient resources, but am wondering who/why there's
such a strong uptick?

A possible reason is that we're currently sieving M899 = 2^899-1,
one of the Mersenne numbers. When this one finishes, the next 16e
target is another "repunit"

R271 = (10^271-1)/(10-1) = 111...1 (271 ones)

the next one after the number R269 = 10, 269- from back in Dec.
Perhaps these are seen as somewhat higher profile targets than
some of the other Cunninghams? Just wondering. -Bruce

Adrian Ware
Send message
Joined: 27 Dec 09
Posts: 1
Credit: 51,568
RAC: 0
Message 371 - Posted: 20 Jan 2010, 9:09:49 UTC

Apologies if my terminology is inaccurate.

I notice that 2,899- threads running 16e are requesting 1.25 gig of memory (only checked a couple), and I only have 2 gig, so that will slow down processing on my machine (running 2 threads).

Not that this will have much impact on the overall scheme of things, but do wonder how many others are in the same boat.

skivelitis
Avatar
Send message
Joined: 29 Mar 10
Posts: 1
Credit: 10,057
RAC: 0
Message 425 - Posted: 4 Apr 2010, 11:28:31 UTC
Last modified: 4 Apr 2010, 11:30:22 UTC

How much Ram do the 14e apps use when work is available? I dont have enough Ram to run the other apps but would love to crunch something.
____________

Greg
Project administrator
Send message
Joined: 26 Jun 08
Posts: 582
Credit: 223,912,432
RAC: 15,881
Message 426 - Posted: 4 Apr 2010, 19:02:09 UTC - in response to Message 425.

The 14e app would use up to about 200 MB, but I don't anticipate using it in the near future. The numbers we are currently working on are simply too large for 14e. If you are interested in NFS sieving with 14e, the RSA Lattice Siever project is currently sieving smaller numbers with 14e.

Tex1954
Send message
Joined: 21 Jun 11
Posts: 2
Credit: 5,118,135
RAC: 0
Message 767 - Posted: 23 Nov 2011, 8:54:58 UTC

I would also point out that Windows 7 uses 1.3GIG on the lasievef 16e tasks and Linux on the same system uses about 800Meg Real and 800Meg Virtual. The two OS's certainly do things differently...

I have 12Gig DDR3 in my i7-950 system (8 threads)to support 8 NFS 16e tasks and it uses it for sure!

This is running 8 lasievef 16e tasks...




Tex1954

Alan Lee
Send message
Joined: 16 Nov 15
Posts: 1
Credit: 0
RAC: 0
Message 1612 - Posted: 16 Nov 2015, 11:48:14 UTC

Hi I am Alan, I just joined the project.
I wonder how can I tame the CPU / memory usage as it is taking to much of my resources.
I am running on MAC OS Yosemite.
Thanks.
____________
Please support my project and my old past papers preserving initiative.

1 · 2 · Next

Message boards : Questions/Problems/Bugs : Project recently started using too much memory?


Home | My Account | Message Boards