SciLINC

Background info here available

Applications for Win and Lin. (Though no WUs are being send atm)

From Background info:

Cheers Saenger

Quite a unique and original project, I might say.

Sever Page shows 57,882 wu's to send, but BOINC gets "No work from Project"

http://www.scilinc.org/SciLINC/server_status.php

It's this way since I signed up on Monday, was probably never any real WU there.

As there are no message boards, nobody will know unless someone a) writes a mail and gets an answer or b) makes fora available.

They have a blog, where you can at least ask questions (and it was already asked). I'll post a link to this thread in there, perhaps someone will come here.

I had 153 before for some reason they stopped being sent. They all ran 2 to 5 seconds. Windows OK, Linux had all errors. My credit totals 0.68 for that.

That number has been stuck at 57,882 ready to send since 6/7/2007.

I sent an email but got no response.

Thats all I know.

SciLINC will be sending new wu by Monday, June 18. Apologies for the delay!
Chris Freeland

No apologies required, but THX anyway.

And THX for the WU (btw: you say "new wu", that's singular, does this mean there will be only one available? )

Yes, there will just be the one and you'll all have to fight over it! No, eventually there will be more than 650,000 WUs. In short, we're sending down OCR text from scientific literature and finding strings that match characteristics of taxonomic names. The processing is not intense, so WUs fly through quickly.

I was going to report I tried to contact another person about SciLINC and got a response, but I see he, Chris Freeland, Has already posted here.

What he didn't mention was, the other guy, Ron Parker, went on vacation, which is sort of what I guessed. Figures things go foobar when no one is watching the store.

Thanks Chris for the email.

I just got a bunch of work. But then the server went down, and many of those tasks are stuck in downloading and uploading status. Each one took about a second to run.

It occurs to me that,

1) all the work takes only about a second to run

and

2) the leadtime is only 1 day, which means all results downloaded will go into EDF almost immediately, and will return results immediately.

So...their site will be HAMMERED with all the upload/download requests. I wonder if their site is up for that kind of traffic? Perhaps that is why it went down almost immediately after offering units.

Yep. Same for me. Windows works fine, Linux is all errors.

Upon completion, there are around 50 files to upload per result!

Can anyone else attest to this project possibly causing BOINC daemon memory/CPU usage to balloon? Saying this because of the combination of just receiving many SciLINC WU's and now over a thousand files located in the project directory. I posted to BOINC forums.

Personally I had three computers hooked up to SciLINC and I only noticed something funny when I was checking the weather. The PC acted sluggish and the download was slow. This is on my Dual core machine with a 5MB cable modem, I should not feel anything slow.

After fianlly pausing BOINC I noticed that Boinc was still taking 97% cpu power at all tiomes. It took me several Boinc Shutdowns to reset the scilinc project only after I locked my firewall. Once I reset, I detached the project. Finally my pc is working right.

I do not know about you but I will wait on this project to get a forum and find out what the &$%% is going on. I still can't clear my laptop, I had to pull the battery.

Holy crap! It completely locked up 3 dual core machines. No amount of the usual remedies would solve the problem. Something got completely so FUBAR, I had to uninstall BOINC, trash the remaining BOINC folder, reboot, and reinstall BOINC. What a cluster (!*&!

I had several (of the same) projects running on each of those machines.

So after the surgery, I attached only one of each project on each machine. Yep. It's this one.

So...I still have copies of the BOINC folders from just before I NUKED them.

Here's my problem. I has one Seasonal Attribution Project job on each machine, only about *3 days* away from completion. AAARGH! Any way to recover them? Little help?

SciLINC locked up my computer also. It downloaded 50 wu's, each taking 6 seconds to calculate on my P3 1GHz.

But in total its been busy/locked up for 8 hours!

Yeah, FUBAR describes it quite well!

I suspended all other projects, set SciLINC to no new work. Now 8 hours later, everythings back to normal again!

My main host downloaded 50 wu's too. But I wont get there within the next 8 hours... *fingers crossed* It has 2 SAPs also (35% and 5%)...

So far I'm attributing this to the huge number of files each WU uses/generates. My project directory reached a peak of nearly 10,000 files, and BOINC had to keep track of every single one of them. The issue is that while BOINC is still doing work, it'll be next to impossible to communicate with it via RPC. However, looking at stdoutdae.txt showed that it was still transferring files. With at least a hundred files per WU, its obviously going to take some time for BOINC to get them transferred out, despite each one being only a kilobyte or so in size. So I'm going to let it cool down for the next several hours with every other project suspended and SciLINC set to no new work. I had to stop BOINC and manually edit client_state.xml to do this. Hopefully SciLINC server will be up long enough.

Lesson learned is:

Do NOT attach to SciLINC!

Otherwise it will take over BOINC and you will be forced to either uninstall it or it will steal time from you and your other projects.

To admins: The solution may be to ZIP up files before transferring them. I believe its just the vast number of file transfers BOINC is trying to handle thats grinding it to a halt. If everything is in a zip BOINC will have far less to deal with and reduces server overhead. But please test before handing out the WUs to everyone, this is a very major and easy to catch problem!

Ok, I see I'm not the only one who had main problems with the up-/dowloading queue. Also the BOINCmanager used over 40MB RAM when it tried to handle this zillions of files.

I finally managed to "kill" BOINC via Windows TaskManager after 30 minutes of upload on dsl for the first 5 or 6 WUs. After restarting BOINC I suspended network and detached. The upload of over 20 WUs of this project would have taken hours I guess. And I won't even imagine how that hurts dial-upper's.

@SciLINC admins: please reduce the number of files for down-/uploading significantly and/or use ZIP as AlphaLaser suggested below!
AND please open your message board for reporting such bugs!

FWIW, I managed to keep the impact of returning SciLINC results to a minimum by increasing the number of total file transfers from 8 to 16 as well as the number of file transfers per project from 2 to the same 16 (using BoincView). This way, the BOINC client flushes the queue quickly and does not page fault like mad as before and it doesn't hog the system anymore.

HTH

How do you do that ?

I left in run to see how bad it could get.

I found that just running the boinc.exe was better. If I had BOINC manager running it was un-responsive and BOINCview would not update this host. With just the core client BOINCview will update sometimes.

I had done the same thing, but set to max of 8 transfers and only have SciLINC running, all others suspended. I tried 16, but 8 seems better for me. I finally got through all the work. Something like 163 tasks and now that the downloads have stopped and all work done its just uploads left. I was up to something like 5,260 max transfers at one point, and its been running for at least 4 hours since I modified it. IT ran all night in slow motion mode. This is one host.

I want to see how long it takes, from now to clear the last 5,000 uploads.

So this message time represents the 5,000 mark.

Using the "Open BOINC core-client configuration" button in the "Hosts" toolbar of BoincView (http://boincview.amanheis.de/).

With 16/16 the system is quite responsive and as soon as it gets ahold of the SciLINC server, it depletes the results queue in seconds.

HTH

I noted on the thread I posted to the BOINC forums that it took about 16-17 hours for the host to fully recover from the situation. I'm guessing the host pulled between 100-150 WUs, I'd check the results page on SciLINC but it appears down at the moment. BOINCView had trouble contacting the host so I just changed settings by editing files. I pushed my settings to 24/24. I might try an experiment of re-enabling SciLINC with these new transfer settings and monitor closely to see if it is good enough to reduce the backlog that occurred in the first place.