Pages: [1]
student_
    Donator
BAM!ID: 73
Joined: 2006-05-10
Posts: 47
Credits: 5,150,513
World-rank: 84,601

2007-04-04 04:30:48

How long is data exported from the projects stored in BONCstats? I've been thinking about storing the various daily . gz files from the projects on a local drive for a sort of longitudinal study of BOINC.

I did a ballpark, generous estimate of 500MB of total zipped data exported per day from all BOINC projects (taking a look at several large projects' .../stats/ directories). I'm not sure about the rate of change on that 500MB, but will assume for simplicity's sake that it's constant. That sets a 200GB harddrive having capacity for about 400 days (200GB/(0.5GB/day)).

With that data I could do detailed analyses on BOINC projects, and make pretty graphs. In the long term I'd like to incorporate graphs (bell curves, etc. - more than simply chronological bar charts and pie charts) of total credit and RAC, active user retention rates, statistics on active CPUs (omitting skewing inactive ones), and other things.

For those who would like, they could also give information to permit access to their detailed statistics in the .../home.php pages on various projects, to give information on individual work unit completion times and credit earned. US participants could offer postal codes to show regional contributions and compete; similar services could be applied for other countries.

Thus far I'm only automatically downloading some .gz files from the projects to my local drive, but those extra things could be implemented with time. Any insight on the data storage aspect of BOINCstats would be a great help. Thanks.
[BOINCstats] Willy
 
Forum moderator - Administrator - Developer - Tester - Translator
BAM!ID: 1
Joined: 2006-01-09
Posts: 9442
Credits: 353,172,950
World-rank: 4,892

2007-04-04 04:50:35

This is mostly limited by hard drive space which is expensive in a server.

There is not automatic deletion of the files but when the server runs low on HD space I delete the export files older than two months old.
Please do not PM, IM or email me for support (they will go unread/ignored). Use the forum for support.
student_
    Donator
BAM!ID: 73
Joined: 2006-05-10
Posts: 47
Credits: 5,150,513
World-rank: 84,601

2007-04-04 05:48:13

I would be using hard drives of my own, so the extra server price wouldn't be a problem. Googling 'hard drive' shows a 500GB HD for USD 135 - giving USD 0.135 per day cost with the 0.5GB combined BOINC daily exported data figure I estimated before (is that 0.5GB about accurate?). I'd be willing to shell out less than 14 cents per day for almost three years of capacity for that. I've got a C2D 2.4GHz CPU, and would also be willing to eventually invest in more and faster RAM than I have now (currently 2 x 512MB DDR2 @ 533MHz, add 2 x 512MB DDR2 @ 800MHz). Is RAM a significant bottleneck in BOINCstats, or what is/are the bottlenecks?

The next thing to do would be to design new graphical charts (different from those here), and a system to manage them for projects, users and hosts.

I wonder if it'd be better to do this for Folding@home instead. Their main statistics site (http://fahstats.com/) doesn't seem nearly as robust. </internal monologue>
[BOINCstats] Willy
 
Forum moderator - Administrator - Developer - Tester - Translator
BAM!ID: 1
Joined: 2006-01-09
Posts: 9442
Credits: 353,172,950
World-rank: 4,892

2007-04-04 10:59:19

I don't know what your exact plans are, but the major bottlenecks in BS performance is HD speed and RAM quantity. 4GB for the either the database server or the webserver is not enough.
Please do not PM, IM or email me for support (they will go unread/ignored). Use the forum for support.
Pages: [1]

Index :: BOINCstats general :: Lifetime of projects' exported .gz on BOINCstats servers and other dry topics
Reason: