Pages: [1] 2
CADENCE
BAM!ID: 62735
Joined: 2008-12-14
Posts: 6
Credits: 5,507,700
World-rank: 81,800

2009-01-08 02:55:53

Hello.

Inspired by a discussion on this forum , I created a small and handy Linux tailored for DC nodes. It will allow you to boot your DC node directly from a 512MB USB stick, so you don't need any harddrives, etherboot configurations, NFS servers etc. And it is also much faster than installing new operating system on your DC machine.

I have been using it myself for the last week or so, and it works pretty well. I even use it to boot my x86_64 machine that has 32bit WinXP installed to crunch faster at night.

To make things more clear and create a base for future improvement, I made this simple website dedicated to this project: http://obfusc.at/ed/dclinux.html

I will try to keep it up to date with new versions.
I'm willing to spend some time improving this, so if you have any feedback, comments, propositions or wishes - please do let me know.

Here is a copy&paste of the advantages of using this special linux version, taken from my website:
- Easy to install. There is no hustle with configuration files, network set-up, or any hardware changes to your DC box.
- Requires only 512MB of disk space, so you can reuse your old USB stick or buy a cheap new one (2GB costs less than $10 these days).
- Has BOINC client installed on it and configured to run out-of-the-box.
- Provides SSH daemon (by default turned off)
- Completely secure. By default, there are no daemons running, except BOINC.
- During system startup /var directory is copied to ramdisk, and then synchronized between ramdisk and filesystem every 6 hours and on system shutdown. This saves USB write cycles which is important as most USB flash drives have limited write operations.
- Has x86_64 kernel. It comes handy for those systems where main OS is 32 bit, but you want to use the machine for crunching at night and take advantage of 64-bit CPU.
- It can spin down your hard drives to lower power consumption, heat and noise.
- It is Debian-based so you can apt-get anything you want and fine-tune the system according to your needs.

So far it supports x86_64 only, but I'm planning to create a version for X86 in the next few days.

Cheers.
CADENCE
BAM!ID: 62735
Joined: 2008-12-14
Posts: 6
Credits: 5,507,700
World-rank: 81,800

2009-01-15 03:31:43

As promised, x86 version is out. I also improved the x86_64 version (which became 0.2 now). I also included information how to copy this image to a USB stick under Windows.
Check it out: http://obfusc.at/ed/dclinux.html
PatHawks
 
BOINCstats SOFA member
BAM!ID: 63474
Joined: 2008-12-30
Posts: 150
Credits: 492,666
World-rank: 310,379

2009-01-20 15:28:31

This is great, man! I've been looking for something just like this.

Now what we need to do is make it installable, so I can dual boot a machine with "Windows (or whatever)" and "BOINC mode," which is totally optimized for each users machine.
CADENCE
BAM!ID: 62735
Joined: 2008-12-14
Posts: 6
Credits: 5,507,700
World-rank: 81,800

2009-01-21 01:33:05

Why do want need it installable as dual boot? The whole idea is that when you run it directly from the USB stick, you don't have to mess up with your hard disk and any configuration. Just plug the stick, set in your BIOS to start from it and voila
Price shouldn't really be a factor here - I recently saw some 1GB USB sticks for $3 since they are trying to get rid of them. With a OS on a USB stick you can also spin-down your hard drives, which means less electricity, noise and heat.

Or maybe I just don't fully understand your idea. Why would you like to boot from a hard drive instead of a USB stick?
PatHawks
 
BOINCstats SOFA member
BAM!ID: 63474
Joined: 2008-12-30
Posts: 150
Credits: 492,666
World-rank: 310,379

2009-01-21 01:43:55

Why would you like to boot from a hard drive instead of a USB stick?

Two things.
1.) No RAM disk to hog system memory.
2.) Custom built kernel with processor optimizations.
noderaser
 
BAM!ID: 13859
Joined: 2006-12-03
Posts: 840
Credits: 508,813,771
World-rank: 3,863

2009-01-21 06:08:47

Maybe you should look at Dotsch's Dotsch_UX instead; it is installable on a hard disk, and is based on Ubuntu...

http://www.dotsch.de/boinc/Dotsch_UX.html
PatHawks
 
BOINCstats SOFA member
BAM!ID: 63474
Joined: 2008-12-30
Posts: 150
Credits: 492,666
World-rank: 310,379

2009-01-21 06:16:57

Maybe you should look at Dotsch's Dotsch_UX instead; it is installable on a hard disk, and is based on Ubuntu...

Doesn't look too bad, but why in the world would anybody want GNOME on a dedicated BOINC system?
Dotsch
Tester
BAM!ID: 833
Joined: 2006-05-27
Posts: 75
Credits: 4,556,084
World-rank: 91,996

2009-01-21 17:35:13

Maybe you should look at Dotsch's Dotsch_UX instead; it is installable on a hard disk, and is based on Ubuntu...

Doesn't look too bad, but why in the world would anybody want GNOME on a dedicated BOINC system?

Linux Beginners like the GNOME installation. But, it is configurable, if you did not like or need it, you can disable it. The each install script for USB, HDD installation would ask you, if you like to disable GNOME. For the diskless systems it's disabled by default.
PatHawks
 
BOINCstats SOFA member
BAM!ID: 63474
Joined: 2008-12-30
Posts: 150
Credits: 492,666
World-rank: 310,379

2009-01-21 17:49:27

The each install script for USB, HDD installation would ask you, if you like to disable GNOME. For the diskless systems it's disabled by default.

Now you've got my attention, sir.

I'm a a Fedora/Gentoo guy myself, but I'm amusing once I install it to a USB/HDD, configuring and making the kernel is all pretty straight forward?
Dotsch
Tester
BAM!ID: 833
Joined: 2006-05-27
Posts: 75
Credits: 4,556,084
World-rank: 91,996

2009-01-21 19:41:01
last modified: 2009-01-21 19:42:29

The each install script for USB, HDD installation would ask you, if you like to disable GNOME. For the diskless systems it's disabled by default.

Now you've got my attention, sir.

I'm a a Fedora/Gentoo guy myself, but I'm amusing once I install it to a USB/HDD, configuring and making the kernel is all pretty straight forward?

With my installation it's quite easy and straight forward. Have a look at this thread http://boincstats.com/forum/forum_thread.php?id=4129 or at my homepage at http://www.dotsch.de/Dotsch_UX.
If you would do this with Fedora or Gentoo some work is required...
CADENCE
BAM!ID: 62735
Joined: 2008-12-14
Posts: 6
Credits: 5,507,700
World-rank: 81,800

2009-01-22 00:03:53

Why would you like to boot from a hard drive instead of a USB stick?

Two things.
1.) No RAM disk to hog system memory.
2.) Custom built kernel with processor optimizations.


The system memory wasted for RAMDISK amounts to less than 100MB. Is it really that much?
I've got two versions there: one for X86 and the other one optimized for x86_64 only. I don't think you can get any real performance bump by compiling kernel on your own machine (but if you can prove me wrong, I may consider doing it the way you proposed ).


PatHawks
 
BOINCstats SOFA member
BAM!ID: 63474
Joined: 2008-12-30
Posts: 150
Credits: 492,666
World-rank: 310,379

2009-01-22 00:10:54

I've got two versions there: one for X86 and the other one optimized for x86_64 only. I don't think you can get any real performance bump by compiling kernel on your own machine (but if you can prove me wrong, I may consider doing it the way you proposed ).

I've got Gentoo installed on a boat anchor.
After I recompiled the kernel with machine specific optimizations, my benchmarks tripled.

Old: 691.17 million flops
New: 1974.28 million flops

And it's not just a benchmark thing. WUs are defiantly being processed noticeably faster.
That's why I'm interested in being able to compile my own kernel.
CADENCE
BAM!ID: 62735
Joined: 2008-12-14
Posts: 6
Credits: 5,507,700
World-rank: 81,800

2009-01-22 01:18:52


I've got Gentoo installed on a boat anchor.
After I recompiled the kernel with machine specific optimizations, my benchmarks tripled.

Old: 691.17 million flops
New: 1974.28 million flops


This difference is too big. There is really something wrong with Gentoo kernel What kind of machine is that? How much memory? Maybe the kernel was so huge that it was consuming all the memory and your application was using swap? Hard to say what happened here, but I suspect this kind of performance boost can only be achieved on few peculiar machines. Would be interesting to see your results using my DCLinux (based on Debian) or Dotsch's Dotsch_UX (based on Ubuntu) on that machine, though.
PatHawks
 
BOINCstats SOFA member
BAM!ID: 63474
Joined: 2008-12-30
Posts: 150
Credits: 492,666
World-rank: 310,379

2009-01-22 02:55:18

What kind of machine is that? How much memory?
It's an old Dell XPS laptop. P4 3.2GHz with 2GB RAM

Maybe the kernel was so huge that it was consuming all the memory and your application was using swap?
I wouldn't be surprised if that's what was happening.
The stock kernel loaded drivers for everything attached to the machine.
My custom kernel has no support for USB, mouse, any filesystems other than ext2. No video drivers. No modules or RAM drives.
It's only enough to get things going and turn it over to BOINC.

Would be interesting to see your results using my DCLinux (based on Debian) or Dotsch's Dotsch_UX (based on Ubuntu) on that machine, though.
I'm really curious too. If I get a chance tonight or tomorrow, I'll try it out and post some benchmarks here.
Dotsch
Tester
BAM!ID: 833
Joined: 2006-05-27
Posts: 75
Credits: 4,556,084
World-rank: 91,996

2009-01-25 14:40:53

The BOINC benchmark is no good way to measure the performance improvements. If you would measure the real perfomance, you need to make a perfomance benchmark with a real science application with the same WU.
From the reports I have seen, gaves a modified/optimized kernel about 1..2 % difference in the floating/integer perfomacne.
PatHawks
 
BOINCstats SOFA member
BAM!ID: 63474
Joined: 2008-12-30
Posts: 150
Credits: 492,666
World-rank: 310,379

2009-01-25 17:21:34

From the reports I have seen, gaves a modified/optimized kernel about 1..2 % difference in the floating/integer perfomacne.
Well, I guess I don't know what to tell you. I found what works best for me, I guess. If something else works better for some, cool.

FWIW, I tried the BOINC LiveCD on a (different) machine, and performance was down 35-40%
I really think a RAM disk is a waste of precious system resources.
Gundolf Jahn
 
BAM!ID: 16586
Joined: 2007-01-07
Posts: 469
Credits: 447,470
World-rank: 326,112

2009-01-25 17:45:40

I really think a RAM disk is a waste of precious system resources.

IMOH not, if it substantially reduces write cycles to the USB-stick.
CADENCE
BAM!ID: 62735
Joined: 2008-12-14
Posts: 6
Credits: 5,507,700
World-rank: 81,800

2009-01-25 20:36:28


I really think a RAM disk is a waste of precious system resources.


I would disagree with this statement. If you run your system from a CD or USB stick and store file system in RAMDISK, you save on another - more expensive and more power hungry - precious system resource: hard disk. In the case of my DCLinux you only "waste" around 100MB of RAM. My main machine has 4 hard drives. At nigh or when I'm at work, I prefer to run it from a USB stick and spin down all 4 disks saving around 40Watts (and extending life time of my hard drives). And I "pay" for that 100MB out of my 4GB. It's a no-brainer really.
Now, I understand that it may look different for older computers, but as long as you have at least 512MB it shouldn't really be an issue in most cases. Not to mention that by creating a dual-boot system you always have the risk of screwing up you existing data on a hard drive.
PatHawks
 
BOINCstats SOFA member
BAM!ID: 63474
Joined: 2008-12-30
Posts: 150
Credits: 492,666
World-rank: 310,379

2009-01-25 20:40:54

I prefer to run it from a USB stick and spin down all 4 disks saving around 40Watts
I guess since I only have one hard disk, it's less of an issue.
Dotsch
Tester
BAM!ID: 833
Joined: 2006-05-27
Posts: 75
Credits: 4,556,084
World-rank: 91,996

2009-01-25 21:05:46
last modified: 2009-01-25 21:16:58

Sorry, dupplicate post.
Dotsch
Tester
BAM!ID: 833
Joined: 2006-05-27
Posts: 75
Credits: 4,556,084
World-rank: 91,996

2009-01-25 21:16:18
last modified: 2009-01-25 21:18:29

From the reports I have seen, gaves a modified/optimized kernel about 1..2 % difference in the floating/integer perfomacne.
Well, I guess I don't know what to tell you. I found what works best for me, I guess. If something else works better for some, cool.

FWIW, I tried the BOINC LiveCD on a (different) machine, and performance was down 35-40%
I really think a RAM disk is a waste of precious system resources.

How exctactly do you have measured the perfomance ? - As I have stated before, the BOINC client benchmark is no accurate way to measure the system perfomance. There was a big discussion in the past, about the manipulation of the BOINC benchmarks via compiler flags or other methods. Also windows has still higher benchmarks as on Linux on the same system. So the questions, is windows faster on the same system, no. In short, measure perfomance with the BOINC client is a wrong method to make a system comparment benchmark.
You need to make a real application benchmark and let the excatly same WU benchmarking against both OSes.

I have found a lot of analysis in the net, which states all that a optimized kernel did only make about 1 to 2 % difference in floating and integer computations. And this is excatly this, what we do here with BOINC.
Dotsch
Tester
BAM!ID: 833
Joined: 2006-05-27
Posts: 75
Credits: 4,556,084
World-rank: 91,996

2009-01-25 21:25:45


I really think a RAM disk is a waste of precious system resources.


I would disagree with this statement. If you run your system from a CD or USB stick and store file system in RAMDISK, you save on another - more expensive and more power hungry - precious system resource: hard disk. In the case of my DCLinux you only "waste" around 100MB of RAM. My main machine has 4 hard drives. At nigh or when I'm at work, I prefer to run it from a USB stick and spin down all 4 disks saving around 40Watts (and extending life time of my hard drives). And I "pay" for that 100MB out of my 4GB. It's a no-brainer really.
Now, I understand that it may look different for older computers, but as long as you have at least 512MB it shouldn't really be an issue in most cases. Not to mention that by creating a dual-boot system you always have the risk of screwing up you existing data on a hard drive.

Full ack. You should pay attention on multi core systems and also on the higher memory using projects like CPDN, Superlink (700MB/Core) or Rosetta(up to 512 MB/core).
Saving on the RAM in a system is absolutly the contra productiviest way you can do. - What cares about 30 Euro more in a 600 Euro system, when you can increase the throughtput drasticly.
PatHawks
 
BOINCstats SOFA member
BAM!ID: 63474
Joined: 2008-12-30
Posts: 150
Credits: 492,666
World-rank: 310,379

2009-01-25 22:03:06

How exctactly do you have measured the perfomance ?
Let it run for a day, look at how many credits it gets.
Dotsch
Tester
BAM!ID: 833
Joined: 2006-05-27
Posts: 75
Credits: 4,556,084
World-rank: 91,996

2009-01-26 10:14:30

How exctactly do you have measured the perfomance ?
Let it run for a day, look at how many credits it gets.

Have you remove the "powernowd" which was bundled and enabled by a mistake, as I have written it in the tips and tricks section of the documentataion ? - The powernowd infects perfomance...
PovAddict
BAM!ID: 115
Joined: 2006-05-10
Posts: 1013
Credits: 5,785,239
World-rank: 79,285

2009-01-26 13:43:26

How exctactly do you have measured the perfomance ?
Let it run for a day, look at how many credits it gets.

Credits from what project? Many base credits on benchmarks, which as Dotsch said, are flawed.
Not running BOINC anymore for several reasons...
Pages: [1] 2

Index :: BOINC :: Distributed Computing Linux on 512MB USB stick
Reason: