putting OpenBSD/AMD64 on my fileserver ^_^
This last sunday I finally found some time to do a fresh install on my fileserver…
click to go the pic on Flickr
My fileserver’s been an AMD MP1800+ before, (yeah yeah yeah, that’s this one, the slashdot-machine ^_^) running OpenBSD and encrypting everything on-the-fly… In that setup I got around 6MB/sec througput… Enough for everyday use, running MP3’s and stuff off it but I want to upgrade my home network to gigabit(just because…) but that seems a little useless if my fileserver can’t even fill up a 100MBit-line…
Sooo… since a while I got this great Asus SK8N with an AMD Opteron 146 on it, 2.0GHz 64-bit and of course, I wanted to know how much throughput I could expect out of this setup. Now at first I didn’t have the time to do a fresh install, so I ran this new hardware on the old OpenBSD-install for about a month, but finally, last sunday, I started from scratch, using my own How-To actually ^_^, and was up & running again in no time…
So, what’s the score!?!?
Some yummy console-output *^_^*
First of all… what are we working on?
scramjet:markie {19} uname -a
OpenBSD scramjet.macstuff.local 3.9 GENERIC#462 amd64
scramjet:markie {20}
...and out of the dmesg-output:
scramjet:markie {21} dmesg
OpenBSD 3.9 (GENERIC) #462: Thu Mar 2 03:52:16 MST 2006
deraadt@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC
real mem = 536211456 (523644K)
avail mem = 448024576 (437524K)
using 13142 buffers containing 53829632 bytes (52568K) of memory
mainbus0 (root)
cpu0 at mainbus0: (uniprocessor)
cpu0: AMD Opteron™ Processor 146, 2000.25 MHz
cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,NXE,MMXX,LONG,3DNOW2,3DNOW
cpu0: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 1MB 64b/line 16-way L2 cache
cpu0: ITLB 32 4KB entries fully associative, 8 4MB entries fully associative
cpu0: DTLB 32 4KB entries fully associative, 8 4MB entries fully associative
...and of course… the crypto-setup!
scramjet:markie {22} df
Filesystem 1K-blocks Used Avail Capacity Mounted on
/dev/sd0a 2062190 595100 1363982 30% /
/dev/sd0d 5605652 7438 5317932 0% /var
/dev/wd0a 192275708 192275682 -9613758 105% /datacrypt/disk1
/dev/wd1a 192275708 192275682 -9613758 105% /datacrypt/disk2
/dev/svnd0c 189129336 153003564 26669306 85% /data/disk1
/dev/svnd1c 189129336 164217244 15455626 91% /data/disk2
scramjet:markie {23}
...copying some files from my desktop to my fileserver… through sftp…
markie@oblique /data/media/video $ sftp scramjet
Connecting to scramjet…
markie@scramjet’s password:
sftp> cd /data/disk2/temp
sftp> mput PaulRobert*
Uploading PaulRobertson_PirateBabysCabanaBattleStreetFight2006.mpg to /data/disk2/temp/PaulRobertson_PirateBabysCabanaBattleStreetFight2006.mpg
PaulRobertson_PirateBabysCabanaBattleStreetFight2006.mpg 100% 113MB 8.1MB/s 00:14
sftp> mput sims*
Uploading sims_evolved_virtual_creatures_1994.mpeg to /data/disk2/temp/sims_evolved_virtual_creatures_1994.mpeg
sims_evolved_virtual_creatures_1994.mpeg 100% 234MB 8.1MB/s 00:29
sftp>
...soooo, that’s around 7-8-9 MB/sec which is nice if you think about it… (it gets encrypted, remember? ^_^ )
...and the top-output while copying these files to the fileserver
load averages: 0.31, 0.26, 0.17 22:53:38
35 processes: 2 running, 32 idle, 1 on processor
CPU states: 21.7% user, 0.0% nice, 37.7% system, 12.5% interrupt, 28.2% idle
Memory: Real: 20M/105M act/tot Free: 385M Swap: 0K/1024M used/tot
26320 markie 35 0 1484K 2520K run – 0:19 11.72% sftp-server
16598 markie 34 0 4964K 3776K run – 0:14 7.96% sshd
25849 markie 2 0 3152K 3324K sleep select 0:06 0.00% smbd
31996 root 2 0 1096K 2104K sleep select 0:01 0.00% sendmail
23991 root 2 0 1548K 2488K sleep select 0:01 0.00% httpd
17124 root 2 0 1076K 1412K sleep select 0:00 0.00% nmbd
32109 root 2 0 668K 1832K idle select 0:00 0.00% sshd
25757 root 2 0 668K 904K idle select 0:00 0.00% cron
21850 _syslogd 2 0 436K 820K sleep poll 0:00 0.00% syslogd
...well, as you can see… this machine isn’t running on it’s toes… not in a long way… sooo… I need to put some work in tweaking and tuning this machine… but I’m also filling this 100Mbit-line almost already… so an investment in a small gigabit-switch should be justified now… (I just needed some reason for that! ;-))
For this install, my old How-To was still pretty current, so rewriting it just for this seemed a little useless… Once I’ve got some fresh disks so I can combine it with a RAID-setup, I’ll rewrite the whole shebang… promise!
UPDATE
....and I did some read-write-tests on my fileserver, with SMB instead of sftp…
from my workstation (Gentoo Linux X86_64, Gnome)
reading from fileserver (screenshot): 7.4MB/sec
writing to fileserver (screenshot): 4.1MB/sec
from my mediacenter (Windows XP MCE)
reading from fileserver (screenshot): 8.7MB/sec
writing to fileserver screenshot: 8.0MB/sec
permalink - 
add to del.icio.us
previous: awesome OMGWTF-patches
next: website of the day, yay! (^o^)
How about some rough stats on smb throughput? I imagine sftp isn’t really the best judge because ssh does compression/encryption on top of the filesystem level stuff.
oh sure, I also tested it from my Media Center to the fileserver, only with a stopwatch and calculated the throughput, which is a little hard to blog, that’s why I put the sftp-output here ;-).
BTW, I get roughly the same results, 7-8-9 MB/sec…
UPDATE
I added some testresults... ;-)