I've been doing a fair bit of disk work on my file server since we got burgled. I had about a gig of photos, and some other (less important) data on my laptop that was lost completely. The only reason it was lost completely is that I didn't have space on the file server. So I've been moving tens of gigs around partitions and disks. And I installed a removable USB drive caddy that I'm going to run tertiary (archive) backups on to. I was using tapes, but they were too small, and that means I have to swap the tape...which means I have to notice that the current one is full, which I generally don't. So I moved to portable HDD's ;)

Anyway, because I've been doing so much disk work, a lot of it with BackupPC, the load on my PC has been high. The load with BackupPC when doing a backup is always high, has been since I started using the program, so that was no surprise. The irritating factor, was that it was taking more than 12 hours to do an archive of my 80G of backed up files. Now it takes about 6 hours to do just the normal backups (incremental or whatever). There's only 24 hours in a day, and it only does 1 backup at a time. Plus, I want the computer to be able to respond to be able to transfer new data to it, and access my file server. When the load reaches about 5, that becomes very hard!

When I do a

hdparm -i /dev/hda

It shows the drive using UDMA 4.

But I think that just shows what the drive electronics is capable. Not what the system is using. Doing:

hdparm /dev/hda

Shows the current configuration of the system in relation to the drive. Which in my case was:

using_dma = 0 (off)
Now, I'd enabled the Via chip for IDE when I upgraded the kernel, because the options where:

Trying to enabled DMA like this:

hdparm -d1 /dev/hda

Yielded the result:

"Operation not permitted"

I.e. it didn't enable DMA like it should. I thought that was a kernel config error, and I was right. It generally means the IDE chip isn't correctly loaded.

My lspci says:

0000:00:1f.1 IDE interface: Intel Corporation 82801AA IDE (rev 02)

Now the options in the kernel I saw closest were:

Intel PII X Chip

Or

Via 82xxx

Is that PII X? I don't think so. But the PII X covers it for some reason. So I enabled the module, and modprobe'd it and it did indeed detect an ICH chip.

So I happily rebooted so the module would be inserted at boot up and DMA disabled. Not so. Once the actual IDE driver gets loaded, you can't re-load an IDE chip (because the generic IDE driver is dependent on the IDE chip driver). Annoying, but it makes sense. So I recompiled my kernel with the PII X in the kernel, re-ran lilo and rebooted. And man that reboot was (relatively) fast. I'm used to waiting 90-120 seconds on my Via C3. I thought it was just because I had so many services loading. Nup. Loads in 20 seconds now :D

And I get the appropriate 20 - 50MB/s sustained from my drives now :)

I like this post too, about using HDParm to tweak the settings even more. There isn't much too it, and it can be fun to do it and benchmark the drive to see the difference ;)

And yes I ripped the title line in this post from Jack Nicholson :)

I was investigating the transfer performance of a USB HDD in Linux and stumbled across a news thread talking about the performance difference in compression prior to a transfer, against transferring the (larger) uncompressed file.

Oh by the way, performance of USB HDD transfers is bad due to no DMA = CPU spikes during the transfer...the faster CPU you have, the better performance you'll get (I get about 5MB/s on a 1GHz Via Nehmiah before it pegs out).

It looks like some systems admins got involved in the thread though, so it became quite humorous :)


Post #176448
By pwhysall
1. What's the fastest compression program? I'm willing to bet that gzip isn't it. this [*] may warrant investigation - I'm going to have a play!
2. Is your gzip appropriately compiled for AMD64, or is it a generic i386 binary?
3. Your Quad Opteron would be much happier in my house. Ship it to me right away!


Post #176458
By broomberg
but it seems the increase in size will bite you more when moving things over a T1.

I'll let you know later. Gotta go setup a new array and more some data.

Note: I disagree. I think the quad opteron is much more happier in my safe, cool, raised floor, generator backup power, 24 x 7 attended, Gbit backbone, 31TB SAN attached, LTO2 backed up, halon workalike protected, 10mbit ethernet internet connected computer room than it could ever possibly be in your house.

And it would get lonely without all the other quad opterons to keep it company.
Oh, you thought it was the only one? nonononono.
That was the 1st box to prove it out. We are standardizing on them for our high end high io compute needs. They will replace the Sun 450s.

How's your Easter been? Had good takings on the loot? Well, even if you didn't, someone else did...they took my loot! I got home last night after visiting my family for Easter, and spending some time with my Grand dad, to find I'd been burgled. Sigh.

So now I'm minus laptop and minus some other misc stuff. Looks like they were interrupted because they packed my LCD screen into my satchel, then left it. All the doors were locked, and they've left the back door open. So if they'd closed the back door, it would have been really weird!

I have to say, thanks for trying to take my laptop though. It has a perspex protector screen, and it leaves wonderful print marks :)
One thing that bugs me though - I brought some comp equipment home and I was dissembling it...I needed some pliers. No problem, my comp tools are pretty organized, so I go to my shelf in my room and....my multigrip is gone. They took my multigrip! That's just rude. I mean it's not worth anything. I should make a sign: "Thieves: BYO MULTIGRIP!" ;)

More irony - I've been meaning to clear more space on the server to backup all my photos that were on my laptop for about 3 months. And I never did it. So now I've lost all my 2008 photos. Sigh. I so can't be bothered swapping the HDD out. Actually I can't be bothered opening the case even ;)

I think it would be really nice and courteous, if the thieves dropped the HDD on the laptop on to DVD for me and sent it to me. I mean, the insurance will probably cover the laptop, and it's just stuff. Who really cares? But the data would be nice to have back.

So why am I posting now and not last night? Well I got woken up by a knock at the door, so I answered bleary eyed, blinked a couple of times, and there was a police chick in front of me. Interesting way to wake up I have to say ;)
Seriously though, she took the prints from the LCD so we'll see what happens. I find it interesting though the way it works with ID: If they can ID the print they call me. Otherwise they don't. It would make a lot more sense to call no matter what to at least let us know. I swear, Government cost cutting is way worse than private sector cost cutting. The private sector is just more up front about it so they come off looking bad... :)