IPCop and pfSense

August 7th, 2009 by james.pearce

I recently decdided to replace my Netgear FVX538 Modem / Router with a PC based firewall / router. The Netgear keeps crashing, dropping the ADSL link, and “forgtting” the WiFi password on reset. I’ve tried upgrading the firmware, and nothing seems to make it any better.

Anyway, the two PC distributions I decided to deploy for testing on real hardware were IPCop (1.4.20) and pfSsense (1.2.2). pfSense is incredibly slick and full featured. It really is an enterprise firewall OS in my view. The reporting options are great, and making changes / advanced configuration is sensible and easy (relative to the difficulty of performing the same changes on a normal *nix OS or a consumer firewall / modem device).

IPCop lacks a bunch of the features of pfSense, but none that affect me. The main difference is in the web interface. It’s just not as nice. It’s a little sloppier to use, and it looks a lot sloppier. It’s not terribly ugly, but it doesn’t look like something that would be produced by a commercial outfit, while pfSense does. The IPCop interface is relatively easy to edit though, it’s mainly just CGI files and a single CSS file.

There are a couple of other major function differences between the two distributions however. Firstly, pfSense is FreeBSD based (actually, based on m0n0wall), and IPCop is Linux based. I’m sure that’s the result of the practical differences that I found. What I’m talking about is the hardware support offered by the two operating systems. I had *a lot* of trouble getting pfSense to even boot from CD on Pentium hardware. Pentium 2 hardware was ok however. Even Pentium 1 MMX chips failed though. I’m sure it’s a minboard problem, as I’ve experienced the same issues with FreeBSD as well. Basically, some sort of incompatibility with the mainboard chipset causes the bootloader to fail to boot. Sometimes an immediate reset occurs, sometimes it crashes, and sometimes it fails with an error message. I tried more than half a dozen board / CPU combinations for Pentium grade hardware, and none worked with pfSense. Additionally, pfSense regused to acknowledge the presence of my Realtek chipset cards. I understand that they are cheap cards and will not offer the performance of a 3Com or Intel Pro card, but to not detect at all? That’s weird. FreeBSD has support for them, but it’s as if the kernel that pfSesnse compiled for their OS doesn’t include it. Weird.

I only tried IPCop on 2 Pentium grade systems, because that’s all I needed to try it on. I tried on a P120 and it failed to boot. I then tried it on a P90 and it worked fine. The system has a lot of RAM for a Penitum, 96MB, but it was nice to see it boot!

As to the performance, pfSense complains about having less than 128MB of RAM, and quite rightly so. It eats 64MB without doing anything. It also spikes my Pentium 2 350 CPU up to 80% load without network traffic. By spike, I mean when the OS is doing “housekeeping” activities.

On the Pentium 90 with IPCop, RAM usage didn’t exceed 32MB when not under network load, and the CPU didn’t exceed 20% usage without load. That’s a big difference; the numbers are smaller even on a ratio, so they are *much* smaller when taking into account the fact that it was running on a system that is 4x slower than the pfSense system.

What about stats for operating with load? I don’t know; I already decided to go with pfSense. Yes, it needs more hardware resources, and is more fussy, but it *is* better. I’m tired of compromising with routers. I’ve been through about 8 routers in the past 10 years, and I’m just tired. I’m 95% confident I can do everything I need to in pfSense right now, and have future support in it. With IPCop, I can probably do what I want in it now, but I’m not confident about future support, and I don’t like using the web interface.

PCI GFX FTW!

August 6th, 2009 by james.pearce

I recently redeployed my main desktop computer to be a virtualisation hub. I’m going to run a headless VirtualBox setup on Ubuntu with it. It’ll only be using CLI. At the moment the specs of the desktop are:

  • 3G DDR2
  • 2.66Ghz Core2Duo
  • Asus P5K
  • ATI Radeon 2900XT

Now, when idle, Core2Duo’s are quite power efficient. And I’m going to be merging 3 physical boxes into this one, but none of them do continuous work (hence why I’m merging them). For some reason, when sitting idle (I haven’t installed the real OS yet, it’s just a testing version of Debian Lenny on there with no services), the machine was still spewing out heat.
It turns out that even on a CLI, the Radeon card gets quite toasty. I mean, not hot by gaming standards, but hot compared to say, an entry level card. I happened to have an entry level card around: an ATI X1300. So I thought, I’ll quickly swap them out, ensure that it was indeed the GFX card, and go merrily on my way. No such luck. The X1300 was a dud. Sometimes the PC would get to the POST if I wiggled the card, only to complain that it didn’t have a card, and other times it just wouldn’t POST at all.
Now, this is a 2008 model system. I mean, it’s early 2008, but it was top of the line. But it has PCI cards. And I figured I had a TNT2 PCI version around somewhere, or something. The “or something” turned out to be a generic PCI card. I also had an S3 trident, but I wanted to see if the generic card worked. It looked older than the S3. That means it’s about 20 years old now. I didn’t even know if recent motherboards are able to detect and boot with a PCI graphics card only. I do now. They can. Mine did. Sweeeet! It makes _no heat_. There’s not even a heat sink on the card. It doesn’t display the BIOS graphics splash screen properly (the whole “powered by Intel” image), but it displays the actual text fine, BIOS is fine, VESA images is fine (i.e. the Linux boot splash screen). It’s so cool. It also takes up a fraction of the space that the 2900XT did.

Now my system runs cool and uses less power, I’m happy[ier] :)

Site Update: New Database

August 5th, 2009 by james.pearce

The site has been (semi) offline for a couple of days now. I took the database server that drives the site down because I’m virtualizing a lot of my infrastructure. Because it took longer than I thought it would to get everything back online, only cached pages were being served. The cache eventually times out and requires regeneration from the database, which it couldn’t do, and returned error 500 pages! Anyway, the new (virtualized) database server is up, so the site is back online :D

SCP With No Encryption: Why Not?

August 4th, 2009 by james.pearce

I’ve read lots of forum posts recently where someone asks how to turn encryption off for an SSH session; specifically for an SCP transfer. Every one I’ve seen has been flamed for asking this. One common response I see is:

“the encryption doesn’t take up enough of the CPU to warrant the kind of exposure on a modern CPU, you’re probably I/O limited anyway”

GARBAGE. On my Via C3 Nehmiah @ 900Mhz (it’s a 1.2Ghz chip in a 100Mhz FSB capable motherboard hence the slowed clock), my CPU taps out at 3.7MB/s on a 100Mb/s network. An it taps out on the SSH daemon, not the I/O time. Using NFS I can pull 9 to 10MB/s at CPU tap-out. Encrypting at wire speed DOES take up significant CPU time. Normal SSH terminal connections, sure, negligible. Bulk SCP connections, it’s real. Just take a look at the performance measurements taken on a Via C3 on this Linux / Via Padlock OpenSSH enabling tutorial.

Modern distributions of linux (i.e. kernel 2.6.27+ based), seem to have patched the OpenSSH (and hence the SSHD) to use the hardware encryption on the Via chip (Padlock), and I can pull 9-10MB/s at CPU tap-out on that with SCP. A P3 733Mhz also taps out at 3.5MB/s with the same Linux (Ubuntu) though, so it’s definitely the software being optimized for the Via chip.

At the end of the day though, on my local LAN (wired), I don’t really care about the encryption of the file transfer. What I care about is the ubiquity of the SSH protocol. I’m also the only one using it to access files, so I’m not using it to replace NFS, I’m just using it to access my private files (which are sometimes quite large) using the already-configured ACL (PAM). Why can’t I disable the encryption for the SSH data transfer in V2? Sigh.

Debian Crontab Fun

August 1st, 2009 by james.pearce

I have a couple of entries that I added into my system crontab file a while back that I noticed immediately working. I’ve been running the command manually for a while because I couldn’t be bothered invetigating. When I did sit down to investigate, I found that running:

crontab /etc/crontab

Makes crontab echo the output to STDOUT. Including errors. It’s kind of annoying how those errors don’t appear in my system message log normally, but at least I found the error! The error - what looked like wordwrap wasn’t, and so crontab was seeing a syntax error ;)

IBM Netfinity 3500 M20 Dual Core Hardware

July 27th, 2009 by james.pearce

When I got a hold of one of these beasties, it only had 1 processor installed. The other socket had a terminator in it. I thought, “Cool, a REAL dual processor machine, I’ll just buy a matching P3 733 off eBay and have some fun!”. Wrong. The board only comes with 1 Voltage Regulator Module - for the primary proc. It’s the VRM that supplies power to the proc. Attempting to start without the secondary VRM but with 2 procs: no POST. No hardware was blown, it just didn’t POST. I had a lot of trouble finding this out, because I couldn’t get the correct technical manual for the M20 (all I could find on Google was the M10), and nothing about needing a secondary VRM was listed in the tech manual that I had.


So, off to eBay again, and no luck on getting one of these. At least, not under the guide of a part for an IBM M20. I Googled a bit and found a post on a technical help website indicating a similar problem. He said he’d used a VRM from a Compaq, and that it had worked. He gave the part number (Compaq spares code 329267-001) and whacked it into eBay. Hey prestor, there is one in the UK. I’m in Australia, so that’s a little annoying, but oh well, it’s only $30. I could spend more at a restaraunt, does it really matter if it doesn’t work. So I forked out the dough, the VRM arrived a week (fast airmail yay!), and I gave it a go. Result? It worked! I thought it was such a longshot, I certainly was (pleasantly) surprised when it worked :)

So anyway, I played with the server for a couple of days as a dual processor machine, and then packed it back away into my archive of computers. I’m such a hardware junkie ;)

Embedded CCTV With Linux

July 20th, 2009 by james.pearce

Well, sort of. Apparently Zone Minder will do it. I downloaded the Live CD and tried it, and it wouldn’t boot properly (might be media problems). I couldn’t get it to finish booting. I found another Xubuntu based LiveCD but the system I was running it in only had 256MB of RAM and it didn’t like me installing SSH to get remove access. I’m not really sure why X11 is enabled on a LiveCD for a web application server. Oh well.

I finally discovered that Zoneminder is available in Debian sid (unstable). So I set about installing this. I got it all set up, it seems to work well (128M+ memory minimum, otherwise there are OOM issues). Unfortunately, it doesn’t seem to like my web cams. I have 2 USB web cams. One is a Creative WebCam NX Pro, and the other is a Logitech notebook web cam. Both are detected by the module auto-loader and have appropriate modules inserted. The Logitech camera completely fails in Zone Minder: The Zmu application fails to obtain the setting information from it. The Creative worked a little better, I was able to get all the settings except the Palette information. Unfortunately, neither V4Lv1 or V4Lv2 would give me a properl image. V4Lv1 gave me no image or just completely black (with no timestamp tag, so that means it was failing to obtain an image from the device). V4Lv2 gave me a “dirty” image; the image was half covered with moving horizontal lines and the rest of the screen was just black. Oh well. Guess I’ll give up on the Embedded CCTV with Linux project. It’s all fun though, I’m always looking for ways to use that old hardware! :)

A Little Bit Of Computing Hardware History

July 14th, 2009 by james.pearce

A friend of mine needed a little story write-up, with pictures, of computer hardware history in the last 20 years. I Google’d to find something like this, and couldn’t. Most hardware history articles are something along the lines of “Computing began in World War 2 with supercomputers and has progressed steadily to personal computers and now to laptops”. Not really the granularity that he was looking for. I decided to take the photos myself - I have nearly 20 years of equipment in my house. That’s kind of a side effect of discovering Linux in the mid 1990’s - it ran on that hardware then, and still does :) I’ve decided to post the article, with pictures, here, so that it might be useful to someone else on Google. Take what I’ve written with a grain of salt though, it wasn’t written to be a scientific journal article, only as a quick overview!

CPU’s

Top to bottom, left to right
Intel 486 Overdrive, Intel Pentium 1, IBM Cyrix
Pentium 2
Intel Celeron, AMD Duron, Intel Pentium 3
Pentium 4 (1st generation – 400Mhz FSB), Pentium 4 (3rd generation – 800Mhz FSB)

Note the increase in physical packaging size as technical CPU advances exceeded the fabrication technology advancements and economically feasible. The initial Pentium 2 was not able to be manufactured in a single chip that was small enough and had enough pins to draw power with, and it was uneconomical to construct it out of a single chip that it was feasible to do at the time. Within 24 months, fabrication technology had caught up, and CPU’s were moved back to a single chip form and made smaller. Current (2009 Q2) Core 2 Duo’s are smaller than Pentium 4 chips. The size of the chip is directly proportional to the transistor size and number of transistor used. The number of transistors has been growing in accordance with Moore’s law, and the fabrication technology has been shrinking accordingly. The smallest transistor size of a current (2009 Q2) CPU is 45nm (mass production debuted late 2007 by Intel). The shift to 32nm is road-mapped for 2009 by Intel.

The transistor size has a direct impact on the thermal output of the CPU which peaked with the Pentium 4 at approximately 150 watts (Pentium 4 3.4Ghz). Early x86 generation CPU’s did not require heatsinks (286, 386). The 486 and Pentium required passive cooling. Higher model Pentium’s required active air cooling, which has been the norm ever since. Thermal dissipation was one of the key inhibitors in maintaining the increasing performance of single core computers.

HDD’s

From left to right: 3.5”, 2.5”, Solid state
Not shown: 5.25”, 1.8”

HDD’s have been a very slow moving technology from the physical packaging outset. Manufacturers of consumer are notorious at maintaining upwards interface compatibility. Almost all 3.5” drives are IDE, and the original IDE drives manufactured circa 1990 will still work in the most recent IDE computers of 2006 (but not vice versa). The consumer market changed the physical interface to SATA circa 2003.

The SCSI technology used in server computing (not shown) uses the same packaging, but has undergone several physical interface changes similar to CPU socket changes.

The fast moving trend is to make platter size smaller. 5.25”, 3.5”, 2.5” - now common in server computers – and 1.8” (now common in laptops). The platter size shrinkage has been done without platter storage capacity shrinkage, resulting in an increase in aureal density of the platter and corresponding throughput increase as reading the same physical size off the disk results in more bytes being read than before.

The long moving trend is away from mechanical devices to solid state devices, as fabrication technology for high-density storage microchips has become economically and technologically feasible in sizes large enough to be useful to end users (64G solid state devices are currently available as options in mainstream computers – 2009 Q2).

Motherboards

Bottom: Intel 468; CPU embedded on mainboard
Top, left to right: Pentium, Pentium 2, Pentium 3

Note the change in peripheral interfaces: ISA only in the 486, ISA / PCI combination in the Pentium and Pentium 2, and PCI only in the Pentium 3. Pentium 4 introduced PCI express, and current (2009 Q2) dual processor boards are moving to PCIe only. Mainboard manufacturers are evidently good at maintaining clear upgrade paths, maintaining backward compatibility with the previous generation of peripheral connectivity so that the user does not have to upgrade the whole system.

The other two user-replaceable componentry that has changed over the course of mainboard history is the CPU socket and RAM socket. This has been necessitated by the change in physical CPU and RAM interface presentation. The RAM interface changes are generally driven by changes in technology, while the change in CPU circuitry is often pre-emptive by the manufacturer. This was evident in the Pentium 2, when socket and slot versions were available: the physical interface was different, but the logical interface was not. Converters were made for mainboards so that newer socket processors could be fitted to older slot mainboards.

As of the Pentium 2, the power interface also changed. 386, 486 and Pentium mainboards operated using AT connections. The Pentium 2 introduced the ATX power specification that changed the physical interface. The Pentium 4 introduced an additional 12V power connector to allow the processor to draw extra power: The Pentium 2 power draw was in the area of 25 watts; the Pentium 4 reached 150 watts. Dual processor mainboards used an updated ATX specification that combined this 12V connector into the main power connection, and this is the currently (2009 Q2) used power specification.

Each physical change in RAM and CPU that has required changes to a mainbaord have also impacted the bus layout. Chip integration technology has also resulted in only 2 main chips on mainboards, starting with the Pentium line up. Note how the 486 mainboard has several prominent chips on it, while the other mainboards only have 2 (plus the CPU). These two chips are designated the south and north bridge chips after the logical location proximity to the CPU. The Northbridge interfaces between the CPU and the Southbridge, as well as providing the memory management chip and AGP interface control (and thus including a GPU on motherboards that include it). The Southbridge is responsible for peripheral I/O (USB, Serial, Parallel (ports), DMA, PCI, RTC). These chips are also usually located in the same physical proximity order as the logical proximity by mainboard designers in order to minimise physical bus tracks. The Southbridge is often near the peripheral ports, while the Northbridge is between the CPU and RAM: On the Pentium 2 mainboard shown, the Northbridge is just under the heatsink of the CPU, the Southbridge is right next to the PCI ports.

RAM

Top to bottom: EDO RAM SIMM, SDRAM DIMM, DDR SDRAM DIMM, DDR SDRAM SO-DIMM (chips in chronological order)

Physical chip size of RAM has increased since the 486 era. The chip density has increased faster, however, and thus the larger physical size is representative of an increase in size of an order of several magnitudes. Typical sizes per memory module were:
72pin EDO SIMM: 1 to 4MB
SD DIMM: 16 to 128MB
DDR DIMM: 64 to 512M (1st generation), 256 to 1G (2nd generation), 512M to 8G (current - 2009 Q2 – generation)

Small form factor DDR RAM is logically the same as normal DDR RAM, but uses smaller physical interface. Note the chip size is the same on the normal DDR RAM as it is on the full size DDR module. Small form factor modules only include 2 chips per side of the module, and thus have a smaller capacity than their full factor counterparts: The current (2009 Q2) maximum small form factor DDR RAM capacity is 4G, compared to 8G in the full size form.

The interface technology of RAM has changed with each generation, but the general construction has not. Memory is still accessed by row / column blocks of bits. Initial Single Inline Memory Modules had a single set of electrical contacts and required installation in pairs in order to present a logical memory banks. Dual Inline Memory Modules had two sets of electrical contacts; one per module side. This is the physical interface.

The Extended Data Output interface on SIMM’s brought burst mode technology. Synchronous Dynamic technology was introduced in the mainstream in the mid 1990’s coinciding with the Pentium 2. This interface allowed interleaving between modules to enable faster throughput. Double Data Rate SDRAM simply involves changing the way the electrical signal is delivered / interpreted; it transfers data on the rising and falling edge of a clock cycle. Other than that, it is the same as the SDRAM that was developed in the mid 1990’s.

Decreasing transistor size and more efficient chip layout software has enabled RAM chips to benefit from decreased latency (access time). Typical access time on a SIMM was 70ns+, current (2009 Q2) latency is in the order of 5ns. Latency has been under 10ns since DDR RAM was introduced circa 2001 however, and this reveals that improvements in RAM technology are stagnating compared to processor improvements.

FreeBSD & Splunk: Terminated

July 11th, 2009 by james.pearce

Well I’m sad to say that I am terminating my efforts to run Splunk on FreeBSD (for now). Why? For some reason, my home installation of Splunk has stopped recording data delivered over the configured network ports. I presume something has become corrupt, because the connection is made successfully, it is just not recorded in Splunk (according to live tail). I did try to upgrade and discovered 2 thigs.

Firstly, the latest FreeBSD stable edition is 7.2 and Splunk are STILL only offering their application for 6.1. This is to do with the threading issue that has changed between 6 and 7, but 12 months after the OS release they still don’t offer an application for it, meaning we still have to hack the OS a little bit to get the application to work. That’s unsatisfactory vendor support IMO.

Secondly, their upgrade instructions don’t really work for FreeBSD. They suggest simply installing the package over the top of the existing package. FreeBSD doesn’t allow that unless you force it (pkg_install -F). When I tried forcing it, the install terminated silently. I mean, pkg_info doesn’t show that it is installed, starting Splunk reveals it is the old version…it just didn’t install. And there is no error saying why.

So, I can’t upgrade, and a complate reinstall means I have to hack the app to get it working agian. I just can’t be bothered. When I move to virtualization I’ll have a more powerful server and I’ll reinstall the Linux version of Splunk. At least they are supporting the stable kernel on that!

Jailbreak & Unlocking an Iphone 3G w/ 2.2.1 FW

July 10th, 2009 by james.pearce

How to do this EASILY via software using Mac OS X?

I spent a long time trying to figure this out. Ziphone doesn’t seem to do it (it doesn’t brick the iPhone, it just forces a restore). Some of the others are too hard (SSH’ing to the iPhone, come’on really…script it or something!). Sure, I understand the reason why 2.2 works and 2.2.1 doesn’t for the unlock (the baseband upgrade), and honestly, I don’t care. I just want to be able to click a button and follow some on screen instructions ;)

My final (working) solution? Don’t. Upgrade to 3.0 and do it with that.

Basically, get a copy of redsn0w to jailbreak the 3G 3.0 firmware and install Cydia, then add the Ultrasn0w repository and install ultrasn0w. Complete! I also did this on an un-activated iPhone by doing the install via Wifi instead of 3G.

Complete instructions for obtaining and installing redsn0w are available at iClarified.

Complete instructions for the jailbreak + unlock technique are available at Quickpwn (obviously, using the instructions above for the jailbreak…it really doesn’t matter how you jailbreak your phone, only that it /is/ jailbroken to enable the Cydia install).