TorrentFlux (actually TorrentFlux B4rt is what I’m using) is a PHP / AJAX Bitorrent front end that uses the Java Azereus BT client to actually connect into the network. I use it because this way I can run BT on one of my little home servers that is on 24/7 rather than having to leave my huge desktop computer on 24/7 or using my laptop for BT.

One problem I’ve always had with TorrentFlux / Azereus is the amount of memory it consumes. I have a feeling that is a side effect of using a Java BT client: There is probably a lot of connection table caching going on. Don’t have a solution to that one (yet). Another problem I’ve had with it is that it’s /slow/ compared to normal desktop BT clients. Sure, I expect it to be a little slower but I mean it’s slllllloooooowww. When adding a new file it takes 2 – 3 minutes to download it and another 90 seconds to sart it (it says “processing” for that time). My original solution to this was to improve the performance of the database: I shifted from using SQLite to using MySQL. I found that bogged down the server I run it on too much, so when I rolled out a dedicated MySQL box, I moved the database over to that. Well, now it didn’t bog down the server with the BT client; it bogged down the MySQL server. The weird thing is, with the performance monitoring data I had, I couldn’t figure out where the slow down was coming from. The CPU wasn’t tapping out, nor the memory. What I found was that hte I/O was thrashing.

I ran MyTop for a little while to examine the queries on the TorrentFlux database and discovered that the database gets hammered with status updates about the files that are in the queue. The specific queries are always of the form:

SELECT xxxx FROM tf_log WHERE file=’xxxx’

So obviousl I checked the details of the tf_log table. It’s a huge table holding hundreds of thousands of records an almost a hundred meg. I checked the indexes and suprisingly found that file was not one of the indexes. No wonder it’s so slow and the I/O is thrashing; every time a WHERE query is performed it’s having to do table scans! What I did was create an INDEX for the file column in the tf_log and now my TorrentFlux is about 5x faster when adding / removing / pausing / resuming downloads. The exact query you would need to issue to MySQL would be:

CREATE INDEX ON tf_log (file(20));

I’m limiting the index to 20 characters of the file name because I don’t generally have many files with similar names in my list. This way the index is kept relatively small compared to the full filename size but still having a high probablity of being unique i.e. more efficient.