June 24, 2009 by Hangman
Our good friend Hangman over at Valadiléne.org is writing a series of tutorials beginning with Part 1 on How to Create a Torrent Tracker with PHP and XBTT, and wanted to share his knowledge with FSF readers. If there’s enough demand, the series will be expanded to include future coding goodies including ratio system, upload form with torrent validation, and search engine. We recommend all aspiring tracker developers to stay tuned into his blog; surely there’s more interesting tidbits to come. Hangman is also the brain behind the new tracker vgmTorrents.net, which is actually based on WordPress as a frontend…
After some attempt, I found a few projects, but too bad they seemed all but well written and easily customizable. Looking at the code, I had the feeling they all shared the same codebase; they all looked really messy and with very little error handling. Definitely not what I expected, at least. I tried and tried again, but without satisfying results.
So I decided to do it by myself.
I didn’t have too much time to spend on it though, so I chose WordPress as a framework and created some custom pages for torrent details, browse section, upload form and so on. So I coded data presentation (the Browse section with search engine, paging etc.), torrent management (the upload form with .torrent files validation etc.), and the “get file” part, (i.e. the code that fetches passkeys in torrent files when downloads are requested). Finally some additional user management code with stats, shoutbox for torrent pages, and a ratio system that simply ensures people don’t start downloading a GB of stuff and end up seeding not even a kilobyte back.
In this series of articles we won’t use WordPress or any other framework/CMS, but instead we’ll learn the basic concepts to build the critical parts of a torrent site, explaining it in a way that will be abstract enough to be applied to any system or framework. It is assumed that you have good knowledge of both PHP and the system you want to build your tracker on (best if it was written by you, so you exactly know how things work). Also, knowledge of how BitTorrent works and the BitTorrent protocol are recommended. Just print those pages and have a quick read, hopefully it will be enough.
On the server side, we will use the XBT Tracker, a fast and lightweight C++/MySQL based BitTorrent tracker. This is the piece of software that handles communication between peers. XBTT needs a MySQL database to store data such as peer activity, amount of downloaded and uploaded data and so on.
So here we sum up our requirements:
- • VPS or dedicated server
- • Web server such as Apache or Lighttpd with working PHP
- • MySQL database
- • Knowledge of PHP and rough understanding of how BitTorrent works
Optional requirements:
- • A working PHP framework with user authentication (for private trackers)
Installing XBTT
Assuming that you have a working SSH connection to your machine, let’s install some required packages.
If you run Debian or derivatives (such as Ubuntu), type this:
If you are on CentOS, Fedora Core or Red Hat, type this instead:
Now it’s time to download and compile the source.
svn co https://xbtt.svn.sourceforge.net/svnroot/xbtt/trunk/xbt/misc xbt/misc
svn co https://xbtt.svn.sourceforge.net/svnroot/xbtt/trunk/xbt/Tracker xbt/Tracker
This will download the code from the XBTT SVN repository on your machine. Now let’s compile it.
./make.sh
Unless anything wrong happened, we have just compiled XBTT and are ready to configure and run it. Let’s make a copy of the empty configuration file that shipped with the package, and fill it out with a text editor. XBTT only needs to know our MySQL host, database name, user and pass.
nano xbt_tracker.conf
Now save the file (on nano it’s CTRL+X, then y when prompted). Before running the tracker, we need to fill the database we’ve chosen with the default XBTT tables. Just import xbt_tracker.sql in your MySQL:
…and we’re done with it.
Type this to run the server:
Public or private tracker?
XBTT should now be listening on the default port 2710 (don’t forget to leave it open if you use a firewall such as iptables). Our announce URL is:
http://example:2710/announce
By default, the server will track any torrent and add it to the xbt_files table if not present (based on hash checking). Also, anonymous announcing and scrape are enabled in our vanilla install.
It’s up to us to choose whether we want our tracker to be public or private. If it’s going to be public, it’s just a matter of letting our users upload .torrent files and telling them to use our announce URL in the torrent creation step. On the other side, if we want it to be private, we will need some more work.
First of all, we will need an authentication system for our website, which I assume you already have. We need it because we don’t want everybody out there being able to share their stuff through our tracker, so we’ll need some sort of way to let users register, log in and upload/access the files.
Assuming we got this, we need to modify it so that when an user registers, it will create a new record in the XBTT xbt_users table, preferably with the same index (so that it will simple to fetch data in our scripts using a simple JOIN costruct).
This, translated into practice, means adding a SQL query in our user registration script, after form validation and insert query. I assume you have access to the last inserted ID relative to your new user:
Now that we’ve handled this, we need to change some values in the xbt_config table. The settings we’re interested in are auto_register, anonymous_announce, anonymous_connect and anonymous_scrape. Let’s set all of them to 0 (or insert them if they’re not there). Also, we need to set (or insert) the torrent_pass_private_key field, which we’ll use when generating passkeys for our torrents (we’ll see how later). We can generate it with the SQL MD5() function.
Passkeys in .torrent files
When an user adds to his BitTorrent client a torrent file containing our announce URL, the client tries to connect to that address. If the server response is okay, the client starts getting some data from the server such as seeds and peers information, and eventually starts downloading and seeding. When the tracker has to decide whether the response should be okay or not, it considers the following things: is anonymous_announce set to on? If yes, add the client to xbt_users, otherwise check if it exists in xbt_users and choose whether to send an error message back or let it connect. Then it checks for auto_register: if it’s set to 1, add the torrent file to xbt_files if not present yet, otherwise if it’s set to 0, do the same kind of check as for users, this time looking for the torrent hash in xbt_files.
How does XBTT check if an user is present in xbt_users and can have access to the tracker? It simply looks for a passkey in the announce URL. The format of an announce URL that includes a passkey is the following:
http://example:2710/passkey/announce
Now there’s one problem: the torrent files created and uploaded by users will obviously only contain the “clean” announce URL, so basically, when an user wants to download a torrent file, we need to generate a passkey based on his user ID, the torrent hash and the tracker passkey we set earlier, then replace the announce URL in the torrent file with it.
Fortunately, this can be done in a simple script. We will need a bencode library for this task. Bencode (pron. “bee-encode”) is the encoding used in .torrent files for structuring metadata. There are plenty of libraries to handle it and they are very easy to use. Assuming you have included one in your script and you’ve played with it to understand how its API works, here’s some generic code to generate the passkey and insert it in the file (no error checking for the sake of this example).
//
// we assume we have the user id in $uid
// and the torrent hash in $info_hash
//
$data=bdecode(‘example.torrent’);
$user=mysql_fetch_row(mysql_query("SELECT torrent_pass_version FROM xbt_users WHERE uid={$uid}"));
$torrent_pass_version=$user[0];
$config=mysql_fetch_row(mysql_query("SELECT value FROM xbt_config WHERE name=’torrent_pass_private_key’"));
$torrent_pass_private_key=$config[0];
// torrent_pass_version is a simple integer that should be incremented everytime you want to reset the user passkey
$passkey=sprintf(‘%08x%s’, $uid, substr(sha1(sprintf(‘%s %d %d %s’, $torrent_pass_private_key, $torrent_pass_version, $uid, pack(‘H*’, $info_hash))), 0, 24));
// build our new announce URL containing the generated passkey
$announce_url=parse_url("http://example:2710/announce");
$announce_url=sprintf(‘http://%s:%d/%s/announce’, $announce_url[‘host’], $announce_url[‘port’], $passkey);
// info key in the decoded dictionary (see wiki.theory.org/BitTorrentSpecification)
$decoded=$data[‘info’];
// build the new torrent file
$new_data=sprintf(‘d8:announce%d:%s4:info%se’, strlen($announce_url), $announce_url, bencode($decoded));
$new_size=strlen($new_data);
?>
With the $new_data and $new_size variables, we are now able to output our torrent file containing a valid passkey for our user. We only need to echo it as a torrent file (application/x-bittorrent) and raise a download dialog using the PHP header() function.
Conclusion
So, we’re done with the first part of this tutorial! We’ve seen how to compile and run the XBTT tracker on our machine and looked at some code to build the barebones of a basic private tracker.
Of course, to make it a fully functional website, we still need stuff like an upload form which checks for the MIME type of the uploaded file (and optionally the private flag of the torrent), plus some code to display the files contained in the torrent (filenames and sizes) and scrape data (active seeds and peers). Also, if we’re building a private tracker we’ll need some sort of ratio system which disables leeching for users with ugly ratios. Moreover, if our torrent database gets large, a search engine becomes necessary, otherwise our users will get lost soon.
All of this will be covered in the next parts of this guide if there’s enough demand. I hope this article helped you in getting a simple and basic system to work on.
Source from:
http://valadilene.org/2009/06/24/how-to-create-a-torrent-tracker-with-php-and-xbtt-part-1.
Archived by FSFmirror
