[Home]

Bittorrent Observer

The objective of this project is to understand the failure rate of peers in a P2P network. I first finished some theoritical analysis of the problem, then decided to put it into practice to see how it works in real world P2P networks. The aim is about to estimate the failure rate by only using live information gathered by individial peers online.

Simulation

I built a discrete simulator to verify my theory, tested the statistical model against a well known Overnet trace data-set for accuracy and the results are pretty good:

Bittorrent Observer

A distributed system called Bittorrent Observer was then built to monitor the Bittorrent network and estimate the peer failure rate in this network. The system is built in a hybrid architecture:
  • it follows a client-server architecture, all nodes send collected data to a central storage server for processing.
  • at the same time, all nodes form a P2P network so they can cooperate to collect failure rate statistics without the coordination of any server.
The architecture of the Bittorrent Observer is shown below:
The server component is a BOINC server attached to a MySQL database back-end. It is only used to store collected data. The Work Generator on the server side tells the clients (i.e. peers) nothing more than "please keep working".

The peers (also BOINC clients) contains a modified version of the Rasterbar libtorrent. They actively ping other peers in the Bittorrent network to check their availability and then estimate the failure rate:

By-Product

I deployed the above system on 245 nodes, operated for over 12 months. Some of the by-products of this project is really interesting.

Users from different countries have differnt habbit in terms of how long they will be online:

I also had a look at what are being searched (via Kademlia), the most popular western content type is music, while in Asia, more people are looking for adult movies.