|

A Torifying Tale: Our experiences building and running Tor servers

Gustaf

The following post is adapted from a presentation at Chaos Communication Camp 2015. You can see the presentation in its entirety in the video below.

Access Now is first and foremost a technology group. In 2009, following the Iranian post-election crackdown, a group of technologists began a series of interventions to keep Facebook, YouTube, Twitter, independent blogs, and email clients accessible and unmonitored. As millions challenged the election results, this network of technologists provided critical support that helped this burgeoning democracy movement participate in global dialogue.

Part of this network of technologists eventually coalesced into Access Now. We are dedicated to defending and extending the rights of users at risk around the world. Many of you know about our 24-7, 365 helpline for civil society. We also provide capacity for the Tor network.

Tor is a network to relay and attempt to anonymize internet traffic. Internet traffic enters the network through known entrances called “bridges”; it is rerouted through various relays and exits on a known exit point. If one group controls both bridges and exit nodes, it would be possible to de-anonymize that traffic. Access Now provides support for 20 Tor exit nodes, but does not also run bridges.

Here is what we have learned running Tor exit nodes.

1.) Hosting Issues: Location

If you look at a map of where Tor users are, you’ll notice there are a lot in the United States and in Europe. We believe that Tor is good for those users and it appears to work well, in part because the vast majority of the Tor infrastructure is located in the United States and Europe. However, Tor seems to be too slow in places like Namibia. There are a tiny number of Tor users in Africa, but there is certainly a lot of need. It may be that users are unfamiliar with Tor, but it seems more likely that we simply need more infrastructure to speed up the network to make it more usable. We need more Tor servers in Africa and Asia where users at risk may be located.

2. Hosting Issues: Commercial Operators

 

A commercial provider is happy to sell you a contract to host a Tor server. They’ll always say yes to taking your money. However, they don’t understand what that entails and then problems can arise. There are consequences for running a Tor exit node. It is very important to explain to the service provider what you are doing. They must understand or you can spend a lot of time getting your servers running only to have the access turned off. Here are a few of the problems we’ve encountered.

Abuse Claims

We know that the Tor network is fantastic for protecting the identity of users at risk. It is also used by people for nefarious purposes. When you run exit nodes, people will, unfortunately, be attacked  — and if they contact the hosting provider directly, it can become an issue.

Sustained Bandwidth:

 When most people purchase access from a provider they purchase bandwidth.  For the most part, this is easy for the providers. There might be peaks, but most organizations don’t use their entire bandwidth all the time. However, if you run a big Tor exit node, the traffic is solid and sucks up a lot of bandwidth. If the hosting provider is not prepared, it can cause an issue.

Retribution Attacks:

 If someone is attacked through your Tor exit node, they can just fire up a DDoS attack instead of going through the normal process of notification. This can result in exceeding bandwidth and numerous other problems.

Law Enforcement:

 Occasionally a provider has shut down our access and won’t explain why. This may be due to pressure from law enforcement. This could be an even greater issue in countries where Tor is less common.

3. Server Balance

We started by using one server to host one node. However, we’ve learned that it works better if we run a server with a large number of virtual exits nodes on top of that. I can’t tell you what hardware you will need because every environment is different. You will need to take a systematic approach to determine what works in your environment.

You will also need to optimize over time. As you optimize one issue — such as latency — you might discover another problem such as memory or disk space. It requires constant attention. Hosting virtual servers allows you more flexibility to juggle those resources.

4. Document EVERYTHING

 

We had a massive array failure and we lost a large number of nodes. We scrambled to rebuild it. I turned to a colleague and asked for his notes to reconfigure only to discover that no one had taken notes. That meant a year of fine tuning needed to be learned again. Be prepared for catastrophic failure so that you are able to recover quickly.

5. Handling Abuse Claims

 

When we started this in 2009 — and through 2012 — we were frequently visited by men in dark suits and dark glasses. This seems to be changing. Law enforcement seems to understand Tor, but occasionally you may have an issue.

It is really important how you handle the victims when abuse claims do arise. You need to explain what Tor is and you do need to acknowledge that Tor can be the harbinger of malicious attacks. You need to spend the time going through what can be done to prevent further attacks. There are strategies to prevent traffic from Tor and you need to help them.

There is also a human side of this. Humans have been the victim of abuse and they are very upset. You need to treat them with the utmost respect. You need to take the time to listen to them, to hear their problems, and to work with them to help them. You also need to explain why you are hosting a Tor network and what it means for someone in Syria who might be in a life or death situation.

Finally, expect this. Working with a victim is not something that only happens once a year. Have a system in place to handle issues as they arise. Access Now manages a global 24-7, 365 helpline which allows us to respond quickly. Other organizations would need to develop a system that makes sense for them.