dnstt performance

These are tests of dnstt download performance.

David Fifield <david@bamsoftware.com>

Download speed tests ()

I did some experiments of download performance of the DNS tunnel. tl;dr a DNS tunnel can go faster than you may think, but the choice of resolver matters a lot.

I tried downloading a 10 MB file through the tunnel, using a selection of resolvers and DNS transports. I cut off the download after 10 minutes. "none" is the special case of no intermediate recursive resolver (the tunnel client sends queries directly to the tunnel server). The server was located in Fremont, US and the client in Tokyo, JP. There was about 100 ms of latency between the two hosts. Download rates are the median of 5 trials. The dnstt tag was v0.20200430.0.

Cloudflare's DoH and DoT resolvers are both fast. Google's DoH resolvers is much faster than its DoT server (I noticed the DoT server terminating TCP connections every 200 KB or so). Comcast's DoH and DoT resolvers have about the same middling performance. Quad9's DoT resolver is notably slow; there's clearly something wrong there, whether it's the resolver or how the tunnel uses it. For comparison, the download rate of an untunnelled, direct TCP transfer was 4666.3 KB/s.

resolvertransportdownload rate
noneUDP187.1 KB/s
CloudflareDoT156.9 KB/s
CloudflareUDP156.4 KB/s
GoogleDoH135.1 KB/s
CloudflareDoH133.5 KB/s
ComcastDoT68.5 KB/s
ComcastDoH66.3 KB/s
Quad9UDP58.9 KB/s
GoogleUDP43.1 KB/s
PowerDNSDoH38.0 KB/s
GoogleDoT35.4 KB/s
Quad9DoH30.9 KB/s
Quad9DoT1.2 KB/s

I repeated the experiment with iodine, an existing DNS tunnel. iodine works over plaintext UDP only. dnstt is faster than iodine in every case, except for the Quad9 DoT resolver. It is possible to run iodine over a DoH proxy; I didn't try that myself but Sebastian Neef reports 4–12 KB/s when tunneling iodine through dnscrypt-proxy.

resolvertransportdownload rate
noneiodine14.6 KB/s
Googleiodine1.8 KB/s
Cloudflareiodine1.4 KB/s
Quad9iodine0.3 KB/s

This graph shows the 5 trials under each experimental condition and gives an idea of the variance. Steeper lines are better.

Line graph showing the number of bytes downloaded over time for dnstt/UDP, dnstt/DoH, dnstt/DoT, and iodine, over a selection of public resolvers.

Download speed tests ()

I ran another set of tests to evaluate the effects of tuning certain performance parameters. While working on Champa, I found that increasing the size of smux Config.MaxStreamBuffer and tweaking other parameters could improve download speeds. For background, see these links:

I sought to see if a similar improvement in performance was possible in dnstt. As it turns out, it's possible to greatly speed up a UDP transport directly to the dnstt server (i.e., using -udp and giving the address of the server itself, without a recursive resolver in the middle)—but this is a discouraged configuration. When using the recommended configuration of recursive resolver in the middle, I was able to achieve modest improvements in download speed in only a small number of configurations. I re-ran the download test using v0.20200430.0, the same version used in the previous round of testing, and compared it to v1.20210803.0, which has the performance tweaks.

As in the previous tests, I tried downloading a 10 MB file for up to 10 minutes. The server was located in Fremont, US and the client in Tokyo, JP. There was about 100 ms of latency between the two hosts. The following table summarizes the results. The rates shown are the median of 3 trials.

noneUDP186.0 KB/s332.5 KB/s+78.7%
GoogleDoH132.7 KB/s134.6 KB/s+1.4%
CloudflareDoT88.9 KB/s112.8 KB/s+26.9%
CloudflareDoH98.2 KB/s97.4 KB/s−0.7%
ComcastDoH75.2 KB/s72.7 KB/s−3.3%
GoogleUDP57.7 KB/s70.4 KB/s+22.0%
PowerDNSDoH35.6 KB/s34.9 KB/s−2.2%
Quad9DoH20.7 KB/s31.0 KB/s+49.4%*
Quad9UDP47.5 KB/s22.2 KB/s−53.3%*
GoogleDoT44.2 KB/s14.4 KB/s−67.5%*
Quad9DoT0.9 KB/s1.6 KB/s+86.2%
CloudflareUDP0.9 KB/s0.8 KB/s−4.6%

For a few more tables, see the commit messages of:

Some observations on the results:

The Quad9/UDP and Google/DoT configurations (marked with a * in the table above) look like they have decreased in speed by more than half. Taking a closer look at the amount of data downloaded over time, we can see some unusual features that help explain these changes. One of the three Google/DoT trials in the new version finished faster than in the old version (3.5 minutes versus 4 minutes), but the other two trials, the TLS connection was broken after a minute (which in itself is not unusual), but re-establishing the TLS connection failed because of a DNS error. Compare the files 2021-08-02/data/2-de15c5a5/google_dot_1.dnstt.client.log and 2021-08-02/data/2-de15c5a5/google_dot_2.dnstt.client.log with 2021-08-02/data/2-de15c5a5/google_dot_3.dnstt.client.log. In two of the three Quad9/UDP trials, there is an obvious stairstep pattern: the download quickly makes progress, then stalls, then makes progress, then stalls, and so on. I don't know what is going on here, except that it may be some kind of rate limiting at the server. A similar phenomenon is visible with Quad9/DoH in the old version. There, the cause of the stalls is visible in the logs: a server response status of "500 Internal Server Error" or "502 Bad Gateway" causes dnstt-client to self-throttle for 10 seconds. See the files data/0-3254c1c8/quad9_doh_1.dnstt.client.log and data/0-3254c1c8/quad9_doh_2.dnstt.client.log.

Line graph showing the number of bytes downloaded over time for dnstt/UDP, dnstt/DoH, and dnstt/DoT, over a selection of public resolvers, in both the v0.20200430.0 and v1.20210803.0 versions.

PTPerf (Umayya et al. IMC )

"PTPerf: On the Performance Evaluation of Tor Pluggable Transports"
Zeya Umayya, Dhruv Malik, Devashish Gosain, Piyush Kumar Sharma
ACM Internet Measurement Conference (IMC)
Local cached PDF

The authors tested dnstt with Tor, along with 11 other Tor circumvention transports. They reported notably poor performance and reliability with dnstt, with a failure to download files ranging in size from 5 MB to 100 MB in more than 80% of attempts (Section 4.6). They tested only one resolver, -doh https://doh.opendns.com/dns-query. They did not experiment with other resolvers, nor with DoT or UDP modes.