Httptap - Monitor All HTTP and HTTPS Connections of Any Linux Application

As an alert Linux sysadmin, you may want to monitor web traffic for specific services. Here's why?

• Telemetry detection: Some tools with sensitive user data go online when they shouldn't. Good examples are offline wallet or note-taking applications.
• Application debugging when something goes wrong.
• High traffic usage: 4G or 5G connections are usually limited, so it's better for the wallet to stay within the limits.

The situation becomes complicated on servers due to the popularity of containers, mostly Docker or LXC.

How to identify an application's traffic within this waterfall?

Httpat from Monastic Academy is a great solution for this purpose. It works without root access, you only need write access to /dev/net/tun to be able to work with TUN virtual device used for traffic interception.

Installing Httpat

The application is written in Go and the binary can be easily downloaded from the Github release page using these three commands one by one:

wget -c https://github.com/monasticacademy/httptap/releases/latest/download/httptap_linux_$(uname -m).tar.gz
tar xf httptap_linux_$(uname -m).tar.gz
sudo mv httptap /usr/bin && rm -rf httptap_linux_$(uname -m).tar.gz

Install with Go:

go install github.com/monasticacademy/httptap@latest

Another way is to check your Linux distribution repository for the httptap package. The Repology project is great to see which distributions currently have a Httptap package.

On Ubuntu 24.04 and later, the next AppArmor restrictions should be disabled:

sudo sysctl -w kernel.apparmor_restrict_unprivileged_unconfined=0
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0

Practical examples for common use-cases

For a quick start, let's load the website "linuxhandbook.com" using curl:

httptap -- curl -s -o /dev/null https://linuxhandbook.com

Looks great, it tells us that curl used 141714 bytes for a GET request with code 200, which is OK. We use -s -o /dev/null to prevent any output from the curl to see what Httptap does.

---> GET https://linuxhandbook.com/
<--- 200 https://linuxhandbook.com/ (141714 bytes)

Let's try google.com website, which use redirects:

httptap -- python -c "import requests; requests.get('https://google.com')"

---> GET https://google.com/
<--- 301 https://google.com/ (220 bytes)
decoding gzip content
---> GET https://www.google.com/
<--- 200 https://www.google.com/ (20721 bytes)

It works and notifies us about 301 redirects and archived content. Not bad at all.
Let's say we have a few instances in the Google Cloud, managed by the cli tool called gcloud. What HTTP endpoints does this command use? Let's take a look:

httptap -- gcloud compute instances list
---> POST https://oauth2.googleapis.com/token
<--- 200 https://oauth2.googleapis.com/token (997 bytes)
---> GET https://compute.googleapis.com/compute/v1/projects/maple-public-website/aggregated/instances?alt=json&includeAllScopes=True&maxResults=500&returnPartialSuccess=True
<--- 200 https://compute.googleapis.com/compute/v1/projects/maple-public-website/aggregated/instances?alt=json&includeAllScopes=True&maxResults=500&returnPartialSuccess=True (19921 bytes)

The answer is compute.googleapis.com.

OK, we have Dropbox storage and the rclone tool to manage it from the command line. What API endpoint uses Dropbox?

$ httptap -- rclone lsf dropbox:
decoding gzip content
---> POST https://api.dropboxapi.com/2/files/list_folder
<--- 200 https://api.dropboxapi.com/2/files/list_folder (2119 bytes)

The answer is loud and clear again: api.dropboxapi.com.
Let's play a bit with DoH - encrypted DNS, DNS-over-HTTPS. We will use Quad9, a famous DNS service which supports DoH via https://dns.quad9.net/dns-query endpoint.

$ httptap -- curl -sL --doh-url https://dns.quad9.net/dns-query https://linuxhandbook.com -o /dev/null
---> POST https://dns.quad9.net/dns-query
<--- 200 https://dns.quad9.net/dns-query (83 bytes)
---> POST https://dns.quad9.net/dns-query
<--- 200 https://dns.quad9.net/dns-query (119 bytes)
---> GET https://linuxhandbook.com/
<--- 200 https://linuxhandbook.com/ (141727 bytes)

Now we can see that it makes two POST requests to the Quad9 DoH endpoint, and one GET request to the target - linuxhandbook.com/ to check if it works correctly, all with success.
Let's take a look under the hood - print the payloads of the DNS-over-HTTPS requests with --head and --body flags:

./httptap --head --body -- curl -sL --doh-url https://dns.quad9.net/dns-query https://linuxhandbook.com -o /dev/null---> POST https://dns.quad9.net/dns-query
> Accept: */*
> Content-Type: application/dns-message
> Content-Length: 35
linuxhandbookcom
<--- 200 https://dns.quad9.net/dns-query (83 bytes)
< Content-Type: application/dns-message
< Cache-Control: max-age=300
< Content-Length: 83
< Server: h2o/dnsdist
< Date: Sun, 09 Feb 2025 15:43:37 GMT
linuxhandbookcom
                ,he
                   ,he
                      ,CI�
                          ---> POST https://dns.quad9.net/dns-query
> Accept: */*
> Content-Type: application/dns-message
> Content-Length: 35
linuxhandbookcom
<--- 200 https://dns.quad9.net/dns-query (119 bytes)
< Server: h2o/dnsdist
< Date: Sun, 09 Feb 2025 15:43:38 GMT
< Content-Type: application/dns-message
< Cache-Control: max-age=300
< Content-Length: 119
linuxhandbookcom
                ,&G CI�
                       ,&G he
                             ,&G he
---> GET https://linuxhandbook.com/
> User-Agent: curl/8.11.1
> Accept: */*
<--- 200 https://linuxhandbook.com/ (141742 bytes)
< Cache-Control: private, max-age=0, must-revalidate, no-cache, no-store
< Pagespeed: off
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< X-Origin-Cache-Control: public, max-age=0
< Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=bAHIntCPfaGgoUwEwhk5QWPETFvnq5K9Iw60TGIAcnTisEfo%2BjKulz%2FJP7rTPgmyznVSc%2BSwIOKtajz%2BZTg71To4BuapDd%2BKdgyar%2FpIGT76XWH9%2FVNMyliYqgceD7DwuBmiPr3F77zxa7b6ty8J"}],"group":"cf-nel","max_age":604800}
< Server: cloudflare
< Cf-Ray: 90f4fa286f9970bc-WAW
< X-Middleton-Response: 200
< X-Powered-By: Express
< Cf-Cache-Status: DYNAMIC
< Alt-Svc: h3=":443"; ma=86400
< Date: Sun, 09 Feb 2025 15:43:48 GMT
< Display: orig_site_sol
< Expires: Sat, 08 Feb 2025 15:43:48 GMT
< Response: 200
< Set-Cookie: ezoictest=stable; Path=/; Domain=linuxhandbook.com; Expires=Sun, 09 Feb 2025 16:13:48 GMT; HttpOnly
< Strict-Transport-Security: max-age=63072000; includeSubDomains; preload
< X-Middleton-Display: orig_site_sol
< Server-Timing: cfL4;desc="?proto=TCP&rtt=0&min_rtt=0&rtt_var=0&sent=0&recv=0&lost=0&retrans=0&sent_bytes=0&recv_bytes=0&delivery_rate=0&cwnd=0&unsent_bytes=0&cid=0a7f5fbffa6452d4&ts=351&x=0"
< Content-Type: text/html; charset=utf-8
< Vary: Accept-Encoding,User-Agent
< X-Ezoic-Cdn: Miss
< X-Sol: orig
< Nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
<!DOCTYPE html><html lang="en" class="group/html min-h-screen has-inline-code-block





         has-gray-scale-Slate
    " data-prismjs-copy="Copy" data-prismjs-copy-error="Error" data-prismjs-copy-success="Copied"><head><meta charset="UTF-8"/>
...

Fantastic! Httptap just intercepted the HTTP headers thanks to the --head option and the payloads because the --body option was used.

HAR

To work more comfortably with HTTP requests and responses, Httptap supports HAR format:

httptap --dump-har out.har -- curl -Lso /dev/null https://linuxhandbook.com

There are many HAR viewer applications, let's open it in Google HAR Analyzer:

More useful Httptap options:

• --no-new-user-namespace - run as root without user namespace.
• --subnet and --gateway - subnet and gateway of network inteface visible for subprocess.
• --dump-tcp - dump all TCP packets
  --http HTTP - list of TCP ports to intercept HTTPS traffic on (default: 80)
  --https HTTPS - list of TCP ports to intercept HTTP traffic on (default: 443)

Httptap runs the process in an isolated network namespace and also mounts an overlay filesystem for /etc/resolv.conf to make sure the correct DNS is used. The Linux namespace is a list of network interfaces and routing rules, and httptap uses it to not affect network traffic on the system.

It also injects a Certificate Authority to be able to decrypt HTTPS traffic. Httptap creates a TUN device and runs the subprocess in an environment where all network traffic is routed through this device, just like OpenVPN.

Httptap parses the IP packets, including inner TCP and UDP packets, and writes back raw IP packets using a software implementation of the TCP/IP protocol.

Advanced - modifying requests and responses

Currently there are no interface or command line options to do this, but it's possible with simple source code modification. Basic Go programming skills are required, of course.

The code that handles HTTP requests is here, and the code that handles responses is a few lines below that. So it's very easy to modify outgoing traffic in the same way as a normal GO HTTP request modification. Real expamples: modify or randomize application telemetry by inserting random data to make it less readable.

Conclusion

There are a few related tools that I find interesting and would like to share with you:

• Wireshark - if you want to know what's going on your network interfaces, the real must-have tool.
• OpenSnitch - interactive application firewall inspired by Little Snitch for macOS.
• Douane - personal firewall that protects a user's privacy by allowing a user to control which applications can connect to the internet from their GNU/Linux computer.
• Adnauseam - "clicking ads, so you don't have to".

I hope you enjoy using Httptap as much as I do 😄