Using tshark to Decrypt SSL/TLS Packets

I’m going to walk you through the process of decoding SSL/TLS traffic from a pcap file with the server’s private key using tshark (command-line version of Wireshark). You can, of course, always use ssldump for the same purpose.

I assume you know how SSL/TLS works, and basic understanding of how Wireshark works, and why we use it.

I will start with getting a sample encrypted traffic that includes the handshake part (important for decryption later). For that purpose, we are going to use openssl command to generate a pair of server certificate and key. And then run the HTTPS server with openssl’s s_server command on port 4443 (or any other port you may like) using the generated certificate and key. Then we will issue a GET request to HTTPS server via curl. In the mean time, we will collect the traffic with tshark and will save the data into ssltest.pcap file.

# [1] create RSA cert and key pair
openssl req -new -x509 -out server.crt -nodes -keyout server.pem -subj /CN=localhost

# [2] run the server using the above
openssl s_server -www -cipher AES256-SHA -key server.pem -cert server.crt -accept 4443

# [3] from another console session, start capturing the traffic, on loopback interface
# (you will need to change lo0 to the relevant interface on your system.
tshark -s0 -w ssltest.pcap -i lo0

# [4] generate traffic from another console
curl -vk https://localhost:4443

# [5] Ctrl+C on the tshark command at [3], and stop the openssl server at [2]

At this point, we should have the file called ssltest.pcap from tshark, and server.crt/server.pem from openssl commands.

Next, we are going to read the pcap file and decode the traffic.

# [1] it shows the encrypted traffic
tshark -r ssltest.pcap

# [2] for details of the packets
tshark -r ssltest.pcap -V

# [3] for decrypted data; ssl.keys_list points to the RSA key
# added -x for hex dump
# At the output you should see the message in packet detail:
#  >>> Decrypted SSL record (16 bytes):
# And the decrypted data:
# >>> Hypertext Transfer Protocol
# >>>    GET / HTTP/1.1\r\n
tshark -r ssltest.pcap -V -x -o "ssl.debug_file:ssldebug.log" -o "ssl.desegment_ssl_records: TRUE" -o "ssl.desegment_ssl_application_data: TRUE" -o "ssl.keys_list:127.0.0.1,4443,http,server.pem"

# [4] inspecting ssldebug.log output from [3]
# You should see the following messeage near the top of the file:
#   >>> ssl_init private key file server.pem successfully loaded.
cat ssldebug.log

In Wireshark GUI, we can follow “SSL stream” that will dump the ASCII output from the stream. How are we going to do it with tshark?

# We add -z to show the statistics with option 'follow,ssl,ascii,1'
# to follow ssl stream number 1
# -q to suppress packet dumps
tshark -r sslsample.pcap -q -o "ssl.keys_list:127.0.0.1,4443,http,server.pem" -z "follow,ssl,ascii,1"

You will see the output similar to below:

===================================================================
Follow: ssl,ascii
Filter: tcp.stream eq 1
Node 0: 127.0.0.1:55041
Node 1: 127.0.0.1:4443
78
GET / HTTP/1.1
Host: localhost:4443
User-Agent: curl/7.43.0
Accept: */*


	1802
HTTP/1.0 200 ok
Content-type: text/html


<_pre>

s_server -www -cipher AES256-SHA -key server.pem -cert server.crt -accept 4443
Ciphers supported in s_server binary
TLSv1/SSLv3:AES256-SHA
---
Ciphers common between both SSL end points:
ECDHE-ECDSA-AES256-SHA     ECDHE-ECDSA-AES128-SHA     ECDHE-ECDSA-DES-CBC3-SHA
ECDHE-RSA-AES256-SHA       ECDHE-RSA-AES128-SHA       ECDHE-RSA-DES-CBC3-SHA
ECDH-ECDSA-AES256-SHA      ECDH-ECDSA-AES128-SHA      ECDH-ECDSA-DES-CBC3-SHA
ECDH-RSA-AES256-SHA        ECDH-RSA-AES128-SHA        ECDH-RSA-DES-CBC3-SHA
DHE-RSA-AES256-SHA         DHE-RSA-AES128-SHA         EDH-RSA-DES-CBC3-SHA
AES256-SHA                 AES128-SHA                 DES-CBC3-SHA
ECDHE-ECDSA-RC4-SHA        ECDHE-RSA-RC4-SHA          ECDH-ECDSA-RC4-SHA
ECDH-RSA-RC4-SHA           RC4-SHA                    RC4-MD5
---
New, TLSv1/SSLv3, Cipher is AES256-SHA
SSL-Session:
    Protocol  : TLSv1
    Cipher    : AES256-SHA
    Session-ID: B9AE3B24559606A2723F987F21E9C202EDB19366098286083F3BDCDABE45B300
    Session-ID-ctx: 01000000
    Master-Key: 98DC04D8CD7AE943A08BE013CD4C7D0608950BC201B953BC12755EC9B4804D453148173B00043EF6A01CAC43F7B0005C
    Key-Arg   : None
    Start Time: 1453795701
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---
   2 items in the session cache
   0 client connects (SSL_connect())
   0 client renegotiates (SSL_connect())
   0 client connects that finished
   2 server accepts (SSL_accept())
   0 server renegotiates (SSL_accept())
   2 server accepts that finished
   0 session cache hits
   0 session cache misses
   0 session cache timeouts
   0 callback cache hits
   0 cache full overflows (128 allowed)
---
no client certificate available


===================================================================

Running WordPress with Nginx on ArchLinux

I just moved this blog over to Nginx server from Apache httpd server. I’m pretty satisfied with the overall result. I had to take some time to convert my current httpd configuration over to Nginx, since the new server does not support .htaccess or mod_redirects. This is my current requirements for move over:

  • The site is available on both HTTP and HTTPS.
  • “wp-admin” session is forced to use SSL.
  • I have “quicklook” (to check my server status) and “webalizer” directories under the blog, and they are protected by HTTP BasicAuth.
  • HTTP BasicAuth is to be carried out via SSL.
  • To enforce gzip compression on HTTP connection while disabling it on HTTPS.

Basically I followed the ArchLinux wiki for the implementation, and I will briefly describe what I did.

Nginx (pronounced “Engine X”) is a light-weight open-source http server. Its low resource consumption is the primary purpose for the moveover, and it’s suitable for my server on the cloud.

Firstly, I needed to install the package. And installed “php-cgi” package which is used to provide fastcgi interface to PHP.

~$ sudo pacman -S nginx php-cgi

Then, I configured fastcgi daemon, and add it to rc.d. So the following script was needed to be added to /etc/rc.d as “fastcgi”

#!/bin/bash

. /etc/rc.conf
. /etc/rc.d/functions

case "$1" in
  start)
	stat_busy 'Starting Fastcgi Server'
	if /usr/bin/php-cgi -b 127.0.0.1:9000 &
	then
		add_daemon fastcgi
		stat_done
	else
		stat_fail	fi
	fi
	;;
  stop)
	stat_busy 'Stopping Fastcgi Server'
	[ -e /var/run/daemons/fastcgi ] && kill $(pidof php-cgi) &> /dev/null;
	if [ $? -gt 0 ]; then 
		stat_fail
	else
		rm_daemon fastcgi
		stat_done
	fi
	;;
  restart)
	$0 stop
	$0 start
	;;
  *)
	echo "Usage: $0 {start|stop|restart}"
sac

And I gave it an executable permission:

~$ sudo chmod +x /etc/rc.d/fastcgi

What that script does is to have php-cgi process to listen on port 9000. Now, we would be able to start/stop/restart the daemon with “sudo /etc/rc.d/fastcgi start”. But the script will not be automatically started when the unit is rebooted. It needs to be added to /etc/rc.conf. So I added fastcgi to the rc.conf. Here’s the snippet.

...
DAEMONS=(syslog-ng ... fastcgi nginx ...)
...

Then I edited the /etc/nginx/conf/nginx.conf file to point to my blog physical directory. We need to add two servers, one for HTTP and one for HTTPS. This is my sample configuration for server myfineblog.local

    server {
        listen       80;
        server_name  myfineblog.local;
        access_log      /var/log/httpd/myfineblog.local-access.log;
        error_log       /var/log/httpd/myfineblog.local-error.log;
        root            /srv/http/myfineblog;
        gzip            on;

        location ~ ^/(wp-admin|quicklook|webalizer)/* {
            rewrite ^/(.*) https://myfineblog.local/$1 permanent;
        }

        location / {
            index  index.html index.htm index.php;
            root                /srv/http/myfineblog;
            if (!-e $request_filename) {
                rewrite ^.+/?(/wp-.*) $1 last;
                rewrite &.+/?(/.*\.php)$ $1 last;
                rewrite ^(.+)$ /index.php?q=$1 last;
            }
        }

        location ~ \.php$ {
            fastcgi_pass   127.0.0.1:9000;
            fastcgi_index  index.php;
            fastcgi_param  SCRIPT_FILENAME  /srv/http/myfineblog/$fastcgi_script_name;
            include        fastcgi_params;
        }
    }

Line 3 defines the server name (so we can configure virtual hosts based on names).
Line 4-5 defines the access logs for this web site.
Line 6 is the physical location of the web site on local system.
Line 7 is used to turn on gzip.
Line 9-11 is redirect to SSL by sending HTTP redirect if the uri contains any of wp-admin or quicklook or webalizer)
Line 13-21 is the definition of website directory and an equivalent scripts for Apache’s mod_rewrite.
Line 23-29 is the connection to the fastcgi daemon we configured above. It is *important* to change the SCRIPT_FILENAME variable to suit the real physical path of the wordpress script.

To enable SSL server, I assume we already have the certificate and key for the website. The configuration looks the same but it will have SSL options enabled and Basic HTTPAuth section for a certain directories.

    server {
        listen          443;
        server_name     myfineblog.local;
        ssl                     on;
        ssl_certificate         /etc/ssl/certs/myfineblog.crt;
        ssl_certificate_key     /etc/ssl/private/myfineblog.key;
        ssl_session_timeout     5m;
        ssl_ciphers             HIGH:MEDIUM;
        ssl_prefer_server_ciphers       on;
        ssl_protocols           SSLv3 TLSv1;

        root                    /srv/http/myfineblog;
        access_log              /var/log/httpd/myfineblog.local-ssl_access.log;
        error_log               /var/log/httpd/myfineblog.local-ssl_error.log debug;
        gzip                    off;

        location ~ ^/(quicklook|webalizer)/* {
                auth_basic      "Private Section";
                auth_basic_user_file    /srv/http/myfineblog/.htpasswd;
        }
        location / {
                index   index.html index.htm index.php;
                root    /srv/http/myfineblog;
        }
        location ~ \.php$ {
            fastcgi_pass   127.0.0.1:9000;
            fastcgi_index  index.php;
            fastcgi_param  SCRIPT_FILENAME  /srv/http/myfineblog/$fastcgi_script_name;
            fastcgi_param  HTTPS on;
            include        fastcgi_params;
        }
    }

This configuration turned on “SSL”, disabling SSLv2 and weak ciphers. It enabled HTTP Basic Authentication for two directories. I disabled gzip on SSL stream. And it tells the fastcgi server to turn HTTPS on.

And started the daemons with “/etc/rc.d/fastcgi start” and “/etc/rc.d/nginx start”.

SSL and HTTP Basic Authentication

In general, when I want to force the browser to access certain part of my website via https if the request is made with http, I would put a .htaccess inside that web directory.

RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}

But when I want to protect the directory with HTTP Basic Auth, it creates double authentication. I’ll expand this section after I captures the headers.

As a quick workaround, I use this hack in .htaccess

SSLOptions +StrictRequire
SSLRequireSSL
AuthUserFile /home/minn/.htpasswd
AuthType Basic
AuthName "Private Section"
Require valid-user
ErrorDocument 403 https://www.minnmyatsoe.com/private/