cipherdyne.org

08 November, 2008

Recently on the fwknop mailing list, Mark V asked whether Single Packet Authorization is compatible with virtual Linux instances in Amazon's Elastic Cloud (EC2) computing service. After signing up for an account and running a few tests, it turns out that fwknop can function properly on an EC2 instance and wrap an SPA hardening layer around SSHD. The Amazon online documentation on how to access Linux instances over SSH recommends adding new rules for each IP source address you want to use to access SSHD, which can become tiresome if you are on a network running DHCP. With fwknop, you don't need to do any such thing - you can access SSHD via SPA from anywhere, and anyone else scanning for SSHD on the instance can't even tell that it is listening.

To re-create the steps listed in this blog post, it is necessary to have signed up for an EC2 account and to have created an SSH keypair so that instances can be launched from the command tools provided by Amazon. For this post, all commands on the fwknop client side are executed on an Ubuntu 8.04 system to gain access into the Amazon cloud.

First, let's launch a new virtual instance of Fedora 8 (the command ec2-describe-images -o self -o amazon can be used to find instance identifiers for Fedora 8 systems):

[ubuntu]$ ec2-run-instances ami-abcd1234 -k cdyne-keypair
RESERVATION    r-abcd1234   111111114274    default
INSTANCE       i-abcd1234   ami-abcd1234    pending cdyne-keypair   0
m1.small       2008-11-07T20:18:43+0000     us-east-1c   aki-abcd1234
ari-abcd1234

[ubuntu]$ ec2-describe-instances i-abcd1234
RESERVATION     r-abcd1234  111111114274    default
INSTANCE        i-abcd1234  ami-abcd1234
ec2-70-100-249-20.compute-1.amazonaws.com
domU-12-31-00-00-00-00.compute-1.internal   running cdyne-keypair   0
m1.small        2008-11-07T20:18:43+0000    us-east-1c   aki-abcd1234
ari-abcd1234

Now, we allow SSH access temporarily from the Ubuntu IP address (which will be managed by fwknop after we get it installed and set up instead). We also allow all UDP packets over port 62201 through from any IP since this is the default port and protocol used by fwknop to send SPA packets, and we need such packets to reach far enough into the cloud so that it will be seen by the fwknopd sniffer running on the Fedora 8 instance:

[ubuntu]$ ec2-authorize default -p 22 -s 123.1.1.1/32
GROUP       default
PERMISSION  default ALLOWS  tcp  22  22  FROM  CIDR  123.1.1.1/32

[ubuntu]$ ec2-authorize default -P udp -p 62201
GROUP       default
PERMISSION  default ALLOWS  udp  62201  62201

We can now log into the instance as root:

[ubuntu]$ ssh -i cdyne-keypair root@ec2-70-100-249-20.compute-1.amazonaws.com
The authenticity of host 'ec2-70-100-249-20.compute-1.amazonaws.com
(70.100.249.20)' can't be established.
RSA key fingerprint is 2a:aa:aa:c9:6f:aa:aa:aa:a0:48:7d:db:d6:aa:aa:aa.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ec2-70-100-249-20.compute-1.amazonaws.com,
70.100.249.20' (RSA) to the list of known hosts.

         __|  __|_  )  Fedora 8
         _|  (     /    32-bit
        ___|\___|___|

 Welcome to an EC2 Public Image
                       :-)
    Base

 --[ see /etc/ec2/release-notes ]--

We have a shiny new EC2 instance running Fedora 8. For fwknop to run properly, we'll need the perl-devel and libpcap-devel packages installed, and then we'll install fwknop (some output below has been abbreviated) and configure the /etc/fwknop/access.conf file for SPA access to SSHD. We also add iptables rules to allow packets that are part of established connection through, and drop all other attempts to communicate with SSHD. Finally, we start fwknopd: [root@domU-12-31-00-00-00-00 ~]# yum install perl-devel libpcap-devel
[root@domU-12-31-00-00-00-00 ~]# cd /usr/local/src/
[root@domU-12-31-00-00-00-00 src]# wget http://www.cipherdyne.org/fwknop/download/fwknop-1.9.9-pre7.tar.gz
[root@domU-12-31-00-00-00-00 src]# tar xfz fwknop-1.9.9-pre7.tar.gz
[root@domU-12-31-00-00-00-00 src]# cd fwknop-1.9.9-pre7
[root@domU-12-31-00-00-00-00 fwknop-1.9.9-pre7]# ./install.pl
[...]
[+] It appears that the following network interfaces are attached to the
system:
eth0
lo
Which network interface would you like fwknop to sniff packets from? eth0

[root@domU-12-31-00-00-00-00 ~]# cat > /etc/fwknop/access.conf
SOURCE: ANY;
OPEN_PORTS: tcp/22;
KEY: thisisatestkey;
FW_ACCESS_TIMEOUT: 30;

[root@domU-12-31-00-00-00-00 ~]# iptables -F
[root@domU-12-31-00-00-00-00 ~]# iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
[root@domU-12-31-00-00-00-00 ~]# iptables -A INPUT -p tcp --dport 22 -j DROP
[root@domU-12-31-00-00-00-00 ~]# /etc/init.d/fwknop start
Starting fwknopd:       [ OK ] In the installation of fwknop above, we selected eth0 as the Ethernet interface that fwknopd sniffs for SPA packets, and then we configured the /etc/fwknop/access.conf file with a symmetric key for SPA access to SSHD over TCP port 22. Also, with iptables now deployed to drop SSH communications, we cannot even see that SSHD is listening under an nmap scan:

[ubuntu]$ nmap -P0 -p 22 -sT ec2-70-100-249-20.compute-1.amazonaws.com

Starting Nmap 4.53 ( http://insecure.org ) at 2008-11-08 01:33 EST
Interesting ports on ec2-70-100-249-20.compute-1.amazonaws.com
(70.100.249.20):
PORT   STATE    SERVICE
22/tcp filtered ssh

This is where fwknop comes in. After running the fwknop client and typing the shared encryption key, we can now access the Fedora 8 instance with SSH:

[ubuntu]$ fwknop -A tcp/22 -a 123.1.1.1
-D ec2-70-100-249-20.compute-1.amazonaws.com

[+] Starting fwknop client (SPA mode)...
[+] Resolving hostname: ec2-70-100-249-20.compute-1.amazonaws.com
[+] Enter an encryption key. This key must match a key in the file
    /etc/fwknop/access.conf on the remote system.

Encryption Key:

[+] Building encrypted Single Packet Authorization (SPA) message...
[+] Packet fields:

        Random data:    9189958885117496
        Username:       mbr
        Timestamp:      1226119984
        Version:        1.9.9-pre7
        Type:           1 (access mode)
        Access:         123.1.1.1,tcp/22
        SHA256 digest:  EPOjHE2ANS3Fe3MHxDpok13qHDGbYziFiohv6CyHQXs

[+] Sending 161 byte message to 70.100.249.20 over udp/62201...

[ubuntu]$ ssh -i cdyne-keypair root@ec2-70-100-249-20.compute-1.amazonaws.com

         __|  __|_  )  Fedora 8
         _|  (     /    32-bit
        ___|\___|___|

 Welcome to an EC2 Public Image
                       :-)
    Base

In his original post, Mark had suggested a tighter level of integration with Amazon's EC2 service by building in some notion of instance identifiers into SPA communications. With the basic network access now validated and SPA in its raw form demonstrated to be compatible with EC2, additional integration points can be achieved. Incidentally, port knocking would also be compatible with the EC2 service, but I would not recommend using it because port knocking would look like a port scan to any IDS that Amazon may have deployed to monitor the cloud network. With SPA, only a single packet with an encrypted payload is involved, and this is unlikely to trigger an IDS alarm.

18 September, 2008

A key advantage to using Single Packet Authorization (SPA) over port knocking is the fact that it is trivially easy to harden SPA packets against replay attacks by including a significant number of random bytes within each SPA packet. Then, by tracking the SHA-256 message digest (or the digest from some other suitable cryptographic hashing algorithm) for all incoming SPA packets, the server can enforce that only unique SPA packets ever result in elevated access through a default-drop firewall policy. Any duplicate SPA packet will match a previously calculated digest, and such a packet is therefore flagged as a replay attack (subject to the usual concerns surrounding the minuscule chance of a hash collision - though for SHA-256 this is exceedingly unlikely). In the case of SPA packets generated by the fwknop client, a full 16 bytes of random data is included at the beginning of each packet before it is encrypted.

Another benefit of randomizing SPA packet data - beyond thwarting replay attacks - is that it becomes more difficult to write signatures for intrusion detection systems to detect SPA traffic. The structure of SPA packets generated by the fwknop client is designed to minimize sections that remain the same from one invocation of the client to the next - such identifying sections could be used to write Snort rules to detect fwknop SPA packets. This blog post contains examples of such Snort rules which look for the base64-encoded versions of the string "Salted__" or the hex bytes "0x8502" at the very beginning of UDP packets over port 62201 (the default port used by fwknop). There is also a rule to look for trailing '=' characters at the end of such packets in order to detect the marker used by base64 encoding when the length of the original data is not a multiple of four. These predictable bytes are artifacts of the Crypt::CBC module, GnuPG, or base64 encoding itself, but any recent release of fwknop strips these bytes out of SPA packets before sending them on the wire. The fwknopd server adds them back in if necessary before attempting to base64 decode and decrypt incoming SPA packets.

So, given that fwknop uses CBC mode for symmetric encryption and random bytes are at the very beginning of the payload data, one would expect that SPA packets from fwknop - when viewed via a sniffer - are essentially random variations of base64-encoded data, correct? And this should remain true across all SPA packets that are encrypted with the same encryption key, and even across the same access requests?

Let's see how close fwknop gets to this...

What we need to see is the distribution of byte values across each byte position in a large sampling of SPA packets. That is, for the first byte in each packet across our sample, can we find a relation to any of the other first bytes in the other packets? This process needs to be repeated across all bytes in every packet. To rigorously test for randomness properties we could also get more sophisticated and use the NIST Statistical Test Suite, but for now we'll just focus on using Gnuplot to visualize the byte distribution across two sets of 20,000 SPA packets. If Gnuplot shows any recognizable features or relationships across our sample sets, then we know that we have more work to do. For reference, if you want to duplicate the analysis in this blog post or perform your own analysis of the SPA packet data sets (20,000 packets each), you can download them.

We need two capabilities: 1) the ability to automatically generate 20,000 SPA packets with the fwknop client and store them in a file (one packet to a line), and 2) the ability to parse the file and count the number of different characters in each byte position across all 20,000 SPA packets and print this data to a file so that it can be graphed. These goals are accomplished with this patch to fwknop (which adds a new command line argument --Save-packet-append to be released in fwknop-1.9.8) and this script base64_byte_frequency.pl respectively.

Now, let's collect the first set of 20,000 SPA packets. Each packet is encrypted in this case with the Rijndael cipher, and we use the --Include-salted and --Include-equals command line arguments to produce SPA packets that are compatible with older fwknopd servers: [spaclient]$ for ((i=1;i<=20000;i+=1)); do fwknop -A tcp/22 -a 127.0.0.1 -D 127.0.0.1 --Save-packet --Save-packet-file spa_artifacts.pkts --Save-packet-append --get-key spa.pw --Include-salted --Include-equals > /dev/null; echo $i; done
1
2
3
...
19998
19999
20000
[spaclient]$ head -n 2 spa_artifacts.pkts
U2FsdGVkX1/s7OmvQtYk9UnQuGg8htJ+19pru9NdhflmGhS9d/9fFET7jnPKWsk3/lnd CbGprkkUBkEYXNORP8RU8ZhkCS+pexZ2J+FbQzmwiuD7/9nA1DAmBGnUQi7mLyZYtOGm iCa8yL9IaP43DVhMflAEf+tQGaPw5xUd2/0=
U2FsdGVkX18NuTMq50ef6hjD0t16ZUnKrVYjirVW81UIRNDfclS812vZAWTdcio8t3GC QVxZz/uwrJsjkzevD0OhxJhba+CQVeKuJpfUOhskDQMtYDcH2hkGeDRK9Oc6AfLjO5Fd JXxBJuf8pmxfLC2iWkfAfooCJpjkOm+afH4= All of the SPA packets generated by the command above are written to the file spa_artifacts.pkts, and now let's execute the base64_byte_frequency.pl script against it. This script will create two new files spa_artifacts.pkts.2d and spa_artifacts.pkts.3d for graphing in two and three dimensions. The 2D file contains a simple count of each base64 encoded character, and the 3D file contains a count of each character at each position in the collection of SPA packets. We'll use Gnuplot to graph this data as follows: [spaclient]$ ./base64_byte_frequency.pl spa_artifacts.pkts
[+] Parsing SPA packets from: spa_artifacts.pkts...
[+] Writing 2D results to: spa_artifacts.pkts.2d...
[+] Writing 3D results to: spa_artifacts.pkts.3d...
[spaclient]$ cat > spa_artifacts_2d.gnu
reset
set title "2D SPA packet randomness (artifacts included)"
set terminal png transparent nocrop enhanced
set output "spa_artifacts_2d.png"
set xlabel "char"
set ylabel "frequency"
plot 'spa_artifacts.pkts.2d' using 1:2 with lines title '(char,freq)'
[spaclient]$ cat > spa_artifacts_3d.gnu
reset
set title "3D SPA packet randomness (artifacts included)"
set terminal png transparent nocrop enhanced
set output "spa_artifacts_3d.png"
set xlabel "position"
set ylabel "char"
set zlabel "frequency"
set view 72,9
splot 'spa_artifacts.pkts.3d' using 1:2:3 with points title '(pos,char,freq)'
[spaclient]$ gnuplot spa_artifacts_3d.gnu
[spaclient]$ gnuplot spa_artifacts_2d.gnu
[spaclient]$ ls -atlr *png
-rw-r--r-- 1 mbr mbr 5977 2008-09-14 09:31 spa_artifacts_2d.png
-rw-r--r-- 1 mbr mbr 4194 2008-09-14 09:31 spa_artifacts_3d.png The last two lines above indicate that Gnuplot has created two graphics files spa_artifacts_2d.png and spa_artifacts_3d.png for plotting the data in two and three dimensions. Because we're graphing base64 characters as integer data, the base64_byte_frequency.pl script maps each base64 character to numerically ordered values beginning at zero. We could have just graphed the hex value of each character directly, but this would result in gaps. For reference, the base64 alphabet is described by the following characters (in order): +, /, 0-9, A-Z, a-z. There is also the equals "=" character, but it is only used as a closing marker. So, the base64_byte_frequency.pl script maps "+" to 0, "/" to 1, "0-9" to 1-11, "A-Z" to 12-37, "a-z" to 38-63, and "=" to 64.

Here is the 2-dimensional graph that Gnuplot has produced: You can see in the graph above that there are several spikes for a few character values. These spikes correspond with the characters U2FsdGVkX1, which is the base64 encoded version of the string "Salted__". Every SPA packet in the spa_artifacts.pkts file begins with this string, and so one would expect more of these characters on average than the others. There is also a sharp dip to 20,000 on the right-hand portion of the graph for the value 64, which corresponds to the = character. This is because there is only one equals character at the end of each SPA packet in the spa_artifacts.pkts file.

Because base64 encoding uses an alphabet of 64 characters, and given that our sample size is 20,000 packets of 171 bytes each (less the ending = marker), we would expect the average number of times each character is represented to be 171 / 64 * 20000 = 53,000 (approximately). That is, we can expect this value if the SPA packets are perfectly random across the sample set. The spikes in the 2D graph throw this expected value off a bit.

Now let's take a look at the 3-dimensional plot of the same data: This graph shows us where the predictable bytes are. We know that each SPA packet begins with U2FsdGVkX1 and ends with =, and the outliers (on the left-hand side and one point on the right-hand side - all with Z-axis values of about 20,000) in the above graph illustrate this. The X-axis represents the byte positions of each packet, the Y-axis are the numeric character codes, and the Z-axis is the number of times a character is counted.

So, given that any recent release of fwknop removes the "Salted__" prefix and any trailing "=" characters, are the graphs of such SPA packets now highly randomized?

The answer is "almost".

Let's produce the same two graphs, but this time for SPA packets produced by any recent version of fwknop (without using the --Include-salted or --Include-equals command line arguments). [spaclient]$ for ((i=1;i<=20000;i+=1)); do fwknop -A tcp/22 -a 127.0.0.1 -D 127.0.0.1 --Save-packet --Save-packet-file spa_no_artifacts.pkts --Save-packet-append --get-key spa.pw > /dev/null; echo $i; done
1
2
3
...
19998
19999
20000
[spaclient]$ head -n 2 spa_no_artifacts.pkts
/ZlXox6SyQsyMpKVj+cxHlmSpMWOaAyxmou5LgThZ58yXlxFrR7BGtqSRJJTELrUgJ2/ RiX9If+NLXEANJSu9nogO8ZG2R6cllQlhT1vM6YiLsktujuV3a7r81I1JSCwC4YZdNQC Y2xwcRGLMEEhykGhp0RKEYlSo
8jX5NECzw4NGVg3LIEhu30ph0uvBa4ijW9/S7IQEQvdj/IPHle8KRoeyHvskj1fd0XU7 beUIb5zCKsFq7NTTUsnwaccULAKQHZz4Qm0ZTVC8st0doDfhU5Bl46LbDXPCejRbSQEd cVgwDwaU8XqEKr8kYjXZ3ZNGA The 2-dimensional graph looks pretty good: Now that the invariant sections have been removed, the length of each packet is 161 characters, so if the randomness were perfect, we would expect the average instance of each character to be 161 / 64 * 20000 = 50,300 (approximately), and this is quite close to the value displayed by the graph. There are still two minor spikes corresponding to the numeric values around 0 and 1, and then again around 10 and 11 on the X-axis. These bytes correspond to the characters +, /, 8, and 9 respectively.

Let's look at the 3-dimensional graph to see if we can get a sense of where these spikes are in terms of byte positions within the SPA packets: So, at the scale shown above, the 3D graph looks decently random - at least there are far fewer outliers than for the first data set of 20,000 packets, and there aren't any other obviously regular structures in the graph. However, if you look to the left-hand portion of the graph, you will see there are small outliers at position 0 on the X-axis - the very first byte of each SPA packet. Looking through the spa_no_artifacts.pkts file, the graph is indeed correct - every SPA packet begins with one of the bytes +, /, 8, or 9.

Why?

The reason is that the current fwknop client code strips the string U2FsdGVkX1 from each SPA packet after the raw encrypted data is base64 encoded, and the encoding itself uses +, /, 8, or 9 after the last "_" in the string "Salted__". Although this is not huge problem, it does mean that it is possible to write a Snort rule that uses a PCRE to look for one of these bytes at the very beginning of UDP payload data. This will be fixed in a future release of fwknop.

The techniques illustrated in this blog post can be applied to SPA packets generated by other implementations as well. Some SPA implementations use a hashed payload instead of an encrypted payload which implies that multiple requests for the same access through a firewall will look quite similar in many byte positions, so it is much easier to detect such SPA packets with signatures in an intrusion detection system. By contrast, the SPA packets produced by fwknop are much harder to write reliable signatures for effective detection in any IDS - particularly when combined with the port randomization features offered in recent releases.

Finally, this blog post has only addressed the randomness characteristics of SPA packets that have been encrypted with the Rijndael cipher. An upcoming blog post will perform the same analysis for SPA packets encrypted with GnuPG.

25 August, 2008

In the latest issue (September/October 2008) of Hakin9 Magazine, I had an article published entitled Advanced SPA with fwknop. It was the goal of this article to introduce the port forwarding capabilities of fwknop that make it possible to reach internal services with automatically generated NAT rules, and also to show how fwknop SPA packets (prior to the 1.9.6 release) could be detected with some well-crafted Snort rules that look for certain encryption and encoding artifacts. Also, with the addition of source IP addresses to SPA digest tracking, it is possible to get a sense of routing paths that might at one time have had sniffers watching for SPA packets if a replay attack against the same fwknopd instance is detected at some later time.

The Snort rules mentioned in the article - updated to take into account the more recent 1.9.6 release - are displayed below. The first Snort rule is designed to look for UDP packets over port 62201 that end with two '=' characters - a potential marker of base64-encoded data (when the original data size was not a multiple of four). The second rule looks for the base64-encoded version of the string Salted__, which is added by the Crypt::CBC module to maintain compatibility with how the OpenSSL library encrypts data. The third rule looks for packets that begin with base64-encoded version of the string 0x8502 which is a marker for data encrypted with GnuPG, and also checks to see of the size of the payload is at least 1000 bytes (SPA packets encrypted with GnuPG tend to be larger than those encrypted with Rijndael). Here are the Snort rules: alert udp any any -> any 62201 (msg:"fwknop pre-1.9.6 SPA traffic"; dsize:>150; pcre:"/==$/"; sid:20080001; rev:1;)
alert udp any any -> any 62201 (msg:"fwknop pre-1.9.2 SPA traffic"; content:"U2FsdGVkX1"; depth:10; dsize:>150; sid:20080002; rev:1;)
alert udp any any -> any 62201 (msg:"fwknop GnuPG encrypted pre-1.9.6 SPA traffic"; content:"hQ"; depth:2; dsize:>1000; sid:20080003; rev:1;) Any recent release of fwknop (greater than 1.9.5) strips out these identifying markers before transmitting SPA packets on the wire, so these rules are no longer effective at detecting fwknop SPA communications. Also, strong port randomization features were added in fwknop-1.9.4, both for the randomization of the SPA packet destination port as well as the port where the actual connection (say, SSH) is made, so UDP port 62201 is not effective either when these features are used.

Finally, here is an excerpt from the conclusion of the article:

   In the continual arms race that is computer security today, having a good understanding of network communications and how to customize an IDS rule set to an emerging protocol is an important skill. Finally, SPA offers a compelling addition to the tools available for effective server defense; I personally sleep more soundly knowing that arbitrary IP addresses around the Internet cannot see that I have an SSH daemon running, and yet I can access it from wherever I like.

28 June, 2008

Next month in NYC the final Hackers On Planet Earth (HOPE) conference will take place from July 18th through the 20th. I will be giving a talk there entitled "Port Knocking and Single Packet Authorization: Practical Deployments", and here is the abstract:

   Port Knocking and its big brother, Single Packet Authorization (SPA), can provide a robust additional layer of protection for services such as SSH, but there are many competing Port Knocking and SPA implementations. This talk will present practical usages of fwknop in Port Knocking and SPA modes, and discuss what works and what doesn't from a protocol perspective. Integration points for both iptables and ipfw firewalls on Linux and FreeBSD systems will be highlighted, and client-side support on Windows will be demonstrated. Finally, advanced functionality such as inbound NAT support for authenticated connections, sending SPA packets over the Tor anonymity network, and covert channel usages will be discussed. With SPA deployed, anyone scanning for a service with Nmap cannot even tell that it is listening; let alone target it with an exploit (zero-day or not).

A goal for the talk will be to start with the most basic port knocking deployment (a shared sequence of only one port) and build from there into encrypted port knocking sequences, and then move into the SPA realm with SPA packets encrypted with Rijndael and finally with GnuPG. Along the way security tradeoffs will be discussed. For example, a shared sequence of a single port allows an extremely simplistic port knocking implementation (so there is less risk of a vulnerability in the port knocking software itself), but then any casual port scan or stray packet that hits the shared port also qualifies as a valid port knock sequence. At the high end, SPA packets encrypted with GnuPG solve all sorts of difficulties with simple port knocking from a protocol perspective, but there is the slight expense of a more complicated implementation (although it is still a lot harder to target an SPA implementation with an exploit than a complicated TCP-based service that advertises its existence to the world under any basic port scan).

At the talk I will also release the next version (1.9.6) of fwknop.

If you are going to be at the HOPE conference, please stop by and say 'hello'. No Starch Press will also be at the conference so I'm sure I will hang out at their booth much of the time as well.

01 June, 2008

After two months of development, the 1.9.4 release of fwknop is available for download. This release introduces new functionality that has implications for hardening both fwknop SPA communications and the follow-on connections that client programs make (such as SSH). Specifically, the 1.9.4 release adds two port randomization options to 1) send the SPA packet over a random port between 10,000 and 65,535 (requires updating the default PCAP_FILTER variable on the fwknopd server side), and 2) select a random port that is used as the destination port in a NAT operation on the fwknopd server system. These two options can be used individually or together, and are enabled with two new command line options, --rand-port and --NAT-rand-port respectively, to the fwknop client (see examples below).

The inspiration for adding this functionality came from a post to the "Documentation, Tips & Tricks". Gentoo forum, and I have credited John Brendler with the idea. In response to John's post, I would like to mention however that all Port Knocking / Single Packet Authorization implementations suffer from "piggy-back exploits", including those that select random ports for SPA communications or for NAT operations against follow-on sessions. A "piggy-back exploit" is where an attacker takes advantage of the fact that a firewall rule inserted by a PK/SPA system accepts a connection from an IP address to a destination port specified by the SPA packet(s). So, if the attacker can spoof packets from this source IP, or if the attacker happens to be on the same internal network as the SPA client (and hence sends connections out through the same NAT device), then the firewall rule will accept these packets just as though they originated from the legitimate SPA client system. If an attacker is in a privileged position and can sniff the legitimate session as it is initiated, then one can envision an automated attack that spoofs packets from the same source IP and directs them at the same service. Further, such an "attack" can be made just by watching outgoing connections without paying any attention whatsoever to whether or not a set of SPA packets are sent first - it doesn't matter if the real connection is made to a random translated port on the SPA system; the attacker can see this port in the real connection itself.

Now, should you be concerned about such a piggy-back attack? Not really. First, if the attacker is not going through the same NAT device as the real connection, then any response to a spoofed packet will go back to the spoofed source - not back to the attacker. So, for TCP connections, unless the attacker can effectively perform a sequence prediction attack an existing connection (and even then that is of little use against an encrypted application layer protocol such as SSH), this is not very effective. Second, even if an attacker is behind the same NAT device as the SPA client, just being able to access the targeted service over TCP/IP does not imply an automatic vulnerability; SPA is an additive measure to whatever existing security mechanisms are already in place (barring a vulnerability in libpcap itself in the SPA server for example). Third, there are an awful lot of networks out there to which an attacker will not have such privileged access, and therefore not be in a position able to sniff anything useful. Forth, fwknop minimizes the opportunity for an attacker to conduct a piggy-back attack by maintaining a small window of time (30 seconds by default) for any new firewall rules after receiving a valid SPA packet. By using a connection tracking mechanism built into iptables or ipfw, any connection established during the accept window is allowed to remain open but all attempts to create a new connection must first preceeded with a new SPA packet in order to gain access.

Finally, although port randomization is an enhancement, fwknop has had the ability for a long time to allow the user to select the destination port for SPA packets with the --Server-port argument as well as the destination port for a NAT'd connection to an internal system. Hence, fwknop SPA packets are not always sent over udp/62201. But, I agree that it is useful to add the port randomization features that John Brendler suggested, and this is why I've implemented them in fwknop. Randomizing the SPA destination port along with the destination port of the follow-on connection makes traffic analysis more difficult.

Now, let us see the new --rand-port and --NAT-rand-port options in a practical example. We'll assume that the fwknopd server is at hostname spaserver with IP 11.1.1.1, and the fwknop client runs on the spaclient system with IP 12.2.2.2. We ultimately want to gain access to SSHD on the spaserver system, and we assume that iptables is configured in a default-drop stance for all attempts to communicate with SSHD. Also, there is no requirement to necessarily attempt to gain access only to an SSHD instance running on an internal server via a forwarded port - the iptables PREROUTING chain can forward a port to a local socket as well (based on a routing calculation for the destination IP), and on the fwknop client command line we use the --NAT-local argument for this.

Because the --rand-port option sends the SPA packet over a random destination port, we first need to set the PCAP_FILTER variable as follows in the /etc/fwknop/fwknop.conf file: [spaserver]# vi /etc/fwknop/fwknop.conf
PCAP_FILTER udp dst portrange 10000-65535;

[spaserver]# /etc/init.d/fwknop restart
[+] knopwatchd is running (pid: 17584), stopping daemon
[+] knoptm is running (pid: 17582), stopping daemon
[+] fwknopd is running (pid: 17580), stopping daemon
Starting the fwknop daemons. With the fwknopd server up and running, we now use the fwknop client to gain access to SSHD on the spaserver system via a randomly selected NAT'd port (and we show that SSHD is never accessible over the standard TCP port 22 even from the spaclient system):

[spaclient]$ nmap -sT -n -P0 -p 22 11.1.1.1

Starting Nmap 4.20 ( http://insecure.org ) at 2008-06-02 01:48 EDT
Interesting ports on 11.1.1.1:
PORT   STATE    SERVICE
22/tcp filtered ssh

Nmap finished: 1 IP address (1 host up) scanned in 12.017 seconds

[spaclient]$  fwknop -A tcp/22 --NAT-local --NAT-rand-port \
--rand-port -R -D 11.1.1.1

[+] Starting fwknop client (SPA mode)...
[+] Requesting NAT access for randomized port: 16791
    Resolving external IP via: http://www.whatismyip.org/
    Got external address: 12.2.2.2

[+] Enter an encryption key. This key must match a key in the file
    /etc/fwknop/access.conf on the remote system.

Encryption Key:

[+] Building encrypted Single Packet Authorization (SPA) message...
[+] Packet fields:

        Random data:    4846125760033285
        Username:       mbr
        Timestamp:      1212386108
        Version:        1.9.4
        Type:           5 (Local NAT access mode)
        Access:         12.2.2.2,tcp/22
        NAT access:     11.1.1.1,16791
        SHA256 digest:  ZyWG+nRGYMfWFssJuOy7bhGmJHpHia6T1igaNnVVhqI

[+] Sending 206 byte message to 11.1.1.1 over udp/52949...
    Requesting NAT access to tcp/22 on 11.1.1.1 via port 16791

[spaclient]$  ssh -p 16791 mbr@11.1.1.1
mbr@11.1.1.1's password:
[spaserver]$

The above output shows that the client system indeed has access to the spaserver system via TCP port 16791, which is forwarded to the local SSH daemon. In the /var/log/messages file, fwknopd has written the following messages to syslog: [spaserver]# tail /var/log/messages
Jun 2 01:54:16 spaserver fwknopd: received valid Rijndael encrypted packet from: 12.2.2.2, remote user: mbr, client version: 1.9.4 (SOURCE line num: 151)
Jun 2 01:54:17 spaserver fwknopd: add FWKNOP_PREROUTING 12.2.2.2 -> 11.1.1.1(tcp/16791 to 22) DNAT rule 30 sec
Jun 2 01:54:17 spaserver fwknopd: add FWKNOP_INPUT 12.2.2.2 -> 0.0.0.0/0(tcp/22) ACCEPT rule 30 sec
Jun 2 01:54:58 spaserver fwknop(knoptm): removed iptables FWKNOP_PREROUTING DNAT rule for 12.2.2.2 -> 11.1.1.1(tcp/22), 30 sec timeout exceeded
Jun 2 01:55:11 spaserver fwknop(knoptm): removed iptables FWKNOP_INPUT ACCEPT rule for 12.2.2.2 -> 0.0.0.0/0(tcp/22), 30 sec timeout exceeded In closing, here is an abbreviated version (the randomization options are not duplicated here) of the fwknop-1.9.4 ChangeLog:

• Added the ability to specify the port that SPA packets are sent over with the fwknop client by using the syntax "<host|IP>:<port>". So, for example, to have the client send an SPA packet to 11.1.1.1 over UDP port 12345 (instead of the default of 62201), one could use the following command:
  
  $ fwknop -A tcp/22 -R -D 11.1.1.1:12345
  
  
• Bugfix to add a check for "keep-state" in ipfw policies in addition to the existing "check-state" check (noticed by Sebastien Jeanquier).
• Updated the install.pl script to try to determine the OS type as early as possible during the install process.
• Added the MIN_SPA_PKT_LEN variable with 160 (bytes) as the default. This allows fwknopd to ignore packets that are not at least this many bytes (including packet headers) before any decryption attempt is made.
• Added --time-offset-plus and --time-offset-minus args to the fwknop client command line. This allows the time stamp within an SPA packet to be influenced without setting the system clock (which normal users cannot usually do). This is useful for when the client and server systems have clocks that are out of sync.
• Bugfix on Ubuntu systems to make sure that the fwknop init script is installed with a priority of 99 instead of 20 - this puts fwknop as late as possible within the boot sequence so that the system is ready to run fwknop.
• Bugfix to not open ports that are not specifically requested in an SPA packet even if those ports are listed in the OPEN_PORTS variable in the access.conf file.
• Updated to version 5.47 of the Digest::SHA module.
• Updated to version 0.7 of the IPTables::ChainMgr module (includes perldoc documentation).
• Updated to version 0.6 of the IPTables::Parse module (includes perldoc documentation).
• Added NAT, port randomization, and and time offset option discussions to fwknop(8) man page.

17 April, 2008

Sean Greven has released the next version of his fwknop UI for Windows systems. This release is compatible with all fwknop daemons that are configured to accept SPA packets that have been encrypted with the Rijndael symmetric cipher. The executable can be downloaded here, and the source code is also available here. Finally, here is a screenshot to illustrate generating an SPA packet from Windows 2000 running under VMware on an Ubuntu 7.10 system. The fwknopd daemon is running in --debug mode in the terminal on the right, and you can see the addition of an iptables ACCEPT rule to allow access to SSHD:

05 April, 2008

Most port knocking or Single Packet Authorization implementations offer the ability to passively authenticate clients for access only to a locally running server (such as SSHD). That is, the daemon that monitors a firewall log or that sniffs the wire for port knock sequences or SPA packets can only reconfigure a local firewall to allow the client to access a local socket. This is usually accomplished by allowing the client to connect to the server port by putting an ACCEPT rule in the INPUT chain for iptables firewalls, or adding a pass rule for ipfw firewalls for the client source IP address. For local servers, this works well enough, but suppose that you ultimately want to access an SSH daemon that is running on an internal system? If the SPA software is deployed on a Linux gateway that is protecting a non-routable internal network and has a routable external IP address, it is inconvenient to first have to login to the gateway and then login to the internal system.
       Since the 1.9.2 release, fwknop has supported the creation of inbound port forwarding rules for iptables firewalls via the DNAT target in the PREROUTING chain after receiving a valid SPA packet. This allows direct access to internal systems on non-routable address space from the open Internet. Here is an example - suppose that you are running fwknop on Linux gateway system that is protecting an internal network 192.168.10.0/24 and has an external routable address of 11.1.1.1. Now, suppose that you are on travel somewhere (so your source IP address is not predictable), and you would like to access an SSH server that is running on the internal system 192.168.10.23. Finally, because fwknop is deployed, iptables is configured in a default-DROP stance against all attempts to connect with any SSH daemon; nmap cannot even tell that there is any SSH server listening. Access is granted only after a valid SPA packet is passively sniffed by the fwknopd daemon.

With inbound NAT support configured in fwknop (the config is mentioned below), here is an illustration of fwknop usage in order to gain direct access to SSHD on the 192.168.10.23 from an external network (note that the -R argument instructs the fwknop client to automatically resolve the current IP address - 12.2.2.2 in this case):

[externalhost]$ fwknop -A tcp/22 --Forward-access 192.168.10.23,5001 \
-R -D 11.1.1.1

[+] Starting fwknop client (SPA mode)...
    Resolving external IP via: http://www.whatismyip.org/
    Got external address: 12.2.2.2

[+] Enter an encryption key. This key must match a key in the file
    /etc/fwknop/access.conf on the remote system.

Encryption Key:

[+] Building encrypted Single Packet Authorization (SPA) message...
[+] Packet fields:

        Random data:    6862733471944039
        Username:       root
        Timestamp:      1207404612
        Version:        1.9.3
        Type:           2 (FORWARD access mode)
        Access:         12.2.2.2,tcp/22
        Forward access: 192.168.10.23,5001
        SHA256 digest:  hE4zGafLtQiQiFrep+cSq/wVO7SQhwh65hmLr+ehtrw

[+] Sending 206 byte message to 11.1.1.1 over udp/62201...

Now, port 5001 on the external IP address is forwarded through to the SSH server on the internal 192.168.10.23 system, but only for the client IP 12.2.2.2: [externalhost]$ ssh -p 5001 mbr@11.1.1.1
Password:
[internalhost]$ Graphically, this scenario is illustrated by the following picture. The dotted lines represent the SPA packet from the fwknop client (which only needs to be sniffed by the fwknopd daemon running on the Linux gateway), and the solid arrows represent the SSH connection from the external client through to the internal SSH server.


On the Linux gateway system that is running the fwknop daemon, executing the following command illustrates the additions to the iptables policy to allow the SSH connection to be forwarded to the internal system. These firewall rules are automatically deleted after a 30 second timeout (this is tunable), but any existing SSH connection remains open through the use of the iptables connection tracking capability.

[gateway]# fwknopd --fw-list
[+] Listing rules in fwknop chains...
Chain FWKNOP_INPUT (1 references)
 pkts bytes target prot opt in out source   destination

Chain FWKNOP_FORWARD (1 references)
 pkts bytes target prot opt in out source     destination
   19  2740 ACCEPT tcp  --  *  *   12.2.2.2   0.0.0.0/0    tcp dpt:22

Chain FWKNOP_PREROUTING (1 references)
 pkts bytes target prot opt in out source     destination
    1    60 DNAT   tcp  --  *  *   12.2.2.2   0.0.0.0/0    tcp \
dpt:5001 to:192.168.10.23:22

Finally, the /etc/fwknop/access.conf is configured like so to facilitate this example, and the ENABLE_IPT_FORWARDING variable is also enabled in the /etc/fwknop/fwknop.conf file: SOURCE: ANY;
OPEN_PORTS: tcp/22;
PERMIT_CLIENT_PORTS: N;
ENABLE_FORWARD_ACCESS: Y;
FW_ACCESS_TIMEOUT: 30;
KEY: aesdemokey; In addition to explaining the inbound NAT support offered by fwknop, this blog post also announces the 1.9.3 release of fwknop. Here is the complete ChangeLog:

• Added MASQUERADE and SNAT support to complement inbound DNAT connections for SPA packets that request --Forward-access to internal systems. This functionality is only enabled when both ENABLE_IPT_FORWARDING and ENABLE_IPT_SNAT are set, and is configured by two new variables IPT_MASQUERADE_ACCESS and IPT_SNAT_ACCESS which define the iptables interface to creating SNAT rules. The SNAT supplements of DNAT rules are not usually necessary because internal systems usually have a route back out to the Internet, but this feature accommodates those systems that do not have such a route. By default, the MASQUERADE target is used if ENABLE_IPT_SNAT is enabled because this means that the external IP does not have to be manually defined. However, the external IP can be defined by the SNAT_TRANSLATE_IP variable.
• Added hex_dump() feature for fwknop client so that raw encrypted SPA packet data can be displayed in --verbose mode.
• When ENABLE_IPT_FORWARDING is set, added a check for the value of the /proc/sys/net/ipv4/ip_forward file to ensure that the local system allows packets to be forwarded. Unless ENABLE_PROC_IP_FORWARD is disabled, then fwknopd will automatically set the ip_forward file to "1" if it is set to "0" (again, only if ENABLE_IPT_FORWARDING is enabled).
• Minor bugfix to remove sys_log() call in legacy port knocking mode.
• Minor bugfix to expand both the Id and Revision tags via the svn:keywords directive.

09 June, 2007

The February issue of the "Starter Kit" edition of Hakin9 Magazine contains a lengthy article on fwknop written by Raul Siles. The Starter Kit edition aims to provide introductory articles and tutorials that explain important computer security technologies from the ground up, and I think this is valuable because security is a complex field and there is generally a lack of good material of this kind. Raul's article is entitled "Knock Knock Knocking on Firewall's Door", and is featured on the magazine cover. In the article, Raul provides a history of passive authorization technologies beginning with Port Knocking and extends the discussion into the latest generation of these technologies called Single Packet Authorization. He gives detailed information about how to install and configure fwknop-1.0 on a Fedora Core 6 system, and shows how iptables in a default-drop stance for protected services do not appear underneath the watchful eyes of Nmap. Here is a short excerpt from the article:

"...Although port knocking and SPA can be used to protect any service and its associated ports, they are mainly used to protect and enable access to remote management protocols, and especially long running TCP sessions, such as Secure Shell, SSH (TCP port 22).
      The article presents two configuration example. The first one focuses on allowing access to SSH while the second allows the client to manage remotely the rules that should be applied in the iptables firewall..."

Incidentally, the same issue of Hakin9 includes coverage of the psad and fwsnort projects in another article entitled "Much More Than Just a Firewall" written by Jess Garcia. His article covers several security technologies including honeypots and snort_inline, and makes the case that firewalls can do much more than simple filtering operations on network traffic. I'm happy to see Cipherdyne projects getting more exposure in the security literature.

04 April, 2007

The current issue (May '07) of the Linux Journal, contains part II of my series on Single Packet Authorization (SPA). The article is entitled Protecting SSH Servers with Single Packet Authorization, and appears to not be available yet through the LJ website so I suppose that it may only be in the print edition for a while. Part II is a hands-on treatment of fwknop operations, and covers SPA with both symmetric and asymmetric ciphers. here is a short excerpt from the article:

"...This article gets away from theory and concentrates on the practical application of SPA with fwknop and iptables to protect SSHD from reconnaissance and attack. With this setup on a Linux system, no one will be able to tell that SSHD is even listening under an nmap scan, and only authenticated and authorized clients will be able to communicate with SSHD.
    To begin, we require some information about configuration and network architecture. This article assumes that you have installed the latest version of fwknop (1.0.1 as of this writing) on the same system where SSHD and iptables is running. You can download fwknop from http://www.cipherdyne.org/fwknop/ and install either from the source tar archive by running the install.pl script, or install via the RPM for RPM- based Linux distributions..."

04 March, 2007

For the April security issue of the Linux Journal, I have written the first of a two-part article on the concept of Single Packet Authorization (SPA). The first installment lays a theoretical foundation for why the security architecture and capabilities of SPA are superior to Port Knocking. The second installment (to be published in May) will provide a hands-on examination of how to use fwknop to harden an OpenSSH server behind a default-drop iptables policy. Here is an excerpt from the article:

"...When an attacker is on the prowl in an attempt to exploit a vulnerability in server software (as opposed to client software), the first step is reconnaissance; the attacker needs to locate a target. This process has been brilliantly automated by Nmap, so it is easy to construct a list of target systems that may be ripe for compromise. If the attacker has found a zero-day vulnerability in server software that you happen to be running, you don't want to appear in this list of targets! Both port knocking and Single Packet Authorization use a packet filter configured in a default-drop stance and simultaneously provide service only to those IP addresses that can prove their identity via a passive mechanism. No TCP/IP stack access is required to authenticate remote IP addresses via this passive means. Nmap cannot even tell that a server is running when protected in this way, and it does not matter even if the attacker has a zero-day exploit..."