Network scanning, OS fingerprinting part 1

One of the  primary challenges you face is  detecting  cyber attacks and cyber threats at their earliest stage.

If you use, the Cybrer Kill Chain framework developed by Lockheed Martin, you will likely find them in the "Reconnaissance" phase, when and intruder first seeks a target and attempts to identify vulnerabilities.

But when it comes to network scanning, this isn’t as simple as it seems. How do you distinguish between "innocent" scans performed by an application flaw or misconfigurations from spoofed IP addresses and serious bad-guy scans whose intention is to weaponize and then deliver some malicious code?

If you respond to every scan by blocking the source IP, then, sooner or later, you will end up with a gigantic blacklist that probably blocks some IPs you actually shouldn't. If those happen to belong to a business partner, customer or cloud application that your organization is using, it may be a while before you realize and fix it. In the case of a business partner or customer, your organization is losing trust and with a cloud application, it is basically a denial of service (DoS) —despite the good intention.

So, your goal should be to focus only on those scanning attempts that can be dangerous or worth tracking, instead of blocking them all.  We’ll take a look in this three-part series, starting with a familiar activity.

(subhead) What’s your operating system?

When an attacker wants to scan and determine the operating system (OS), this can be a situation that warrants monitoring and, perhaps, blocking. Why? When an attacker has information about the OS, he can estimate the detection mechanisms he needs to avoid and weaponize the malware.

A very popular, open source tool called Nmap, with option "-O" specified, makes OS detection easier. Nmap is widely used for that purpose, and even the source code and libraries are publicly available and used also by other tools. Let’s look at the OS detection mechanism and see how Nmap creates an OS fingerprint based on the information received from the client.

At the end we will elaborate about effective SIEM detection and give few examples of rules/searches that can help us detect an attacker running Nmap with OS detection against our systems.

To detect the OS, Nmap sends to the target a few specially crafted packets and, based on the received information, it creates an OS “fingerprint.” Because of the different implementation of the TCP/IP stack and respective RFCs by individual vendors, the response to Nmap special packets produced by operating systems varies. The OS fingerprint is really just a record of all the values received from the target, tucked into a database. When the OS scan is performed, Nmap compares the received values with its database and can determine quite precisely the possible operating system.

What do the NMAP database and fingerprint look like?

Based on the system that is running Nmap, the fingerprint database can be located in different places. If you want to see the content of the database, look for a file “Nmap-os-db,” then choose your favorite viewer to open it.

For comparison, we have chosen two fingerprints from this file: Windows 7 and Cisco 19xx router with IOS 15.4. 

Fingerprint Microsoft Windows 7

Class Microsoft | Windows | 7 | general purpose

CPE cpe:/o:microsoft:windows_7

SEQ(SP=F6-100%GCD=1-6%ISR=10B-115%TI=I%CI=I%II=I%SS=S%TS=7)

OPS(O1=M523NW8ST11%O2=M523NW8ST11%O3=M523NW8NNT11%O4=M523NW8ST11%O5=M523NW8ST11%O6=M523ST11)

WIN(W1=2000%W2=2000%W3=2000%W4=2000%W5=2000%W6=2000)

ECN(R=Y%DF=Y%T=7B-85%TG=80%W=2000%O=M523NW8NNS%CC=N%Q=)

T1(R=Y%DF=Y%T=7B-85%TG=80%S=O%A=S+%F=AS%RD=0%Q=)

T2(R=Y%DF=Y%T=7B-85%TG=80%W=0%S=Z%A=S%F=AR%O=%RD=0%Q=)

T3(R=Y%DF=Y%T=7B-85%TG=80%W=0%S=Z%A=O%F=AR%O=%RD=0%Q=)

T4(R=Y%DF=Y%T=7B-85%TG=80%W=0%S=A%A=O%F=R%O=%RD=0%Q=)

T5(R=Y%DF=Y%T=7B-85%TG=80%W=0%S=Z%A=S+%F=AR%O=%RD=0%Q=)

T6(R=Y%DF=Y%T=7B-85%TG=80%W=0%S=A%A=O%F=R%O=%RD=0%Q=)

T7(R=Y%DF=Y%T=7B-85%TG=80%W=0%S=Z%A=S+%F=AR%O=%RD=0%Q=)

U1(DF=N%T=7B-85%TG=80%IPL=164%UN=0%RIPL=G%RID=G%RIPCK=G%RUCK=G%RUD=G)

IE(DFI=N%T=7B-85%TG=80%CD=Z)

Fingerprint Cisco 1900-series router (IOS 15.4)

Class Cisco | IOS | 15.X | router

CPE cpe:/o:cisco:ios:15.4

SEQ(SP=102-10C%GCD=1-6%ISR=106-110%TI=RD%CI=RD|RI%II=RI%TS=U)

OPS(O1=M564%O2=M564%O3=M564%O4=M218%O5=M564%O6=M564)

WIN(W1=1020%W2=1020%W3=1020%W4=1020%W5=1020%W6=1020)

ECN(R=Y%DF=N%T=FB-105%TG=FF%W=1020%O=M564%CC=N%Q=)

T1(R=Y%DF=N%T=FB-105%TG=FF%S=O%A=S+%F=AS%RD=0%Q=)

T2(R=N)

T3(R=N)

T4(R=Y%DF=N%T=FB-105%TG=FF%W=0%S=A%A=Z%F=R%O=%RD=0%Q=)

T5(R=Y%DF=N%T=FB-105%TG=FF%W=0%S=A%A=S+%F=AR%O=%RD=0%Q=)

T6(R=Y%DF=N%T=FB-105%TG=FF%W=0%S=A%A=Z%F=R%O=%RD=0%Q=)

T7(R=Y%DF=N%T=FB-105%TG=FF%W=0%S=A%A=S%F=AR%O=%RD=0%Q=)

U1(DF=N%T=FB-105%TG=FF%IPL=38%UN=0%RIPL=G%RID=G%RIPCK=G%RUCK=G%RUD=G)

IE(DFI=S%T=FB-105%TG=FF%CD=S)

Every fingerprint has one or more Class lines. Each contains four well-defined fields: vendor, OS family, OS generation and device type. The fields are separated by the pipe symbol (|). After the Class line you can see Common Platform Enumeration (CPE) ,the structured naming scheme based on generic syntax for uniform resource identifiers.

Before we will start decoding subsequent rows, let's refresh our knowledge about TCP/IP protocols.

What do the IP header and TCP header look like?

To better understand the whole magic about Nmap OS fingerprinting, we should be familiar with IP, TCP, UDP and ICMP protocols. We’ll focus on the header format of the two most used for OS detection: TCP and IP.

IP header

The IP header format with description can be seen on Figure 1. Nmap during OS detection crafts the IP header and monitors the IP header in response with focus on Identification field (IP ID), Don't fragment bit in IP Flags and Type of Service.

Figure 1- IP header

TCP header

The TCP header format with description can be seen on Figure 2. Nmap during OS detection crafts the TCP header and monitors it in response, with focus on TCP flags, Window field and TCP options.

TCP options can be also referred to by their acronyms: SACK, for Selective ACK; MSS, for Maximum segment size; NOP, for No Operation; EOL, for End of Option List.

Figure 2 - TCP header

On the example in Figure 3, we have open TCP timestamp option (number 8) and from there we can see Timestamp TSval value 4294967295 (0xFFFFFFFF) and TSecr value 0 (0x00000000).

Figure 3 - WireShark output

Now if we know on which parts in our protocol headers we should focus on, we can move onto the explanation of which data or packets  Nmap is sending to detect the target OS. This recap will also help us to monitor the OS detection performed on our devices. Since we know where we should search the variables, it depends whether or not we have the traffic dump, and if we know for which values we are searching. 

What Nmap sends

Let’s see what makes the packets Nmap sends to probe the target  special.

First, Nmap sends six packets. In their TCP header, you can see the following parameters:

Packet #1: window scale (10), NOP, MSS (1460), timestamp (TSval: 0xFFFFFFFF; TSecr: 0), SACK permitted. The window field is 1.

Packet #2: MSS (1400), window scale (0), SACK permitted, timestamp (TSval: 0xFFFFFFFF; TSecr: 0), EOL. The window field is 63.

Packet #3: Timestamp (TSval: 0xFFFFFFFF; TSecr: 0), NOP, NOP, window scale (5), NOP, MSS (640). The window field is 4.

Packet #4: SACK permitted, Timestamp (TSval: 0xFFFFFFFF; TSecr: 0), window scale (10), EOL. The window field is 4.

Packet #5: MSS (536), SACK permitted, Timestamp (TSval: 0xFFFFFFFF; TSecr: 0), window scale (10), EOL. The window field is 16.

Packet #6: MSS (265), SACK permitted, Timestamp (TSval: 0xFFFFFFFF; TSecr: 0). The window field is 512.

From the Packet #1 to #6 following records are derived:

SEQ – Sequence analysis of the probe packet

OPS – TCP options received

WIN – window sizes for the responses of Packet #1-#6

T1 – various test values for packet #1

Then we can see the packet that helps to determine the Explicit Congestion Notification (ECN) values. For this reason, Nmap sends a packet with reserved bit and SYN, ECN, CWR flags set. The urgent field value of 0xF7F5 is used even though the urgent flag is not set. The acknowledgment number is zero, sequence number is random, window size field is three. TCP options are Window Scale (10), NOP, MSS (1460), SACK permitted, NOP, NOP. The probe is sent to an open port.

If we follow the same order as in Nmap fingerprint, we should describe packets corresponding to T2-T7 rows. The packets with corresponding TCP and IP header are:

• T2 TCP null (no flags set) and a window field of 128. IP DF bit is set. Packet is sent to an open port.
• T3 TCP packet with SYN, FIN, URG, PSH flags set and a window field of 256 to an open port.
• T4 TCP ACK packet and a window field of 1024. IP DF bit is set. Packet is sent to an open port.
• T5 TCP SYN packet and a window field of 31337. IP DF is not set. Packet is sent to a closed port.
• T6 TCP ACK packet and a window field of 32768. IP DF is set. Packet is sent to a closed port.
• T7 TCP packet with the FIN, PSH, URG flags set and a window field of 65535. Packet is sent to a closed port.

Now we can move to the U1 row in the Nmap fingerprint. This probe sends a UDP packet to a closed port. In the data field is character 'C' (0x43) repeated 300 times. The IP ID value is set to 0x1042 (4162) for operating systems which allow us to set this. If the port is truly closed and there is no firewall in place, Nmap expects to receive an ICMP port unreachable message in return. That response is then subjected to the R, DF, T, TG, IPL, UN, RIPL, RID, RIPCK, RUCK, and RUD tests that.

Last but not least, we have IE row. The IE test involves sending two ICMP echo request packets to the target. The first one has the IP DF bit set, a type-of-service (TOS) byte value of zero. ICMP type 8 and a code of nine (even though it should be zero or 16 based on RFC792, RFC2780), the ICMP sequence number 295, a random IP ID and 120 bytes of 0x00 as the data payload.

The second ping query is similar, except a TOS of four (IP_TOS_RELIABILITY) is used. The ICMP type is 8 and code is zero. 150 bytes of 0x00 data is sent, and the ICMP request ID and sequence numbers are incremented by one from the previous query values.

The results of both of these probes are combined into an IE line containing the R, DFI, T, TG, and CD tests. The R value is only true (Y) if both probes elicit responses. The T, and CD values are for the response to the first probe only, since they are highly unlikely to differ. DFI is a custom test for this special dual-probe ICMP case.

____________________________________________________________________

In the first part of our article, we have covered the OS detection with Nmap fingerprint examples, refresh our knowledge about TCP and IP headers and explained what Nmap sends. In the next sections we will cover how to decode Nmap fingerprint and talk about detection methods.