Network scanning, OS fingerprinting part 2

How to decode the NMAP fingerprint

In the first post of the series, we explored what you can find in NMAP fingerprint database. We only covered Class and CPE line. So, let's continue with the most interesting part. Take the below sample and try to decode the values based on the description:

SEQ(SP=F6-100%GCD=1-6%ISR=10B-115%TI=I%CI=I%II=I%SS=S%TS=7)

OPS(O1=M523NW8ST11%O2=M523NW8ST11%O3=M523NW8NNT11%O4=M523NW8ST11%O5=M523NW8ST11%O6=M523ST11)

WIN(W1=2000%W2=2000%W3=2000%W4=2000%W5=2000%W6=2000)

ECN(R=Y%DF=Y%T=7B-85%TG=80%W=2000%O=M523NW8NNS%CC=N%Q=)

T1(R=Y%DF=Y%T=7B-85%TG=80%S=O%A=S+%F=AS%RD=0%Q=)

T2(R=Y%DF=Y%T=7B-85%TG=80%W=0%S=Z%A=S%F=AR%O=%RD=0%Q=)

T3(R=Y%DF=Y%T=7B-85%TG=80%W=0%S=Z%A=O%F=AR%O=%RD=0%Q=)

T4(R=Y%DF=Y%T=7B-85%TG=80%W=0%S=A%A=O%F=R%O=%RD=0%Q=)

T5(R=Y%DF=Y%T=7B-85%TG=80%W=0%S=Z%A=S+%F=AR%O=%RD=0%Q=)

T6(R=Y%DF=Y%T=7B-85%TG=80%W=0%S=A%A=O%F=R%O=%RD=0%Q=)

T7(R=Y%DF=Y%T=7B-85%TG=80%W=0%S=Z%A=S+%F=AR%O=%RD=0%Q=)

U1(DF=N%T=7B-85%TG=80%IPL=164%UN=0%RIPL=G%RID=G%RIPCK=G%RUCK=G%RUD=G)

IE(DFI=N%T=7B-85%TG=80%CD=Z)

SEQ field values

TCP ISN sequence predictability index (SP)

This value measures the ISN (initial sequence number) variability. It roughly estimates how difficult it would be to predict the next ISN from the known sequence of the first six probe responses. This test is only performed if at least four responses were seen.

The calculation uses the difference array (seq_rates) and GCD values discussed in later sections. If you are interested in detailed description how this field is calculated, please see https://nmap.org/book.

Please keep in mind that this test is only done for OS detection purposes and is not a full-blown audit of the target ISN generator. There are many algorithm weaknesses that lead to easy predictability even with a high SP value.

TCP ISN greatest common divisor (GCD)

The SEQ test sends six TCP SYN packets to an open port of the target machine and collects SYN/ACK packets back. Each of these SYN/ACK packets contains a 32-bit initial sequence number (ISN). This test attempts to determine the smallest number by which the target host increments these values. For example, many hosts (especially old ones) always increment the ISN in multiples of 64,000.

The first step in calculating this is creating an array of differences between probe responses. The first element is the difference between the 1st and 2nd probe response ISNs. The second element is the difference between the 2nd and 3rd responses. There are five elements if NMAP receives responses to all six probes. Since the next couple of sections reference this array, we will call it diff1. If an ISN is lower than the previous one, NMAP looks at both numbers and decide if they need to be subtracted or counted up (including wrapping the 32-bit counter back to zero). The smaller of those two values is stored in diff1. So the difference between 0x20000 followed by 0x15000 is 0xB000. The difference between 0xFFFFFF00 and 0xC000 is 0xC0FF. This test value then records the greatest common divisor of all those elements. This GCD is also used for calculating the SP result.

TCP ISN counter rate (ISR)

This value reports the average rate of increase for the returned TCP initial sequence number. Recall that a difference is taken between each two consecutive probe responses and stored in the previously discussed diff1 array. Those differences are each divided by the amount of time elapsed (in seconds—will generally be about 0.1) between sending the two probes which generated them. The result is an array, which we'll call seq_rates containing the rates of ISN counter increases per second. The array has one element for each diff1 value. An average is taken of the array values. If that average is less than one (e.g. a constant ISN is used), ISR is zero. Otherwise, ISR is eight times the binary logarithm (log base-2) of that average value, rounded to the nearest integer.

IP ID sequence generation algorithm (TI, CI, II)

There are three tests that examine the IP header ID field of responses. TI is based on responses to the TCP SEQ probes. CI is from the responses to the three TCP probes sent to a closed port: T5, T6, and T7. II comes from the ICMP responses to the two IE ping probes. For TI, at least three responses must be received for the test to be included; for CI, at least two responses are required; and for II, both ICMP responses must be received.

For each of these tests, the target's IP ID generation algorithm is classified based on the algorithm below. Minor differences between tests are noted. Note that difference values assume that the counter can start over. In that case the difference between an IP ID of 65,100 followed by a value of 700 is 1,136. The difference between 2,000 followed by 1,100 is 64,636. Here are the calculation details:

If all of the ID numbers are zero, the value of the test is Z.

If the IP ID sequence ever increases by at least 20,000, the value is RD (random). This result isn't possible for II because there are not enough samples to support it.

If all of the IP IDs are identical, the test is set to that value in hex.

If any of the differences between two consecutive IDs exceeds 1,000, and is not evenly divisible by 256, the test's value is RI (random positive increments). If the difference is evenly divisible by 256, it must be at least 256,000 to cause this RI result.

If all of the differences are divisible by 256 and no greater than 5,120, the test is set to BI (broken increment). This happens on systems like Microsoft Windows where the IP ID is sent in host byte order (little endian) rather than network byte order (big endian). It works fine and isn't any sort of RFC violation, though it does give away host architecture details which can be useful to attackers.

If all of the differences are less than ten, the value is I (incremental). Nmap allows difference up to ten (rather than requiring sequential ordering) because traffic from other hosts can cause sequence gaps.

If none of the previous steps identify the generation algorithm, the test is omitted from the fingerprint.

Shared IP ID sequence Boolean (SS)

This Boolean value records whether the target shares its IP ID sequence between the TCP and ICMP protocols. If our six TCP IP ID values are 117, 118, 119, 120, 121, and 122, then our ICMP results are 123 and 124, it is clear that not only are both sequences incremental, but they are both part of the same sequence. If, on the other hand, the TCP IP ID values are 117–122 but the ICMP values are 32,917 and 32,918, two different sequences are being used.

This test is only included if II is RI, BI, or I and TI is the same. If SS is included, the result is S if the sequence is shared and O (other) if it is not.

If you are interested in detailed description how this test is calculated, please see https://nmap.org/book.

TCP timestamp option algorithm (TS)

TS is another test which attempts to determine target OS characteristics based on how it generates a series of numbers. This one looks at the TCP timestamp option (if any) in responses to the SEQ probes. It examines the TSval (first four bytes of the option) rather than the echoed TSecr (last four bytes) value. It takes the difference between each consecutive TSval and divides that by the amount of time elapsed between NMAP sending the two probes which generated those responses. The resultant value gives a rate of timestamp increments per second. NMAP computes the average increments per second from all consecutive probes and then calculates the TS as follows:

1. If any of the responses have no timestamp option, TS is set to U (unsupported).
2. If any of the timestamp values are zero, TS is set to 0.
3. If the average increments per second falls within the ranges 0-5.66, 70-150, or 150-350, TS is set to 1, 7, or 8, respectively. These three ranges get special treatment because they correspond to the 2 Hz, 100 Hz, and 200 Hz frequencies used by many hosts.
4. In all other cases, Nmap records the binary logarithm of the average increments per second, rounded to the nearest integer. Since most hosts use 1,000 Hz frequencies, A is a common result.

OPS field values

TCP options (O, O1–O6)

Some platforms don't implement all options (which are, of course, optional).

Options test values

Option Name	Character	Argument (if any)
End of Options List (EOL)	L
No operation (NOP)	N
Maximum Segment Size (MSS)	M	The value is appended. Many systems echo the value used in the corresponding probe.
Window Scale (WS)	W	The actual value is appended.
Timestamp (TS)	T	The T is followed by two binary characters representing the TSval and TSecr values respectively. 0 if the field is zero and 1 otherwise.
Selective ACK (SACK)	S

As an example, the string M5B4NW3NNT11 means the packet includes the MSS option (value 0x5B4) followed by a NOP. Next comes a window scale option with a value of three, then two more NOPs. The final option is a timestamp, and neither of its two fields (timestamp value, timestamp echo reply) were zero. If there are no TCP options in a response, the test will exist, but the value string will be empty. If no probe was returned, the test is omitted.

WIN field values

Window length of responses for first 6 packets (Packet #1-#6). It is common to see the same value for all six responses.

ECN, T1 – T7, IE field values

Responsiveness (R)

This test simply records whether the target responded to a given probe. Possible values are Y and N. If there is no reply.

IP don't fragment bit (DF)

The IP header contains a single bit which forbids routers from fragmenting a packet. This test records Y if the bit is set, and N if it isn't.

Don't fragment (ICMP) (DFI)

This is simply a modified version of the DF test that is used for the special IE probes. It compares results of the don't fragment bit for the two ICMP echo request probes sent.

DFI test values

Value	Description
N	Neither of the ping responses have the DF bit set.
S	Both responses echo the DF value of the probe.
Y	Both of the response DF bits are set.
O	The one remaining other combination—both responses have the DF bit toggled.

IP initial time-to-live (T)

IP packets contain a field named time-to-live (TTL) which is decremented every time they traverse a router. If the field reaches zero, the packet must be discarded. This prevents packets from looping endlessly. Because operating systems differ on which TTL they start with, it can be used for OS detection. Nmap determines how many hops away it is from the target by examining the ICMP port unreachable response to the U1 probe. That response includes the original IP packet, including the already-decremented TTL field, received by the target. By subtracting that value from our as-sent TTL, we learn how many hops away the machine is. Nmap then adds that hop distance to the probe response TTL to determine what the initial TTL was when that ICMP probe response packet was sent. That initial TTL value is stored in the fingerprint as the T result. Nmap accepts the value within the range specified in fingerprint (for example T=7B-85). In most cases, the expected value will be 0x80 (for our 7B-85 range), but for cases when there are multiple asymmetric routes to the target the calculated value can vary.

IP initial time-to-live guess (TG)

It is not uncommon for Nmap to receive no response to the U1 probe, which prevents Nmap from learning how many hops away a target is. Firewalls and NAT devices love to block unsolicited UDP packets. But since common TTL values are spread well apart and targets are rarely more than 20 hops away, Nmap can make a pretty good guess anyway. Most systems send packets with an initial TTL of 32, 60, 64, 128, or 255. So, the TTL value received in the response is rounded up to the next value out of 32, 64, 128, or 255. Sixty is not in that list because it cannot be reliably distinguished from 64. It is rarely seen anyway. The resulting guess is stored in the TG field. This TTL guess field is not printed in a subject fingerprint if the actual TTL (T) value was discovered.

Explicit congestion notification (CC)

This test is only used for the ECN probe. That probe is a SYN packet which includes the CWR and ECE congestion control flags. When the response SYN/ACK is received, those flags are examined to set the CC (congestion control) test value

CC test values

Value	Description
Y	Only the ECE bit is set (not CWR). This host supports ECN.
N	Neither of these two bits is set. The target does not support ECN.
S	Both bits are set. The target does not support ECN, but it echoes back what it thinks is a reserved bit.
O	The one remaining combination of these two bits (other).

TCP miscellaneous quirks (Q)

This tests for two quirks that a few implementations have in their TCP stack. The first is that the reserved field in the TCP header (right after the header length) is nonzero. This is particularly likely to happen in response to the ECN test as that one sets a reserved bit in the probe. If this is seen in a packet, an "R" is recorded in the Q string.

The other quirk Nmap tests for is a nonzero urgent pointer field value when the URG flag is not set. This is also particularly likely to be seen in response to the ECN probe, which sets a non-zero urgent field. A "U" is appended to the Q string when this is seen.

The Q string must always be generated in alphabetical order. If no quirks are present, the Q test is empty but still shown.

TCP Sequence (S) and Acknowledgement (A)

This test compares the TCP sequence and acknowledgment number from the probe that elicited the response. It then records the appropriate value as shown in the table below.

S and A test values

Value	Description
Z	number is zero.
A	Sequence number is the same as the acknowledgment number in the probe.
A+	Sequence number is the same as the acknowledgment number in the probe plus one.
S	Acknowledgement value is the same as the sequence number in the probe
S+	Acknowledgement value is the same as the sequence number in the probe plus one.
O	number is something else (other).

TCP flags (F)

This field records the TCP flags in the response. Each letter represents one flag, and they occur in the same order as in a TCP packet (from high-bit on the left, to the low ones). The value AS represents the ACK and SYN bits set, while the value SA is illegal (wrong order).

F test values

Character	Flag name	Flag byte value
E	ECN Echo (ECE)	64
U	Urgent Data (URG)	32
A	Acknowledgment (ACK)	16
P	Push (PSH)	8
R	Reset (RST)	4
S	Synchronize (SYN)	2
F	Final (FIN)	1

TCP RST data checksum (RD)

Some operating systems return ASCII data such as error messages in reset packets. When Nmap encounters such data, it performs a CRC32 checksum and reports the results. When there is no data, RD is set to zero. Some of the few operating systems that may return data in their reset packets are HP-UX and versions of Mac OS prior to Mac OS X.

IP total length (IPL)

This test records the total length (in octets) of an IP packet. It is only used for the port unreachable response elicited by the U1 test.

Unused port unreachable field nonzero (UN)

An ICMP port unreachable message header is eight bytes long, but only the first four are used. RFC 792 states that the last four bytes must be zero. A few implementations (mostly ethernet switches and some specialized embedded devices) set it anyway. The value of those last four bytes is recorded in this field.

UDP (U1) related parameters

Returned probe IP total length value (RIPL)

ICMP port unreachable messages (that are sent in response to the U1 probe) are required to include the IP header which generated them. This header should be returned just as they received it, but some implementations send back a corrupted version due to changes they made during IP processing. This test simply records the returned IP total length value. If the correct value of 0x148 (328) is returned, the value G (for good) is stored instead of the actual value.

Returned probe IP ID value (RID)

The U1 probe has a static IP ID value of 0x1042. If that value is returned in the port unreachable message, the value G is stored for this test. Otherwise, the exact value returned is stored. Some systems, such as Solaris, manipulate IP ID values for raw IP packets that Nmap sends. In such cases, this test is skipped

Integrity of returned probe IP checksum value (RIPCK)

The IP checksum is one value that we don't expect to remain the same when returned in a port unreachable message. After all, each network hop during transit changes the checksum as the TTL is decremented. However, the checksum we receive should match the enclosing IP packet. If it does, the value G (good) is stored for this test. If the returned value is zero, then Z is stored. Otherwise, the result is I (invalid).

Integrity of returned probe UDP checksum (RUCK)

The UDP header checksum value should be returned exactly as it was sent. If it is, G is recorded for this test. Otherwise, the value actually returned is recorded.

Integrity of returned UDP data (RUD)

This test checks the integrity of the (possibly truncated) returned UDP payload. If all the payload bytes are the expected 'C' (0x43), or if the payload was truncated to zero length, G is recorded; otherwise, I (invalid) is recorded.

ICMP response code (CD)

The code value of an ICMP echo reply (type zero) packet is supposed to be zero. But some implementations wrongly send other values, particularly if the echo request has a nonzero code (as one of the IE tests does).

CD test values

Value	Description
Z	Both code values are zero.
S	Both code values are the same as in the corresponding probe.
<NN>	When they both use the same non-zero number, it is shown here.
O	Any other combination.

__________________________________________________________________________

In first two parts we have covered the mechanisms around Nmap OS detection and networking protocol basics. In our last part we will be talking about how to monitor those activities. We will discuss pros and cons of different detection methods and as usual you can look forward for a few examples that can be applied on the most common tools used.