Monday, March 29, 2010

Blocking Traffic from Foreign Countries - Creating a block list of Supernets using PowerShell

The following PowerShell script will create a list of supernets that are outside of the United States. The networks created by this script are intended to be used to restrict network traffic from foreign countries. The results of this script aren't perfect and aren't intended to be perfect. There is trade off between the size of the list and accuracy, and I chose to err on the side of a shorter list of networks so it would not add and extra burden to the firewall.


Here is the script:


$debug = 0


# Filter for records that aren't in the US or run by ARIN
$records = ([xml]((New-Object System.Net.WebClient).DownloadString("http://www.iana.org/assignments/ipv4-address-space/ipv4-address-space.xml"))).registry.record | ? {
  $_.designation -notlike "*ARIN*" -and
  $_.status -ne "LEGACY"
}


# Create array for holding supernets
$supernets = @()


# Add a property for the Binary representation of the first octet
# Add a property for holding the masked bits, used for finding the supernets
$records | % { $_ |
  Add-Member NoteProperty -Name "Bits" -Value ([Convert]::ToString($_.prefix.Split("/")[0],2)).PadLeft(8,"0") -PassThru |
  Add-Member NoteProperty -Name "MaskedBits" -Value ""
}


# $i is the current number of mask bits used for finding a supernet
for ($i=1; $i -le 8; $i++) {


  # apply the mask, set the masked bits property
  # this get the left most $i of bits
  $records | % {
    $_.MaskedBits = $_.Bits.SubString(0,$i)
  }


  if ($debug) { $numrecords = $records.count }


  # how many /$i networks does it take to fill the current supernet, /1 is 128, /2 is 64 ...
  $supernetsize = [Math]::pow(2,8-$i)


  if ($debug) { "Supernet Size: $supernetsize" }


  # if a full supernet is found, then ...
  # a "full" supernet contains all of the /8 networks to fill the supernet
  $records | group MaskedBits | ? { $_.Count -eq $supernetsize} | % {
    $group = $_


    # create the supernet object and set the properties
    $supernet = "" | Select Prefix,Bits,MaskBits,CIDR
    $supernet.Bits = $group.Name.PadRight(8, "0")
    $supernet.Prefix = [Convert]::ToByte($supernet.Bits, 2)
    $supernet.MaskBits = $i
    $supernet.CIDR = "$($supernet.Prefix).0.0.0/$($supernet.MaskBits)"


    # add the supernet to the collection of supernets
    $supernets += $supernet


    # remove the networks from the full network collection if they were just added as a supernet
    $records = $records | ? { $_.Bits.SubString(0,$i) -ne $group.Name }
  }


  if ($debug) {
    "Matching Supernets Found: $(($numrecords - $records.count)/$supernetsize)"
    $supernets | ? { $_.MaskBits -eq $i }
    "---------------------------------------------------------------------------"
  }
}


#output the results
$supernets | Sort Bits | Select Prefix,Bits,MaskBits,CIDR


Now for a bit of explanation...


As you may already know, most IPv4 addresses are controlled by ARIN (North America), APNIC (Asia Pacific), LACNIC (Latin America and Caribbean), and other similar regional internet registries (RIR). Each one controls the ip addresses for a specific portion of the world. We want to create a list of all networks that are not in the United States, and the closest approximation is the ARIN RIR. The problem is, prior to the establishment of the RIRs, some blocks of IP addresses were handed out directly to organizations (and their status is "legacy"). Most of the companies that have the legacy address spaces are US based and we will assume they are allowed.


First, we get the list of networks from IANA (Internet Assigned Numbers Authority). Our black list will contain all of the networks NOT in the US, so we filter out all of the ARIN controlled IP addresses and the addresses with a "legacy" status. The list is imported in XML format. The nice feature of the xml format is that imports the xml entities as objects, and PowerShell works best with objects. The properties of the objects are prefix, designation, date, status and xref, but we only use the status and the prefix. With these properties defined we can easily use the Where-Object cmdlet (alias ?) to filter out the "safe" address. If you wanted to block a different set of RIRs this is where you could make one simple change to better suit you.


$records = ([xml]((New-Object System.Net.WebClient).DownloadString("http://www.iana.org/assignments/ipv4-address-space/ipv4-address-space.xml"))).registry.record | ? {
  $_.designation -notlike "*ARIN*" -and
  $_.status -ne "LEGACY"
}


After the import and the filter, the $records variable contains all of the addresses blocks we want to blacklist. We could just quit now, but it would be nice to shorten the list. We can combine the address blocks into supernets. We start by creating a variable to hold the array of supernets.


# Create array for holding supernets
$supernets = @()


To make things easier, we can extend the object given to us from xml. Two properties are added, one to hold the bit representation of the first octet, and the second will be used for storing a masked version of those bits.


# Add a property for the Binary representation of the first octet
# Add a property for holding the masked bits, used for finding the supernets
$records | % { $_ |
  Add-Member NoteProperty -Name "Bits" -Value ([Convert]::ToString($_.prefix.Split("/")[0],2)).PadLeft(8,"0") -PassThru |
  Add-Member NoteProperty -Name "MaskedBits" -Value ""
}


The $records variable (as previously stated, contains all the networks to be blocked) is piped into the ForEach-Object cmdlet (alias %). Inside the loop we create the properties and set the Bits property. The initial value for the Masked Bits is blank since we will set that later.


Now that the object is created the way we want it, we start a For loop. The loop will go from 1 to 8, representing each bit in the octet, and it will be used for masking.


# $i is the current number of mask bits used for finding a supernet
for ($i=1; $i -le 8; $i++) {


Now to use our newly created property and set the Mask Bits on each object.


  # apply the mask, set the masked bits property
  # this get the left most $i of bits
  $records | % {
    $_.MaskedBits = $_.Bits.SubString(0,$i)
  }


The $records variable is piped into a ForEach-Object loop. Inside the loop we apply the mask. The mask takes the left most bits. When $i is one, we only look at the leftmost bit. When $i is 2, we take the two leftmost bits, and so on.


Now we need to calculate how many networks it takes to "fill" a supernet with our mask. A mask of 1 will require 128 networks to be full, a mask of 2 will require 64 networks, 3 requires 32, and so on.


  # how many /$i networks does it take to fill the current supernet, /1 is 128, /2 is 64 ...
  $supernetsize = [Math]::pow(2,8-$i)


Now we have the supernet size, so let's see if we have any full supernets.


  # if a full supernet is found, then ...
  # a "full" supernet contains all of the /8 networks to fill the supernet
  $records | group MaskedBits | ? { $_.Count -eq $supernetsize} | % {


We pipe the $records variable into Group-Object, where the grouping is done in the Masked Bits. All of the networks with matching Mask Bits will be put in a group. We then filter all groups that are full by using the Where-Object cmdlet to filter groups that have the required number of elements. If any make it through the filter they are piped into the ForEach-Object cmdlet, where we create the supernet.


    $group = $_


    # create the supernet object and set the properties
    $supernet = "" | Select Prefix,Bits,MaskBits,CIDR
    $supernet.Bits = $group.Name.PadRight(8, "0")
    $supernet.Prefix = [Convert]::ToByte($supernet.Bits, 2)
    $supernet.MaskBits = $i
    $supernet.CIDR = "$($supernet.Prefix).0.0.0/$($supernet.MaskBits)"


    # add the supernet to the collection of supernets
    $supernets += $supernet


    # remove the networks from the full network collection if they were just added as a supernet
    $records = $records | ? { $_.Bits.SubString(0,$i) -ne $group.Name }
  }
}


First we set a the $group variable equal to the current group, $_ represents the group passed into the ForEach-Object cmdlet. Next, we create a variable to hold the supernet and set the properties of the supernet. The Bits property is the masked bits, the Prefix is the decimal equivalent of the masked bits, the Mask Bits is the number of bits used in the mask, and CIDR is just a pretty version of the Prefix and the Mask Bits. For example, the fourth pass through the For loop uses four bits for the mask. It finds a full supernet matching the four leftmost bits 0101. The supernet Bits are 01010000, Prefix is 80, Masked Bits is 4, and the CIDR is 80.0.0.0/4.


The supernet is added to the collection of supernets. We then need to remove the networks in the supernet from the $records variable so we don't use them again. This is done by filtering the $records variable for all networks that match our masked bits. In the case above, we would remove all networks that start with 0101xxxx.


The loop then starts over with a slightly bigger mask which looks from smaller supernets.


All we have left to do is output the results.


#output the results
$supernets | Sort Bits | Select Prefix,Bits,MaskBits,CIDR


Results:
Prefix Bits     MaskBits CIDR
------ ----     -------- ----
     0 00000000        7 0.0.0.0/7
     2 00000010        8 2.0.0.0/8
     5 00000101        8 5.0.0.0/8
    10 00001010        8 10.0.0.0/8
    14 00001110        8 14.0.0.0/8
    23 00010111        8 23.0.0.0/8
    27 00011011        8 27.0.0.0/8
    31 00011111        8 31.0.0.0/8
    36 00100100        7 36.0.0.0/7
    39 00100111        8 39.0.0.0/8
    41 00101001        8 41.0.0.0/8
    42 00101010        8 42.0.0.0/8
    46 00101110        8 46.0.0.0/8
    49 00110001        8 49.0.0.0/8
    58 00111010        7 58.0.0.0/7
    60 00111100        7 60.0.0.0/7
    62 00111110        8 62.0.0.0/8
    77 01001101        8 77.0.0.0/8
    78 01001110        7 78.0.0.0/7
    80 01010000        4 80.0.0.0/4
   100 01100100        6 100.0.0.0/6
   104 01101000        7 104.0.0.0/7
   106 01101010        8 106.0.0.0/8
   109 01101101        8 109.0.0.0/8
   110 01101110        7 110.0.0.0/7
   112 01110000        4 112.0.0.0/4
   175 10101111        8 175.0.0.0/8
   176 10110000        5 176.0.0.0/5
   185 10111001        8 185.0.0.0/8
   186 10111010        7 186.0.0.0/7
   189 10111101        8 189.0.0.0/8
   190 10111110        8 190.0.0.0/8
   193 11000001        8 193.0.0.0/8
   194 11000010        7 194.0.0.0/7
   197 11000101        8 197.0.0.0/8
   200 11001000        6 200.0.0.0/6
   210 11010010        7 210.0.0.0/7
   212 11010100        7 212.0.0.0/7
   217 11011001        8 217.0.0.0/8
   218 11011010        7 218.0.0.0/7
   220 11011100        6 220.0.0.0/6
   224 11100000        3 224.0.0.0/3


I hope that helps.

No comments:

Post a Comment