IPTables and conntrack

When you are using LANforge as a virtual router doing NAT, you might need to see how many NAT table entries you’re handling.  This can be important because NAT entries take memory, and if you want to handle 65,000 simultaneous connections, you might be heading for trouble.

If your LANforge is only generating traffic, you won’t see NAT entries…rather you want to use netstat -ntp to see how many open connections there are.

LANforge uses iptables PREROUTING heavily, forcing each port to have it’s own set of tables. When you type iptables -nvL and see nothing…that’s because nothing is in the tables for your default route, which is probably eth0. You get close with the raw table. Try iptables -S -t raw. You will see PREROUTING entries for every interface:

# iptables -S -t raw
-A PREROUTING -i br2000 -j CT --zone 10001
-A PREROUTING -i eth1 -j CT --zone 10001
-A PREROUTING -i vap13 -j CT --zone 10001
-A PREROUTING -i vap14 -j CT --zone 10001
-A PREROUTING -i eth2 -j CT --zone 10001

This shows we have a CT chain and a zone note for that chain.

When you create a virtual router, add NAT to a port in it, you can view the NAT table entries with conntrack.

* conntrack -L will list them all, but that’s probably not super useful
If you’re running TCP-multicon connections, expect thousands of connections.
* conntrack -C will show how many NAT entries are present, so you can avoid doing a conntrack -L | wc -l

Happy networking!

Java on OS X

Sometimes your Mac has a very old version of Java on it, like 1.6.0_123. This can break running the LANforge client. Let’s learn how to manage this. First, open Terminal and find what version of Java you think you have:

$ java -version

This reports what your JAVA_HOME environment variable (which might not be set) is, so the System will do a /usr/libexec/java_home command for you. You might need to follow these instructions on removing old Java version.

Install a recent version of Java. Then go back to the terminal. You want to make sure that your JAVA_HOME variable is getting set to the version you want. Verify you have a .profile and a .bashrc file in your home directory:

$ ls -l ~/.profile ~/.bashrc

Create them if the don’t exist:

$ touch ~/.profile ~/.bashrc
$ chmod 700 ~/.profile ~/.bashrc
$ echo "source .bashrc" >> ~/.profile

To see the versions of Java installed, use the java_home command:

$ /usr/libexec/java_home -V

The first column of output will be the version number, which is important for running the java_home command. Edit your .bashrc file to export your JAVA_HOME environment variable:

export JAVA_HOME=$(/usr/libexec/java_home -v 1.8.0_122)

Save and open a new terminal. Verify using java -version.

More discussion on setting the version:

Setting up Xming

You can display Wireshark from your LANforge system onto your Windows desktop. You need to install some kind of X client software, and Xming does a fine job. However, X comes with some access controls. On a Linux system, you disable access controls with the command xhost +. This allows any host to display a window on your desktop.

Xming has a similar setting. After you install Xming and start it. you want to use the XLaunch utility it comes with to configure settings. You have to get to that from the Start Menu.

2018-04-06 15_19_16-Greenshot (2)
Use the Start Menu Applications item
2018-04-06 15_17_15-Start
Search the Start Menu for XLaunch

The XLaunch program allows you to configure multiple ways to displaying X clients. Display 0 (zero) is fine.

2018-04-06 15_36_27-Display settings

You do not need to start a client program on Xming start.

2018-04-06 15_36_39-Session type

You do want to enable clipboard. Select No Access Control.

2018-04-06 15_36_50-Additional parameters

Then go ahead and Save on the next screen, save the configuration to your Documents folder.

If there is a running session of Xming, exit it.

2018-04-06 15_15_26-Action center

Double-click on the config.xlaunch you created.

2018-04-06 15_45_47-Action center

Click on Sniff Packets in LANforge and you should see Wireshark start up.

2018-04-06 15_47_34-StartB

Notice that the IP of your desktop should be displayed right next to the Sniff Packets button. You can type in the exact IP and X display number you need.

Redirecting Web Browsing Using Squid and DNSMasq

When testing a captive portal with a virtual station on a LANforge machine, you are faced with a problem of multiple gateways. When the station associates, it gains a DHCP address with a new gateway and DNS server. Your LANforge machine will already have a default gateway and DNS server assigned to it throught the management interface it has on boot. You need to use both. On top of that, you are connected via screen-sharing to a laptop and not to the LANforge machine directly.

When You Have a Serial Console

If you’re lucky, you have a working serial console connection between the screen-share laptop and the LANforge system. I mention working because sometimes the quality of USB serial adapters is spotty. When it works, you have a way to set new values into /etc/resolv.conf and a default gateway.

Getting the station information via the command-line is something you can do using the lf_portmod.pl script:

/home/lanforge/scripts $ ./lf_portmod.pl --manager localhost \
 --card 1  --quiet yes \
 --port_name wlan0  --show_port
 >>RSLT: 0 Cmd: 'nc_show_port' '1' '1' 'wlan0'

Shelf: 1, Card: 1, Port: 3 Type: STA Alias: 
 IP: MASK: GW: VID: 0 ResetState: COMPLETE
 DNS Servers:
 IPv6-Global: DELETED
 IPv6-Link: fe80::a00:27ff:fe0a:973d/64
 IPv6-Gateway: DELETED
 MAC: 08:00:27:0a:97:3d DEV: wlan0 MTU: 1500 TX Queue Len: 1000

Note down your existing route information:

$ route -n > existing_route.txt

Set your route to the laptop so you don’t loose it. Then update your new route:

$ sudo route add -host $laptop-ip dev eth0
$ sudo route set default gw

Next, re-order the entries in your /etc/resolv.conf file:

# nameserver:

And that should allow your browser to work well enough. If you fat-finger the routes, you can renew the DHCP address on your management port (eth0) using the ifup/ifdown commands:

$ sudo ifdown eth0; sudo ip a flush eth0; sudo ifup eth0

And that should restore your management connection to what you had during boot up.

Often–No Serial Console

The problem is, everything your browser will want to do will use your default gateway, not the virtual stations. Changing the default gateway will drop your connection to the LANforge machine and you will be lost from it. When you are developing remotely, you can waste days waiting for your remote counterpart doing hands-on to reboot the LANforge when you lose the network settings. Luckily, LANforge boxes come with squid and dnsmasq installed.

Starting with dnsmasq

We can start by setting up dnsmasq, so that we can direct DNS requests through the stations DNS setting.

  1. Update /etc/resolv.conf to point at localhost:
    # old nameserver:
  2. Update the /etc/dnsmasq.conf to query the station’s DNS server:

Then start dnsmasq:

$ sudo systemctl start dnsmasq

Tail your system logs to check for typos:

$ journalctl -f

Test a DNS query:

$ host www.cnn.com
Using domain server:

www.cnn.com is an alias for turner-tls.map.fastly.net.
turner-tls.map.fastly.net has address
turner-tls.map.fastly.net has address
turner-tls.map.fastly.net has address
turner-tls.map.fastly.net has address
turner-tls.map.fastly.net has IPv6 address 2a04:4e42::323
turner-tls.map.fastly.net has IPv6 address 2a04:4e42:200::323
turner-tls.map.fastly.net has IPv6 address 2a04:4e42:400::323
turner-tls.map.fastly.net has IPv6 address 2a04:4e42:600::323

Do not enable dnsmasq using $ sudo systemctl enable dnsmasq; sudo systemctl daemon-reload. That will take effect after a reboot and you might not remember doing it. Please remember to comment out in /etc/resolv.conf.

Move onto squid

We want to set the tcp_outgoing_address setting in our /etc/squid/squid.conf file.


Start squid:

$ sudo systemctl start squid

Open a terminal and tail squid logs:

$ cd /var/log/squid; sudo tail -F *log

Next configure FireFox

The Network settings in FireFox are under Edit→Preferences and Advanced→Network. Edit the HTTP proxy to be and port 3128. Erase the values for No Proxy for because having and localhost in there will force FireFox to make no connections whatsoever.

Screenshot at 2017-11-10 13:35:08

Verify with tcpdump

Now the fun begins! We want to see web traffic and DNS queries flowing out the station interface and NOT the management interface. Open two windows, for two different tcpdump sessions. Monitor the management port:

$ sudo tcpdump -eni eth0 port 80 or port 53 or port 443

And monitor the station:

$ sudo tcpdump -eni wlan0 port 80 or port 53 or port 443

Using curl

to watch for redirects

Capturing a captive portal redirect is usually a matter of sending any HTTP request over the station and the response will be a 302 (or 307) redirect.

$ source ~/lanforge.profile # sets path
$ which curl # should be /home/lanforce/local/bin/curl
$ curl -svLki -4 
   --interface wlan0 \
   --localaddr \
   --dns-interface wlan0 \
   --dns-ipv4-addr \
   --dns-servers \

Use the portal-check.pl script

In the /home/lanforge/scripts directory, you might have a portal-check.pl script. This script performs most of the outlines steps discussed so far. It is useful for helping validate the environment and gathering data to use with portal-bot.bash.

A response looks like:

< Location: http://portal.example.com:8080/
< Content-Type: text/html; charset=UTF-8
< Date: Fri, 10 Nov 2017 22:05:01 GMT
< Expires: Sun, 10 Dec 2017 22:05:01 GMT
< Cache-Control: public, max-age=2592000
* Server gws is not blacklisted
< Server: gws
< Content-Length: 219
< X-XSS-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<H1>301 Moved</H1>
The document has moved
<A HREF="http://portal.example.com:8080/">here</A>.


When diagnosing behavior of captive portals, often they are constructed to redirect to arbitrary high-level ports. Feel free to add more ports to your tcpdump filter. But at this point, curl is probably going to be your best friend.

Temporarily Changing the Default Gateway to a Virtual Station

When testing a virtual station and you need to verify with a browser that you can reach a captive portal page, or browse the Internet–you want to change the default gateway of the system to the gateway that the virtual station was assigned. This is a manual change, because when the virtual station associates, the default route of the system doesn’t change. If it did, you’d probably lose connection between the LANforge resource and your management station would probably be lost.

After checking your associated station, you can write a script to help set the new default gateway. Here’s an example with some fixed values:

echo "Please do this command in screen"
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - #
# Use this script to modify your routes to allow a station      #
# to become the default route. Should maintain route for your   #
# existing connection to management laptop through eth0.        #
# This script is an example to be modified.                     #
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - #
set -x
ip a show eth0
ip a show sta100
ip route show

# this adds address to station, referred to below
# ip a add dev sta100

# make static route to old gateway
ip route add dev eth0

# make static route to mgt laptop
ip route add dev eth0

# now update default route to go out sta100
ip route change default via dev sta100

sleep 1s
ping -nc2
ping -nc2

# adjust this to your work time window
read -t $((2 * 60)) -p "Press Enter to end and reset"

# and set the route back
ip route change default via dev eth0
ip route show
ping -nc2
ping -nc2

Notice that the script suggests using screen.  Why use screen? Because if your terminal session disconnects, the script will continue and reset the previous default gateway. If the script is interrupted (by session disconnection or control-c) your default route will remain running across your AP. (We add default link-local routes to our laptop to help mitigate this.)
There is a default .screenrc file in /home/lanforge. You want to run the script as root. This will work: sudo ./change_gw.sh

This is an example script with some logic to help detect the default gateway, your ssh connection, and you tell it what station to use:

# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - #
#  Use this script to modify your routes to allow a station     #
#  to become the default route. Should maintain route for your  #
#  existing connection to management laptop through eth0.       #
#  This script is an example to be modified.                    #
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - #

if [ -z "$3" ]; then
   echo "Use this script to assign a station on this system to be the new default gateway."
   echo "Usage: $0   "
   echo ""
   echo "Your manager-ip is the LANforge resource in mode 'manager' or 'both' and might"
   echo "not be the machine hosting the virtual station."
   echo "Example, sta3000 on resource 3, manager is"
   echo ""
   echo "./$0 3 sta3000"
   echo ""
   echo "Please do this command in screen"
   exit 1

def_gw_dev=`route -n | awk '/^ / {print \$NF}'`
def_gw_ip=`ip a show $def_gw_dev | awk '/inet /{print \$2}'`
echo "default gw device: $def_gw_dev $def_gw_ip "
sleep 3
manager_ip=`who | perl -ne '/\(([0-9.]+)\)$/ && print "$1\n";'`
#ip a show $def_gw_dev
echo "This is the station you requested:"
ip a show $3
sleep 3

#  make static route to old gateway
echo "ip route add $def_gw_ip dev $def_gw_dev"
ip route add $def_gw_ip dev $def_gw_dev

#  make static route to mgt laptop
echo "ip route add $manager_ip dev $def_gw_dev"
ip route add $manager_ip dev $def_gw_dev
sleep 3

# now update default route to go out sta100
cd /home/lanforge/scripts
sta_gw=`./lf_portmod.pl --manager $1 --card $2 --port_name $3 \
  --show_port --quiet yes \
  | awk '/ IP:/{print \$6}'`

echo "* Changing default gateway in 3 seconds *"
sleep 3s
echo "ip route change default via $sta_gw dev $3"
sleep 1s
ip route change default via $sta_gw dev $3
sleep 1s
ping -nc2 $def_gw_ip
ping -nc2 $manager_ip

# adjust this to your work time window (in seconds)
#  $(( 2 * 60)) == 120, 2 minutes
read -t $((2 * 60)) -p "Press Enter to end and reset"

# and set the route back
echo "changing route back to previous default gw..."
ip route change default via $def_gw_ip dev $def_gw_dev
ping -nc2 $def_gw_ip
ping -nc2 $manager_ip
echo "...done."

To use the script, you would say:

sudo ./change_gw.sh 3 sta3000

You are probably going to need to alter the script. If you hit Ctlc in the script you will need to reset the routes by hand.

For actually using Firefox, you need to operate this from the desktop of LANforge resource hosting the virtual station. So if, in this example, sta3000 is on resource 3, you need to get on the virtual desktop of resource 3. Open a terminal from the virtual desktop or ssh to resource 3 and start screen as user LANforge.

Modifying the Port Bringup Report

The LANforge Port Bringup plugin produces a highly detailed and verbose report including all the wifi events from the stations involved. This plugin can be used to record performance of station association and captive portal logins. When you save the Graphical Report screen, it saves an HTML document with pictures to a directory on your machine.




We see the report files created by saving the report below:


This report can be reformatted, and the log messages at the bottom of the output are a useful resource to recreate graphs of interest. With some accessory JavaScript, we can create a completely different report. I start off by writing a shell script that backs up the original index.html file of the report and concatenates it to the end of a new title block.

Example shell script below:
(please excuse my lazy [] brackets, because  &-lt; &-gt; takes longer to type)

if [ ! -f orig.html ]; then
   mv -v index.html orig.html
   dos2unix orig.html
cp -v "~/report-1s.js"      report.js
cp -v "~/chartist.min.css"  .
cp -v "~/chartist.min.js"   .
cp -v "~/moment.js"         .
cp -v "~/images/header.png" .
# now create a new header
cat ▶ 00-report.html ◀◀ EOF
     [title]Customized Report[/title]
     [link rel='stylesheet' type='text/css' href='chartist.min.css']
     [script src='moment.js'][/script]
     [script src='chartist.min.js'][/script]
     [script src='report.js'][/script]
     [h1]Station Latency Report[/h1]
     [img src='header.png' /]
     [div id="new_chart"][/div]
     [h2]Station event latency[/h2]
     [div class="ct-chart lf_chart" id="link_1"][/div]
     [pre id='log_messages' style='display:none;']
# and append old report like so:
cat orig.html ▶▶ 00-report.html
echo "please edit 00-report.html"

I have example update_report.sh and report.js examples posted for you to download and use.

The basis of the reformatting is to move the contents at the top of orig.html into the new report header. Delete content that you don’t want, but leave the Log Messages section, because it will be parsed by the javascript. The style tag keeps it from showing on the page. The first column of the messages is in Unix epoch seconds and milliseconds.

[pre id='log_messages' style='display:none;']
1499751346.692  Starting loop number: 1
1499751346.697  #[1] Bringing all selected ports down.
1499751346.841  #[1] Bringing up selected ports in batches of 100
1499751346.845  set_port 1 1 7 NA NA NA NA 0 NA NA NA NA 8388610
1499751346.854  set_port 1 1 8 NA NA NA NA 0 NA NA NA NA 8388610
1499751346.854  set_port 1 1 9 NA NA NA NA 0 NA NA NA NA 8388610

Data is collected in the report.js javascript by parsing each line for important events, using regular expressions:

matches = line.match(/^(\d+\.\d+) EVENT: .*? eventType:Link-Up details:Port (sta\d+) is Link UP.$/) || [];

Bar charts are created with the create_bar() function.

Line and point charts are created with the Chartist javascript library. Point charts are line charts with lines turned off. The moment.js library allows the Chartist library to process domain data on a time axis.

Below is an example of what a resulting report looks like:pbu-05