Redirecting Web Browsing Using Squid and DNSMasq

When testing a captive portal with a virtual station on a LANforge machine, you are faced with a problem of multiple gateways. When the station associates, it gains a DHCP address with a new gateway and DNS server. Your LANforge machine will already have a default gateway and DNS server assigned to it throught the management interface it has on boot. You need to use both. On top of that, you are connected via screen-sharing to a laptop and not to the LANforge machine directly.

When You Have a Serial Console

If you’re lucky, you have a working serial console connection between the screen-share laptop and the LANforge system. I mention working because sometimes the quality of USB serial adapters is spotty. When it works, you have a way to set new values into /etc/resolv.conf and a default gateway.

Getting the station information via the command-line is something you can do using the lf_portmod.pl script:

/home/lanforge/scripts $ ./lf_portmod.pl --manager localhost \
 --card 1  --quiet yes \
 --port_name wlan0  --show_port
 
 >>RSLT: 0 Cmd: 'nc_show_port' '1' '1' 'wlan0'

Shelf: 1, Card: 1, Port: 3 Type: STA Alias: 
...
 IP: 10.45.1.2 MASK: 255.255.255.0 GW: 10.0.0.1 VID: 0 ResetState: COMPLETE
 DNS Servers: 8.8.8.8
 IPv6-Global: DELETED
 IPv6-Link: fe80::a00:27ff:fe0a:973d/64
 IPv6-Gateway: DELETED
 MAC: 08:00:27:0a:97:3d DEV: wlan0 MTU: 1500 TX Queue Len: 1000
...

Note down your existing route information:

$ route -n > existing_route.txt

Set your route to the laptop so you don’t loose it. Then update your new route:

$ sudo route add -host $laptop-ip dev eth0
$ sudo route set default gw 10.0.0.1

Next, re-order the entries in your /etc/resolv.conf file:

nameserver: 176.168.1.1
nameserver: 8.8.8.8 
# nameserver: 176.168.1.1

And that should allow your browser to work well enough. If you fat-finger the routes, you can renew the DHCP address on your management port (eth0) using the ifup/ifdown commands:

$ sudo ifdown eth0; sudo ip a flush eth0; sudo ifup eth0

And that should restore your management connection to what you had during boot up.

Often–No Serial Console

The problem is, everything your browser will want to do will use your default gateway, not the virtual stations. Changing the default gateway will drop your connection to the LANforge machine and you will be lost from it. When you are developing remotely, you can waste days waiting for your remote counterpart doing hands-on to reboot the LANforge when you lose the network settings. Luckily, LANforge boxes come with squid and dnsmasq installed.

Starting with dnsmasq

We can start by setting up dnsmasq, so that we can direct DNS requests through the stations DNS setting.

  1. Update /etc/resolv.conf to point at localhost:
    nameserver 127.0.0.1
    # old nameserver:
    nameserver: 176.168.1.1
  2. Update the /etc/dnsmasq.conf to query the station’s DNS server:
    server=8.8.8.8

Then start dnsmasq:

$ sudo systemctl start dnsmasq

Tail your system logs to check for typos:

$ journalctl -f

Test a DNS query:

$ host www.cnn.com 127.0.0.1
Using domain server:
Name: 127.0.0.1
Address: 127.0.0.1#53
Aliases: 

www.cnn.com is an alias for turner-tls.map.fastly.net.
turner-tls.map.fastly.net has address 151.101.1.67
turner-tls.map.fastly.net has address 151.101.65.67
turner-tls.map.fastly.net has address 151.101.129.67
turner-tls.map.fastly.net has address 151.101.193.67
turner-tls.map.fastly.net has IPv6 address 2a04:4e42::323
turner-tls.map.fastly.net has IPv6 address 2a04:4e42:200::323
turner-tls.map.fastly.net has IPv6 address 2a04:4e42:400::323
turner-tls.map.fastly.net has IPv6 address 2a04:4e42:600::323

Do not enable dnsmasq using $ sudo systemctl enable dnsmasq; sudo systemctl daemon-reload. That will take effect after a reboot and you might not remember doing it. Please remember to comment out 127.0.0.1 in /etc/resolv.conf.

Move onto squid

We want to set the tcp_outgoing_address setting in our /etc/squid/squid.conf file.

tcp_outgoing_address 10.45.1.2

Start squid:

$ sudo systemctl start squid

Open a terminal and tail squid logs:

$ cd /var/log/squid; sudo tail -F *log

Next configure FireFox

The Network settings in FireFox are under Edit→Preferences and Advanced→Network. Edit the HTTP proxy to be 127.0.0.1 and port 3128. Erase the values for No Proxy for because having 127.0.0.1 and localhost in there will force FireFox to make no connections whatsoever.

Screenshot at 2017-11-10 13:35:08

Verify with tcpdump

Now the fun begins! We want to see web traffic and DNS queries flowing out the station interface and NOT the management interface. Open two windows, for two different tcpdump sessions. Monitor the management port:

$ sudo tcpdump -eni eth0 port 80 or port 53 or port 443

And monitor the station:

$ sudo tcpdump -eni wlan0 port 80 or port 53 or port 443

Using curl

to watch for redirects

Capturing a captive portal redirect is usually a matter of sending any HTTP request over the station and the response will be a 302 (or 307) redirect.

$ source ~/lanforge.profile # sets path
$ which curl # should be /home/lanforce/local/bin/curl
$ curl -svLki -4 
   --interface wlan0 \
   --localaddr 10.45.1.2 \
   --dns-interface wlan0 \
   --dns-ipv4-addr 10.45.1.2 \
   --dns-servers 8.8.8.8 \
   http://1.1.1.1/

Use the portal-check.pl script

In the /home/lanforge/scripts directory, you might have a portal-check.pl script. This script performs most of the outlines steps discussed so far. It is useful for helping validate the environment and gathering data to use with portal-bot.bash.

A response looks like:

< Location: http://portal.example.com:8080/
< Content-Type: text/html; charset=UTF-8
< Date: Fri, 10 Nov 2017 22:05:01 GMT
< Expires: Sun, 10 Dec 2017 22:05:01 GMT
< Cache-Control: public, max-age=2592000
* Server gws is not blacklisted
< Server: gws
< Content-Length: 219
< X-XSS-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< 
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://portal.example.com:8080/">here</A>.
</BODY></HTML>

 

When diagnosing behavior of captive portals, often they are constructed to redirect to arbitrary high-level ports. Feel free to add more ports to your tcpdump filter. But at this point, curl is probably going to be your best friend.
#

Advertisements

Temporarily Changing the Default Gateway to a Virtual Station

When testing a virtual station and you need to verify with a browser that you can reach a captive portal page, or browse the Internet–you want to change the default gateway of the system to the gateway that the virtual station was assigned. This is a manual change, because when the virtual station associates, the default route of the system doesn’t change. If it did, you’d probably lose connection between the LANforge resource and your management station would probably be lost.

After checking your associated station, you can write a script to help set the new default gateway. Here’s an example with some fixed values:

#!/bin/bash
echo "Please do this command in screen"
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - #
# Use this script to modify your routes to allow a station      #
# to become the default route. Should maintain route for your   #
# existing connection to management laptop through eth0.        #
# This script is an example to be modified.                     #
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - #
set -x
ip a show eth0
ip a show sta100
ip route show

# this adds address to station, referred to below
# ip a add 10.45.1.23/24 dev sta100

# make static route to old gateway
ip route add 192.168.45.1 dev eth0

# make static route to mgt laptop
ip route add 192.168.45.2 dev eth0

# now update default route to go out sta100
ip route change default via 10.45.1.23 dev sta100

sleep 1s
ping -nc2 192.168.45.1
ping -nc2 192.168.45.2

# adjust this to your work time window
read -t $((2 * 60)) -p "Press Enter to end and reset"

# and set the route back
ip route change default via 192.168.45.1 dev eth0
ip route show
ping -nc2 192.168.45.1
ping -nc2 192.168.45.2

Notice that the script suggests using screen.  Why use screen? Because if your terminal session disconnects, the script will continue and reset the previous default gateway. If the script is interrupted (by session disconnection or control-c) your default route will remain running across your AP. (We add default link-local routes to our laptop to help mitigate this.)
There is a default .screenrc file in /home/lanforge. You want to run the script as root. This will work: sudo ./change_gw.sh

This is an example script with some logic to help detect the default gateway, your ssh connection, and you tell it what station to use:

#!/bin/bash
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - #
#  Use this script to modify your routes to allow a station     #
#  to become the default route. Should maintain route for your  #
#  existing connection to management laptop through eth0.       #
#  This script is an example to be modified.                    #
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - #

if [ -z "$3" ]; then
   echo "Use this script to assign a station on this system to be the new default gateway."
   echo "Usage: $0   "
   echo ""
   echo "Your manager-ip is the LANforge resource in mode 'manager' or 'both' and might"
   echo "not be the machine hosting the virtual station."
   echo "Example, sta3000 on resource 3, manager is 192.168.1.101:"
   echo ""
   echo "./$0 192.168.1.101 3 sta3000"
   echo ""
   echo "Please do this command in screen"
   exit 1
fi

def_gw_dev=`route -n | awk '/^0.0.0.0 / {print \$NF}'`
def_gw_ip=`ip a show $def_gw_dev | awk '/inet /{print \$2}'`
def_gw_ip=${def_gw_ip%/*}
echo "default gw device: $def_gw_dev $def_gw_ip "
sleep 3
manager_ip=`who | perl -ne '/\(([0-9.]+)\)$/ && print "$1\n";'`
#ip a show $def_gw_dev
echo "This is the station you requested:"
ip a show $3
sleep 3

#  make static route to old gateway
echo "ip route add $def_gw_ip dev $def_gw_dev"
ip route add $def_gw_ip dev $def_gw_dev

#  make static route to mgt laptop
echo "ip route add $manager_ip dev $def_gw_dev"
ip route add $manager_ip dev $def_gw_dev
sleep 3

# now update default route to go out sta100
cd /home/lanforge/scripts
sta_gw=`./lf_portmod.pl --manager $1 --card $2 --port_name $3 \
  --show_port --quiet yes \
  | awk '/ IP:/{print \$6}'`

echo "* Changing default gateway in 3 seconds *"
sleep 3s
echo "ip route change default via $sta_gw dev $3"
sleep 1s
ip route change default via $sta_gw dev $3
sleep 1s
ping -nc2 $def_gw_ip
ping -nc2 $manager_ip

# adjust this to your work time window (in seconds)
#  $(( 2 * 60)) == 120, 2 minutes
read -t $((2 * 60)) -p "Press Enter to end and reset"

# and set the route back
echo "changing route back to previous default gw..."
ip route change default via $def_gw_ip dev $def_gw_dev
ping -nc2 $def_gw_ip
ping -nc2 $manager_ip
echo "...done."
#

To use the script, you would say:

sudo ./change_gw.sh 192.168.100.41 3 sta3000

You are probably going to need to alter the script. If you hit Ctlc in the script you will need to reset the routes by hand.

For actually using Firefox, you need to operate this from the desktop of LANforge resource hosting the virtual station. So if, in this example, sta3000 is on resource 3, you need to get on the virtual desktop of resource 3. Open a terminal from the virtual desktop or ssh to resource 3 and start screen as user LANforge.

Modifying the Port Bringup Report

The LANforge Port Bringup plugin produces a highly detailed and verbose report including all the wifi events from the stations involved. This plugin can be used to record performance of station association and captive portal logins. When you save the Graphical Report screen, it saves an HTML document with pictures to a directory on your machine.

pbu-01

pbu-02

pbu-03

We see the report files created by saving the report below:

pbu-04

This report can be reformatted, and the log messages at the bottom of the output are a useful resource to recreate graphs of interest. With some accessory JavaScript, we can create a completely different report. I start off by writing a shell script that backs up the original index.html file of the report and concatenates it to the end of a new title block.

Example shell script below:
(please excuse my lazy [] brackets, because  &-lt; &-gt; takes longer to type)

#!/bin/bash
if [ ! -f orig.html ]; then
   mv -v index.html orig.html
   dos2unix orig.html
fi
cp -v "~/report-1s.js"      report.js
cp -v "~/chartist.min.css"  .
cp -v "~/chartist.min.js"   .
cp -v "~/moment.js"         .
cp -v "~/images/header.png" .
# now create a new header
cat ▶ 00-report.html ◀◀ EOF
[html]
   [head]
     [title]Customized Report[/title]
     [link rel='stylesheet' type='text/css' href='chartist.min.css']
     [script src='moment.js'][/script]
     [script src='chartist.min.js'][/script]
     [script src='report.js'][/script]
   [/head]
   [body]
     [h1]Station Latency Report[/h1]
     [img src='header.png' /]
     [div id="new_chart"][/div]
     [h2]Station event latency[/h2]
     [div class="ct-chart lf_chart" id="link_1"][/div]
     [pre id='log_messages' style='display:none;']
EOF
# and append old report like so:
cat orig.html ▶▶ 00-report.html
echo "please edit 00-report.html"

I have example update_report.sh and report.js examples posted for you to download and use.

The basis of the reformatting is to move the contents at the top of orig.html into the new report header. Delete content that you don’t want, but leave the Log Messages section, because it will be parsed by the javascript. The style tag keeps it from showing on the page. The first column of the messages is in Unix epoch seconds and milliseconds.

[pre id='log_messages' style='display:none;']
1499751346.692  Starting loop number: 1
1499751346.697  #[1] Bringing all selected ports down.
1499751346.841  #[1] Bringing up selected ports in batches of 100
1499751346.845  set_port 1 1 7 NA NA NA NA 0 NA NA NA NA 8388610
1499751346.854  set_port 1 1 8 NA NA NA NA 0 NA NA NA NA 8388610
1499751346.854  set_port 1 1 9 NA NA NA NA 0 NA NA NA NA 8388610

Data is collected in the report.js javascript by parsing each line for important events, using regular expressions:

matches = line.match(/^(\d+\.\d+) EVENT: .*? eventType:Link-Up details:Port (sta\d+) is Link UP.$/) || [];

Bar charts are created with the create_bar() function.

Line and point charts are created with the Chartist javascript library. Point charts are line charts with lines turned off. The moment.js library allows the Chartist library to process domain data on a time axis.

Below is an example of what a resulting report looks like:pbu-05

Captive Portal Testing

A popular feature of the LANforge WiFIRE feature set is captive portal testing portal-bot. If you’re considering doing captive portal testing, the first thing to know is that you are not going to be automating a bunch of web browsers (that would be madness). You will be writing a perl script. More specifically, a perl .pm extension to our portal-bot.pl script.

Portal testing can be complex because it normally involves re-creating whatever JSON/AJAX/REST call chain the web pages does with a perl script. This process looks like:

  1. Establish laptop with serial connection to LANforge system and with an Ethernet cable
  2. VNC to LANforge and change default gateway and use browser to document login HTTP process
  3. Write a portal-bot extension that mimics that behavior
  4. Incorporate a logout mechanism

It is very useful (and a requirement for long-running capacity testing) to have an API call to the captive portal controller that will log out a session. Otherwise you cannot repeat the test for each station without manually intervening at the captive portal controller to log out one/some/all logged in stations. This logout mechanism could be web-based or could be ssh based, so long as the mechanism can be unattended.

Please review some of our existing portal-testing documentation:

Controlling ARP Solicitations

When your network endpoints are not changing during testing scenario, transmitting ARP packets at the default rate is arguably wasted bandwidth. You can tune the Linux networking stack to extend the time between ARP broadcasts.

These tunables are in /proc/sys/net/ipv4/neigh and are divided by default and per-device settings. The knobs I find that are useful are:

  • base_reachable_time: a guideline on how often to broadcast for ARP updates
  • gc_stale_time: threshold in seconds on when to consider evicting an arp entry
  • locktime: minimum time to keep an ARP entry

You can set twist these knobs for two ports in a shell script like so:

for f in /proc/sys/net/ipv4/{enp2,enp3}/{base_reachable_time,gc_stale_time,locktime} ; do
    echo 300 > $f
done

This changes the values to 5 minutes.