Tracking Station Traffic thru VRF Interfaces

Typically the virtual WiFi stations that LANforge creates are encapsulated with a virtual router device which holds its own routing table for the set of devices inside it. We do this to isolate the physical host’s routing table from the virtual mobile device we are emulating. This means there are special ways to operate connections through stations. A standard ping will always send packets through the port having the default gateway (which is your management network). You want to send packets to your test network.


Networks

Management: eth0 192.168.1.101/24 gateway 192.168.1.1
Test LAN:   <AP> 10.0.0.1/24 running dhcpd SSID testssid
Test WAN:   eth1 176.16.0.1/24, running dhcpd
Station:    wlan0 10.0.0.175/24 gateway 10.0.0.1 nameserver 10.0.0.1

Ping from the station

To ping from the station you use a version of the command sudo ip vrf exec wlan0 <command>. From a LANforge, you would shorted that a bit and say:

sudo -s
cd /home/lanforge
./vrf_exec.bash wlan0 ping -I wlan0 www.example.com

Ideally what this does is:

  • Places the vrf routing table in the ping process space
  • ping -I wlan0 forces ping to send packets out wlan0
  • ping uses libraries to operate gethostbyname() to resolve http://www.example.com from the nameserver given to the station via DHCP
  • sends an ICMP packet to the host

If this DNS server is not reachable or there is no route to that IP from the first response, ping tries harder and starts asking other interfaces if there is a route. This can dodge VRF and start asking on other interfaces. Ping is so persistent that will also check /etc/resolv.conf. This is troublesome because that was probably generated by the host’s dhclient process running on the management network. This can cause DNS queries out to your management network.

Changing resolv.conf

Changing resolv.conf to list the test net nameserver can help. It’s not great for the times when you want to run updates, but its not difficult to spot failed name lookups.

options timeout:1 attempts:1
nameserver 10.0.0.1

Those settings will restrict the amount of time spent by clients doing lookups.

Checking Route Tables

You can check the route tables available with ip route show table all, or grep for your station to see more quickly what route table is assigned to your station. Usually route tables match the number of the station master interface.

root # ip li sh wlan0
22: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master _vrf17 state UP mode DORMANT group default qlen 1000
link/ether 04:f0:21:3f:00:08 brd ff:ff:ff:ff:ff:ff permaddr 04:f0:21:2a:dc:08

root # ip route show table 17
default via 10.40.0.1 dev wlan0
10.40.0.0/20 dev wlan0 scope link src 10.40.0.75
local 10.40.0.75 dev wlan0 proto kernel scope host src 10.40.0.75
broadcast 10.40.15.255 dev wlan0 proto kernel scope link src 10.40.0.75

Prediction the Route

You can match up the route between a station and a destination using ip route get

 root # ./vrf_exec.bash wlan0 ip route get 10.40.0.1 from 10.40.0.75
10.40.0.1 from 10.40.0.75 via 192.168.92.1 dev eth0 uid 0
cache

You might think that looks weird. In fact, I don’t know why eth0 is there at all.

Using Perf to Trace the calls

This is a fascinating way to use perf (which uses eBPF probably) to see what actually is happening when doing a ping:

# ./vrf_exec.bash wlan0 perf trace --no-syscalls --event 'net:*' ping -R -O -c1 -n -B -I wlan0 lanforge.net
PING www.lanforge.net (10.41.0.1) from 10.40.0.75 wlan0: 56(124) bytes of data.
0.000 ping/1880863 net:net_dev_queue(skbaddr: 0xffff88818fffa400, len: 138, name: "wlan0")
0.007 ping/1880863 net:net_dev_start_xmit(name: "wlan0", skbaddr: 0xffff88818fffa400, protocol: 2048, len: 138, network_offset: 14, transport_offset_valid: 1, transport_offset: 74)
0.029 ping/1880863 net:net_dev_xmit(skbaddr: 0xffff88818fffa400, len: 138, name: "wlan0")
From 10.40.0.1 icmp_seq=1 Destination Host Unreachable

--- www.lanforge.net ping statistics ---
1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms

1.133 ping/1880863 net:net_dev_queue(skbaddr: 0xffff888121376ce8, len: 174, name: "eth0")
1.137 ping/1880863 net:net_dev_start_xmit(name: "eth0", skbaddr: 0xffff888121376ce8, protocol: 2048, ip_summed: 3, len: 174, data_len: 108, network_offset: 14, transport_offset_valid: 1, transport_offset: 34, gso_segs: 1, gso_type: 1)
1.140 ping/1880863 net:net_dev_xmit(skbaddr: 0xffff888121376ce8, len: 174, name: "eth0")

Notice how since 10.41.0.1 is not routable (unlike 10.41.0.1) ping digs down to the interface with the default route (eth0) and attempts to use that.

Using getent

Another easy way to see if your configuration is doing what you want is to check how nsswitch is configured by using getent:

# ./vrf_exec.bash sta0000 getent ahosts www.lanforge.net
10.41.0.1 STREAM www.lanforge.net
10.41.0.1 DGRAM
10.41.0.1 RAW

This is much easier to use, if we were paying attention to what was in our /etc/hosts file, we would have spotted this typo sooner!

WiFi Fundamentals Course

The team at Candela Technologies is pleased to offer a free online semester course about the fundamentals of WiFi technology. This course is being taught by industry experts and all the concepts will be taught in a very practical manner keeping the industry applications of technology in mind.

Format

Classes will be delivered live over webinar sessions every Tue and Thu 4pm IST. All sessions will be recorded and the recordings will be made available on the course webpage.

Audience

This course is designed for junior engineers already working in R&D organizations on WiFi technology, engineering students who are interested in the computer networking and Wireless communications domain and also academicians who are interested in developing new skill courses in engineering programs. Prerequisite for this course is basic understanding of computer networking.

Registration

Anyone interested can register for this course by visiting the course webpage. Interested people can also join the Weekly WiFi wisdom Whatsapp group to get whatsapp updates and participate in conversations related to the course. Links to register for the course and group can be found of course webpage.

LANforge 5.4.4 Released

Ben working on LANforge

We are excited to share the new features in 5.4.4:

  • WiFi 6 and 6e client emulation
  • TR398 issue 2 test suites
  • Android WiFi testing app
  • improved python scripting

New Linux Kernel and Intel Wireless support:

Support recent 5.15 kernel for updated drivers and latest features. This includes several updated drivers for Intel AX200 and AX210 radios.

MediaTek Radio Support

  • Support MTK 4×4 AX radios in tr-398 issue 1 (except MU-MIMO test case, which is not currently supported.)
  • Support MTK 4×4 AX radios in tr-398 issue 2 (except MU-MIMO test case, which is not currently supported.)
  • Support tx-overrides on MTK 4×4 AX NIC (ability to specify NSS, MCS, bw, guard-interval for transmitted data frames.

TR398 Testing Suites

  • Update TR398 issue 2 to support latest proposed test revision.
  • Improve report content for TR398 and TR398i2.
  • Add proposed TR398 issue-3 Airtime Fairness (ATF) test case.
  • Add proposed TR398 issue-3 Quality of Service (QoS) test case.
  • Add proposed TR398 issue-3 Latency test case.
  • Add proposed TR398 issue-3 Multicast test case.
  • Add subtest pass/fail counters to TR398 and TR398i2 KPI reports.

Test Updates

  • Add pass/fail option for Dataplane and Rate vs Range test.
  • Add Latency graphs for the AP-Auto multi-station throughput test.
  • Scale test: Add ping support, improve error checking and feedback to user.
  • Improve RvR test report and procedure.

Android App

Initial support for LANforge pure-java resource on Android phones. Includes TCP, UDP, IPv4, IPv6 and HTTP/HTTPS protocol support for traffic generation. This app has been used in campus WiFi testing.

Python Scripting

Improved python scripting support, including:
– auto-generated .py files to support LANforge CLI commands
– cleanup and stability improvements
– various new automated python tests

Regression testing of the scripting library fixed many areas. We are introducing support for Python virtual environments. Python Pypi libraries are now installed in user environment and do not require OS python libraries.

Fedora Support

We continue focusing our product around Fedora, and support python scripting on Fedora 27 and newer. Our latest products are installed using Fedora 34.

LANforge 5.4.1 Released

20160722-splitters-anvil-430
We’re happy to announce LANforge 5.4.1, that supports a number of new features:

  • Add TR-398 automated WiFi test support.
  • Support /AC on 2.4Ghz on /b/g/n/AC radios (2.4Ghz /AC)
  • Add support for wired traffic test to the Dataplane scenario test.
  • Support 5.2 kernel for latest drivers and kernel features.
  • Support IPSEC interfaces.
  • Support CT850a 2D attenuator turn-table.
  • Support ComXim MT series turn-table (for instance: MT200RUWL20)
  • Improve performance of creating virtual stations, tested 500 stations per /n radio.

IPTables and conntrack

When you are using LANforge as a virtual router doing NAT, you might need to see how many NAT table entries you’re handling.  This can be important because NAT entries take memory, and if you want to handle 65,000 simultaneous connections, you might be heading for trouble.

If your LANforge is only generating traffic, you won’t see NAT entries…rather you want to use netstat -ntp to see how many open connections there are.

LANforge uses iptables PREROUTING heavily, forcing each port to have it’s own set of tables. When you type iptables -nvL and see nothing…that’s because nothing is in the tables for your default route, which is probably eth0. You get close with the raw table. Try iptables -S -t raw. You will see PREROUTING entries for every interface:

# iptables -S -t raw
-P PREROUTING ACCEPT
-P OUTPUT ACCEPT
-A PREROUTING -i br2000 -j CT --zone 10001
-A PREROUTING -i eth1 -j CT --zone 10001
-A PREROUTING -i vap13 -j CT --zone 10001
-A PREROUTING -i vap14 -j CT --zone 10001
-A PREROUTING -i eth2 -j CT --zone 10001

This shows we have a CT chain and a zone note for that chain.

When you create a virtual router, add NAT to a port in it, you can view the NAT table entries with conntrack.

* conntrack -L will list them all, but that’s probably not super useful
If you’re running TCP-multicon connections, expect thousands of connections.
* conntrack -C will show how many NAT entries are present, so you can avoid doing a conntrack -L | wc -l

Happy networking!

Adding hostnames to dhclient configs

If you have hundreds of stations on your LANforge and want to give them all hostnames, this is certainly possible. Please get the add-dhcp-hostname.pl script from the lanforge-scripts repository.

  1. First bring up your stations with DHCP enabled. This will create your dhclient config files in /home/lanforge/vr_conf.
  2. Admin-down the stations.
  3. With a root shell, go to that vr_conf directory and run this script on the files you want to give hostnames to.
    1. For just one station:
      root# ~lanforge/scripts/add-dhcp-hostname.pl dhclient_sta0000.conf
    2. For all the stations:
      root# ~lanforge/scripts/add-dhcp-hostname.pl dhclient_sta*conf
  4. Admin-up the stations.
    1. If you are sniffing, you will see a DHCP Request packet that has a hostname attribute.
    2. If you are inspecting a LANforge dhcpd server in a virtual router, you would check /home/lanforge/vr_conf/vrcx_eth1_dhcp_lease.db for entries with the setting client-hostname.

That should work.

Java on OS X

Sometimes your Mac has a very old version of Java on it, like 1.6.0_123. This can break running the LANforge client. Let’s learn how to manage this. First, open Terminal and find what version of Java you think you have:

$ java -version

This reports what your JAVA_HOME environment variable (which might not be set) is, so the System will do a /usr/libexec/java_home command for you. You might need to follow these instructions on removing old Java version.

Install a recent version of Java. Then go back to the terminal. You want to make sure that your JAVA_HOME variable is getting set to the version you want. Verify you have a .profile and a .bashrc file in your home directory:

$ ls -l ~/.profile ~/.bashrc

Create them if the don’t exist:

$ touch ~/.profile ~/.bashrc
$ chmod 700 ~/.profile ~/.bashrc
$ echo "source .bashrc" >> ~/.profile

To see the versions of Java installed, use the java_home command:

$ /usr/libexec/java_home -V

The first column of output will be the version number, which is important for running the java_home command. Edit your .bashrc file to export your JAVA_HOME environment variable:

export JAVA_HOME=$(/usr/libexec/java_home -v 1.8.0_122)

Save and open a new terminal. Verify using java -version.

More discussion on setting the version: