macronet

Getting more speed out of OpenWRT-enabled TL-WR1043NDv2

 Hardware, Kernel and drivers, Linux, Network  Comments Off on Getting more speed out of OpenWRT-enabled TL-WR1043NDv2
Mar 202018
 

Basic setup and debug

1Gbps/100Mbps -cable connection, router, couple of cat6 cables, laptop & speedtest.net (note: speedtest.net might not give reliable results on high-speed links).

 

Default speeds (without a router).
Cable modem is setup as a dumb bridge, it’ll just be a media converter between ethernet and the DOCSIS 3.1 -world. First tests were run with laptop hooked up to it with cat6 -cable, public IP-address resided on the laptop and “Hey, this works”. Plugged it into existing infrastructure (WDR4300) and carried on with my normal life.

Then came the next weekend and I had a bit too much time in hand to benchmark – and I found out that the wireless router runs out CPU if I download something big and the speed was capped at about 240Mbps…

What if I’ll change a router?
WDR4300 has an AR9344 CPU running at 560MHz – my reserve WR1043NDv2 has QCA9558 CPU running at 720MHz – a whopping 160MHz (~28.5%) more! I took a look at OpenWRT OpenSSL benchmarks and they confirmed my theory, CPU should be about a quarter faster, but would it translate into real world performance?

This is a bit like comparing apples and oranges, because the CPUs are different, firewall rules are close but not identical, and the latter one doesn’t handle wireless at the same time, but the speed difference is still noticeable.

What if I’ll simplify firewall rules?
Replaced all the default LEDE -firewall rules with just one NAT-rule, just allowed everything, and … about +10% more bandwidth.
How do I get more speed out of a CPU? Overclock!
Had a crazy idea, is it possible to overclock a router? And found out it actually is possible, just backup mtd0 (u-boot) -partition, replace default MHz -values with hex-editor to something else and burn the image back to router. Or use pre-made images or some chinese guys closed-source bootloader.

720MHz to 1GHz (+38%) and +32% more bandwidth with default firewall, nice!

Now let’s try it with a bare minimum firewall…
Same as before, just one NAT-rule.

+11% – and the speed is almost doubled from the start (WDR4300 and ~240Mbps).

What is this “NAT Boost / Hardware NAT” -feature in stock TP-Link firmware?
Back to default CPU speeds, stock firmware has some kind of shortcut for packets going through a NAT. Let’s see if we have..yes, OpenWRT is bringing SFE (Shortcut Forwarding Engine) -support to their later release.

Luckily there were some images ready for testing.

Shortcut forwarding engine, with just a NAT-rule.
Let’s combine that with a minimal firewall setup, just a single NAT-rule.
Shortcut forwarding engine, overclocked.
And what happens if we push the CPU from 720MHz to 1GHz. (With default LEDE-firewall.)
Full speed!
Let’s drop the firewall (use just a single NAT-rule), what’s the maximum attainable speed?

…not bad, just about 3.5 times faster than the original, and just a bit short of “full wirespeed”. But I think I’ll stick with a firewalled version and take that ~80Mbps hit.

Jul 012017
 
Suomenkielinen versio täällä.

Why?

One day I just needed an access control system and reservation calendar to a common use sauna, and it had an electric lock (24VAC) already in place.

Hardware (links to Finnish shops):

Stage 1 (memory card):

Raspberry Pi got the newest Raspbian Jessie, which was at the moment in version 2017-06-21. Kernel for the distribution is 4.9.28-v7+.

Stage 2 (assembly):

  • detach controller from the TFT (TFT-enclosure required this)
  • assemble the TFT enclosure
  • attach the controller to the TFT, make sure both cables are in place
  • drill holes to the bottom of RasPi enclosure (6mm drill), to use TFT -attachment points
  • insert memory card to RasPi
  • attach RasPi with it’s enclosure to TFT
  • attach the cable from TFT to RasPi, along with 5V(pin4) and Gnd(pin6) with jumper wires
  • attach NFC-reader with jumper wires to RasPi, you’ll use pins 1,3,5,11,12,15,16,19,21-26 (f.ex 3.3V, Gnd, i2c & SPI)
    (Explore NFC Board Schematic)
  • attach relay card module to RasPi, relay needs ~80mAh of current so take note of this when deciding the PSU
    pins 2 (5V -> VCC), 7 (GPIO -> In1) and 9 (Gnd -> Gnd) are needed
  • attach the relay between the another power cable going to the electric lock

Stage 3 (software):

Reader uses SPI, so enable SPI at raspi-config. TFT will rotate by adding display_rotate=1 (90 degrees) or display_rotate=3 (270 degrees) to /boot/config.txt – and console font can be made bigger with dpkg-reconfigure console-setup.

For actual use you need a small stack of software. Relay module must be controller (to open/close door) and NFC-card must be read and validated against some user database. And unless you want to keep the user database up-to-date by hand, there should be something for that also…

My solution was a simple python-service, which reads the NFC-card, validates is against data in MariaDB and after that tells the result on screen & activates the relay if necessary. Almost up to date code can be found at https://macronet.fi/dev/nfc/ (maybe later at github also) – it might work, or not.

Library used for reading the NFC-card is nxppy -that and all other requirements are easy to install:

# sudo apt install python-dev python-pip python-mysqldb cmake mariadb-server-10.0
# pip install nxppy

Stage 4 (installation):

Install the system to the place you want, so that you can show a card to the reader. This might need some cables if you want to take the reader further away – please make note that the reader requires 14 wires (haven’t actually tested if it works without i2c, because it uses SPI) so it’s like 2 cat5e cables.

Stage 5 (why internet connection? and then what?)

Make RasPi to call home/VPN, so that you are able to fix problems remotely.

Or “forget” a wireless keyboard receiver to USB-port and use your new NFC-enabled access control system as an IRC-client.

Jul 012017
 
English version here.

Miksi?

Tulipa eräänä päivänä tarvetta toteuttaa kulunvalvonta ja varauskalenteri erääseen yhteiskäytössä olevaan saunatilaan, missä oli ovenkarmissa jo sähkövastinrauta (24VAC) valmiina.

Käytetty rauta:

Vaihe 1 (muistikortti):

Raspberry Pi  sai sisäänsä uusimman Raspbian Jessien, joka tekohetkellä oli versiota 2017-06-21. Jakelun kernelinä toimii 4.9.28-v7+.

Vaihe 2 (kokoonpano):

  • irrota TFT:stä ohjainkortti (valitsemani kotelo vaati tämän)
  • kokoa TFT:n kotelo näytön ympärille
  • kiinnitä TFT:n ohjainkortti takaisin paikoilleen, varmista että molemmat lattakaapelit ovat hyvin kiinni
  • poraa raspin kotelon pohjaosaan oikeisiin kohtiin reiät (6mm terällä) TFT:n kiinnikkeitä varten
  • laita muistikortti raspiin
  • kiinnitä raspi koteloineen TFT:n kiinnikkeisiin
  • kiinnitä näytön lattakaapeli sekä virtajohdot 5V(pin4) ja Gnd(pin6) hyppylangoilla raspiin
  • kiinnitä NFC-lukija hyppylangoilla raspiin, tarvitset pinnejä 1,3,5,11,12,15,16,19,21-26 (mm. 3.3V, Gnd, i2c & SPI)
    (Explore NFC Board Schematic)
  • kiinnitä relekortti raspiin, rele tarvitsee toimiakseen ~80mAh eli huomioi tämä virtalähteen mitoituksessa
    tarvitset pinnit 2 (5V -> VCC), 7 (GPIO -> In1) ja 9 (Gnd -> Gnd)
  • kierrätä sähkölukon toinen virtajohto releen kautta

Vaihe 3 (softa):

Lukija käyttää SPI:tä, eli enabloi SPI raspi-config :sta. TFT käännetään pystyyn lisäämällä /boot/config.txt :n display_rotate=1 (90 astetta) tai display_rotate=3 (270 astetta) ja konsolin fonttia saa kasvatettua dpkg-reconfigure console-setup :lla.

Varsinaista käyttöä varten tarvitaan pieni nippu sovelluksia. Pitää esimerkiksi hallita relekorttia (avata/sulkea ovi) sekä lukea nfc-kortteja ja validoida luettu kortti jotain käyttäjäkantaa vasten. Ja ellei halua täyttää käyttäjäkantaa käsin niin niiden ylläpitoa vartenkin on hyvä olla omat sovelluksensa…

Oma ratkaisu oli toistaiseksi yksinkertainen python-service joka lukee nfc-kortin, validoi sen MariaDB:tä vasten jonka jälkeen kerrotaan ruudulla tulos & aktivoidaan rele tarvittaessa. Lähes ajantasalla olevaa koodia löytyy osoitteesta https://macronet.fi/dev/nfc/ (ehkä joskus myös githubissa) – se voi toimia tai olla toimimatta.

NFC-kortin luentaan käytetty nxppy -kirjasto ja muut tarvittavat riippuvuudet on helppo asentaa:

# sudo apt install python-dev python-pip python-mysqldb cmake mariadb-server-10.0
# pip install nxppy

Vaihe 4 (paikalleenasennus):

Asenna järjestelmä haluamaasi paikkaan paikoilleen niin että nfc-lukijalle voi näyttää korttia. Tämä voi vaatia hieman kaapelointia jos haluat viedä lukijan kauemmaksi raspista – huomioithan että lukija kaipaa 14 kaapelia (ei tosin ole tullut testattua _vaatiiko_ se niitä oikeasti, vai voiko esim. i2c:t jättää kytkemättä koska käyttää SPI:tä) eli 2kpl cat-kaapeleita.

Vaihe 5 (miksi internet? mitä sitten?)

Laita raspi soittamaan kotiin/VPN:ään, että pystyt korjaamaan mahdolliset vikatilanteet etänä.

Tai “unohda” usb-porttiin langattoman näppiksen vastaanotin ja käytä uutta nfc-kulunhallintajärjestelmääsi vaikka irkkaamiseen lokaalisti.

Feb 202016
 

I had a classic problem, old server without a remote KVM capabilities – if it goes down, no other chances to debug it than going next to it, plug in a monitor and a keyboard.

Then, one day, an epiphany. There is a “Console redirection” in BIOS, and after a quick enable & check – it really worked. BIOS was accessible over serial connection.

Next step, /etc/default/grub (under Debian) open and GRUB_CMDLINE_LINUX="" -> GRUB_CMDLINE_LINUX="console=ttyS0 console=tty0" will enable Linux console through serial. Switching from #GRUB_TERMINAL=console -> GRUB_TERMINAL="console serial" enables Grub to work with serial.

A quick update-grub made the changes active, and during reboot it was confirmed that the whole boot from bios screen to linux login console was accessible through serial connection.

 

So what about Raspberry Pi?

setupBy default Pi has a serial console active, which can be accessed through GPIO pins 8 & 10 (Pi B+). This needs to be disabled to make console port usable for userspace, by removing console=ttyAMA0 from /boot/cmdline.txt (or by disabling a correct service). After spending less than 1€ on Ebay (to get “MAX3232 RS232 Serial Port To TTL Converter Module DB9 Connector With Cable”) I had the required hardware to connect them together.

– Fits in 1U?  *** Yes

– Pretty setup?  *** No

– Works?  *** Yes

 

 

 

serialPi was powered via an USB port on the server backplane, a CAT5e -cable was needed for a network connection and the RS232 -converter was plugged into COM1.

Setup through the COM2 header found in the corner of the motherboard would have been nicer, but it just refused to work.

There’s also cables from the Pi into the reset button -pins on the motherboard, because I needed a way to make a hard reset. Power On/Off could also be managed through power button-pins, but for me it’s quite useless because the Pi is powered by the server through USB (and the server is always on).

 

Now I can access the Pi through SSH-session and use it to access console on the main server, if I ever need to do so.

 

GPIO connections (Pi B+ -> other end):
Pin 2 -> RS232 converter: VCC
Pin 6 -> RS232 converter: GND
Pin 8 -> RS232 converter: TXD
Pin 10 -> RS232 converter: RXD
Pin 14 -> Motherboard: Reset GND
Pin 16 -> Motherboard: Reset +

Reset-software (python-rpi.gpio required, don’t blame me if it destroys everything):

#!/bin/python
import RPi.GPIO as GPIO
GPIO.setmode(GPIO.BOARD)
GPIO.setup(16, GPIO.OUT)
# We're currently sinking the current through Pi.
# This causes a "push event" on a reset button.
GPIO.setup(16, GPIO.IN)

Feb 032016
 

Test platform: UpCloud VPS (10vCPU / 32GB) with 50GB MAXIOPS

Different setups compared in this benchmark (only single run):

  • MariaDB 10.1.11 (single node)
  • MariaDB 10.1.11 Galera (three nodes in same DC)
  • MariaDB 10.1.11 Galera (three nodes, one node in another DC)
  • MariaDB 10.1.11 Galera (all three nodes in different DC:s, in the same continent)

Benchmark software used was sysbench-0.5 OLTP from Percona repositories with 24 tables and about 12GB of data. InnoDB configuration changes from (MariaDB repository debian 8) package the defaults are:

  • innodb_buffer_pool_size = 16G
  • innodb_buffer_pool_instances = 16
  • innodb_log_buffer_size = 4G
  • innodb_io_capacity = 10000
  • innodb_flush_neighbors = 0

Benchmark command (read-only):

  • for each in 1 2 4 8 16 32 64; do sysbench --test=/usr/share/doc/sysbench/tests/db/oltp.lua --db-driver=mysql --oltp-table-size=2000000 --mysql-db=sysbench --mysql-user=root --mysql-password=jeespoks --max-time=300 --max-requests=0 --oltp-read-only --oltp-tables-count=24 --oltp-reconnect-mode=random --num-threads=$each run > result-$each.txt; sleep 10; done
 Transactions /s  MariaDB 10.1 (1xFra)  Galera (3xFra)  Galera (1xHel,2xFra)  Galera (1xHel,1xFra,1xLon)
 Prepare time (8t, s)  7m 13.109s  18m 20.911s  51m 19.007s  70m 30.954s
 1 thread, read-only  431.91  428.18  28.23  20.12
 2 threads, read-only  823.80  797.25  44.45  38.72
 4 threads, read-only  1456.73  1289.39  64.45  62.86
 8 threads, read-only  1964.09  1319.78  127.07  122.77
16 threads, read-only  1671.03  1320.83  241.63  218.91
32 threads, read-only  1584.25  1307.16  466.30  399.04
64 threads, read-only  1605.12  1331.82  784.52  581.57

 

Benchmark command (read-write):

  • for each in 1 2 4 8 16 32 64; do sysbench --test=/usr/share/doc/sysbench/tests/db/oltp.lua --db-driver=mysql --oltp-table-size=2000000 --mysql-db=sysbench --mysql-user=root --mysql-password=jeespoks --max-time=300 --max-requests=0 --oltp-tables-count=24 --oltp-reconnect-mode=random --num-threads=$each run > result-$each.txt; sleep 10; done
Transactions /s MariaDB 10.1 (1xFra) Galera (3xFra) Galera (1xHel,2xFra) Galera (1xHel,1xFra,1xLon)
1 thread, read-write  435.76  393.46  27.20  20.31
2 threads, read-write  818.18  742.99  43.41  38.59
4 threads, read-write  1462.60  1230.78  64.21  61.87
8 threads, read-write  1967.40  1305.09  119.53  118.12
16 threads, read-write  1677.83  1321.16  219.10  221.30
32 threads, read-write  1569.35  1321.29  469.44  374.98
64 threads, read-write  1596.89  1328.96  782.56  548.70

 

TODO: rerun & try different optimizations

Jul 292015
 

Just a quick CPU benchmark with Phoronix test suite, comparing -cpu qemu64 and -cpu host (on Xeon E3-1241v3, 4vCPU/4GB virtuals).

Note: single run (fire & forget), virtual machines on the same physical hardware and benchmarks run simultaneously. YMMV.

Benchmark qemu64 host Higher/Lower better?
pts/stream-1.2.0 [Type: Copy] 14166.98 MB/s 14171.50 MB/s Higher
pts/stream-1.2.0 [Type: Scale] 13964.65 MB/s 14007.75 MB/s Higher
pts/stream-1.2.0 [Type: Triad] 15755.66 MB/s 15851.89 MB/s Higher
pts/stream-1.2.0 [Type: Add] 15730.77 MB/s 15863.64 MB/s Higher
pts/apache-1.6.1 25028.87 Requests Per Second 35859.45 Requests Per Second Higher
pts/john-the-ripper-1.5.1 [Traditional DES] 5849000 Real C/S 7179500 Real C/S Higher
pts/john-the-ripper-1.5.1 [Blowfish] 3170 Real C/S 3249 Real C/S Higher
pts/ttsiod-renderer-1.5.0 102.00 FPS 93.75 FPS Higher
pts/x264-1.9.0 86.56 FPS 94.52 FPS Higher
pts/graphics-magick-1.6.1 [HWB Color Space] 173 Iterations Per Minute 170 Iterations Per Minute Higher
pts/graphics-magick-1.6.1 [Local Adaptive Thresholding] 92 Iterations Per Minute 88 Iterations Per Minute Higher
pts/graphics-magick-1.6.1 [Sharpen] 95 Iterations Per Minute 102 Iterations Per Minute Higher
pts/graphics-magick-1.6.1 [Resizing] 159 Iterations Per Minute 156 Iterations Per Minute Higher
pts/himeno-1.1.0 1722.79 MFLOPS 1840.71 MFLOPS Higher
pts/compress-7zip-1.6.0 10173 MIPS 10291 MIPS Higher
pts/c-ray-1.1.0 44.54 Seconds 39.76 Seconds Higher
pts/compress-pbzip2-1.4.0 11.99 Seconds 13.95 Seconds Lower
pts/smallpt-1.0.1 153 Seconds 153 Seconds Lower
pts/crafty-1.3.0 71.60 Seconds 71.48 Seconds Lower
pts/encode-flac-1.5.0 7.38 Seconds 6.12 Seconds Lower
pts/encode-mp3-1.4.0 11.71 Seconds 11.21 Seconds Lower
pts/ffmpeg-2.4.0 18.99 Seconds 14.37 Seconds Lower
pts/povray-1.1.2 310.35 Seconds 248.59 Seconds Lower
pts/tachyon-1.1.1 26.57 Seconds 17.10 Seconds Lower
pts/openssl-1.9.0 286.50 Signs Per Second 543.67 Signs Per Second Lower
pts/mafft-1.4.0 7.40 Seconds 7.24 Seconds Lower
pts/gcrypt-1.0.3 1793 Microseconds 1647 Microseconds Lower

No wonder why UpCloud changed their parameters, -host usually wins…

Jun 222015
 

If you get hit with “RTNETLINK answers: Cannot allocate memory” when trying to add IPv6 default gateway back after losing all IPv6 -connectivity, raise net.ipv6.route.max_size.

The defaults (on my machines) were quite small compared to IPv4:

CentOS 7.0:
# sysctl net.ipv4.route.max_size
net.ipv4.route.max_size = 2147483647
# sysctl net.ipv6.route.max_size
net.ipv6.route.max_size = 4096

CentOS 7.1:
# sysctl net.ipv4.route.max_size
net.ipv4.route.max_size = 2147483647
# sysctl net.ipv6.route.max_size
net.ipv6.route.max_size = 16384

Debian 7.8/8.1:
# sysctl net.ipv4.route.max_size
net.ipv4.route.max_size = 2147483647
# sysctl net.ipv6.route.max_size
net.ipv6.route.max_size = 4096

Jun 092015
 

We have a national IPv6 day today (9.6.2015) in Finland, and this time also the largest ISP:s in Finland are attending.

DNA provides currently IPv6 to their mobile networks and cable networks, other connection types will get their support later.

Elisa provides currently IPv6 to their mobile networks (Elisa and Saunalahti), other connections will get their support later.

Sonera “will deploy it automatically to their customers as soon as they have confirmed that it works on their network”, though no concrete promises or dates. There’s some rumours about 6rd support in their cable networks and old https://fi-fi.facebook.com/sonera/posts/119937098144430 about IPv6 support in their corporate broadband offerings.

 

On the other news, Macronet has an IPv6 -enabled cloud platform running and provides all services from there, customer access will come later.

Jun 022015
 

Finnish ISP DNA revealed their IPv6 -support for cable networks, but just listed “these devices are compatible” without more technical details.

How to get running (I used OpenWRT and EPC3825 was just a bridge):
* use DHCPv6-client
* request an /56 -prefix
* router will get an /128 -address and I found my /56 -network at the routes (with ip -6 route)
* drop a slice from it (/64) with radvd to your lan side
* remember to configure an IPv6 -firewall
* enjoy

Apr 012015
 

Beware, so you (too) won’t be bitten by this.

If you’re running vanilla KVM with default qemu-kvm -packages in RHEL (CentOS and probably others):

https://git.centos.org/blob/rpms!qemu-kvm!/958e1b8dccf9809600478f316ab641d490881fe7/SOURCES!kvm-rhel-Drop-machine-type-pc-q35-rhel7.0.0.patch;jsessionid=mc5igseasal0axkra6qhd4it

In short, q35 -support was a technical preview which is now dropped from qemu-kvm (qemu-kvm-rhev -only feature from now on), back to “pc”…