Saturday, September 1, 2018

Debian and FreeBSD on QEMU with MMX-only CPU

A recent bug (and PR) was opened, aircrack-ng couldn't be built with MMX using a i586 toolchain.

The PR looks pretty simple and just removed some code to allow building with MMX. Building the code will obviously work. There isn't a x86 CPU these days that cannot support anything less than SSE2, which was released after MMX.

So, let's take the opportunity to use qemu to emulate a MMX-only CPU so we can actually test how it runs on such CPU after building it.

If you aren't familiar, qemu is known to emulate a lot of different CPUs, most of the time to play (old) games and using non-x86 CPU. It supports a wide range of x86 CPU too. Using it might sound intimidating but it's actually fairly easy.

Pentium MMX, Pentium 2 (and probably Celeron of that same generation) as well as a few AMD support MMX and do not have any SSE/SSE2 instructions.

Another feature in the Pentium 2 is PAE support. Celeron of the same generation typically don't support PAE. Basically, memory was addressed, like the CPU, with 32 bit, which means a limitation of 4Gb of memory. PAE extended this to 64 bit, allowing more memory on 32 bit CPUs. The OS also has to support it.

The vast majority of Linux distributions these days are built with PAE even if they mention they ship in i386/i486. One exceptions: Debian. However, Debian derivative distributions don't support non-PAE systems. CentOS 6 or 7 may work too but they haven't been tested.

In this example, Ubuntu 18.04 64 bit Desktop was used as a host but it should work on any other currently supported OS. We'll download the latest i386 debian ISO (XFCE or netinst) on the host.

First step is to install qemu and required tools:

apt-get install qemu qemu-system-i386 qemu-tools

Now, let's create a disk of 10Gb called hda:

qemu-img create hda 10G

And finally, we start the VM using qemu:

qemu-system-i386 -cpu pentium2,enforce -cdrom debian-9.5.0-i386-xfce-CD-1.iso -m 2G -show-cursor -net user,hostfwd=tcp::1022-:22 -net nic hda

Let's go over the different options.

-cpu pentium2 will use a typical Pentium 2 system. Details of that system can be found by running man qemu-system-i386. The 'enforce' parameter forces to use the instructions of that CPU only. By default, qemu will run executables on the host CPU as shown here, hence why the use of enforce, to fully emulate how a program would behave on that CPU.

The second option, -cdrom debian-9.5.0-i386-xfce-CD-1.iso will mount the ISO inside the emulated system as a CDROM. This option won't be needed when the system is installed.

-m 2G will give 2Gb of RAM to the emulated system. By default, QEmu allocates 128Mb of RAM, which is definitely not enough for Debian; it requires 139Mb. We just raise it to 1Gb to have some margin when we'll run the desktop.

-show-cursor displays the mouse cursor. Otherwise it is invisible.

-net user,hostfwd=tcp::1022-:22 -net nic is initializing a NIC so we can access the virtual machine SSH server from the host. We'll have to connect using ssh 127.0.0.1 -p 1022

And finally hda, the disk we created previously.

A screen will pop up. The installation process is no different than a regular computer; it will just take much longer while because we are emulating a system.

Restarting the virtual machine later on will use the same command as above minus the -cdrom option.

Here is the output of lscpu on the newly created VM:

user@debian:~$ lscpu
Architecture:          i686
CPU op-mode(s):        32-bit
Byte Order:            Little Endian
CPU(s):                1
On-line CPU(s) list:   0
Thread(s) per core:    1
Core(s) per socket:    1
Socket(s):             1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 5
Model name:            Pentium II (Deschutes)
Stepping:              2
CPU MHz:               2591.957
BogoMIPS:              5183.91
Flags:                 fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 mmx fxsr hypervisor


I bet you never seen a Pentium II overclocked that much :)

Once we got it compiled, to make sure it isn't executed on the host (that supports anything from MMX to AVX2), we will try running the SSE2 version of the executable:

user@debian:~$ aircrack-ng --simd=sse2 -S
Illegal instruction


As expected, it ended up with an Illegal instruction and it means the CPU is fully emulated in the guest (and not executed directly on the host). Anything other than the generic version will result in that error.

So, since the toolchain used is i586, we need to go even further than pentium2 and use the pentium CPU in qemu (P55C). However, we have a serious issue here, there isn't a Linux distribution that supports i586 anymore, not even Debian. If you try to boot it on such platform, it will fail to boot the kernel with a CMOV instruction missing error. So, that would leaves us with Gentoo.

Linux isn't the only option here and BSD supports older CPUs. For example, latest FreeBSD (11.2) still supports i486 CPUs.

The set-up (and results) is identical to what was done for Debian earlier. Installation is straight-forward, exactly the same as you would expect on a real system. And results will be identical.



Tuesday, July 10, 2018

Aircrack-ng 1.3

We're bringing more good stuff in this release. We've been busy fixing bugs left and right, some of them thanks to Coverity Scan, valgrind and other static code analyzers.
We've also refactored some of the code and improved the code quality along the way. We can now successfully build across lots of platforms (Windows, Linux, BSD, OSX) and CPU architectures (x86 and 64 bit, ARM v7, ARM v8, PowerPC, etc)

Aircrack-ng gets a speed bump on pretty much all of the CPU architectures we cover: x86/ARM/PPC. The following graph show the improvements on a Raspberry Pi 3B+.




It may seem that this release is slower than previously (1.2rc3) on non x86 32/64 bit but due to a bug, the cracking speeds were incorrectly calculated. More details can be found in this bug report. On a side note, our benchmark tool is available in build/benchmark.

Here is a benchmark for the NanoPi NEO2:




We had the chance to test Aircrack-ng on a 96-core ARM system ...





... and an IBM Power8 with 160 cores




You can see a significant performance improvement in this release (with the blue line) and you can expect more optimizations in the future, those systems have a lot of potential.

A long-awaited feature has been added: the ability to pause cracking and restart later on. If you intend to pause the cracking at some point in time, start a cracking session with --new-session. You'll be able to restore it using --restore-session. In both cases, the session status is updated every 10 minutes. It works with WEP and WPA/WPA2. Two limitations though: it can only be used with wordlist and they must be files.

Internal changes to aircrack-ng itself and it make is even better than 1.2. It is now back to a single binary. It still compiles the different possible optimizations for a CPU type and loads the fastest optimization based on what the current CPU supports. In the case of x86, the following optimizations will be compiled:
- generic
- SSE2
- AVX
- AVX2

AVX512 is also available but it is strongly recommended to compile it in only if the CPU running aircrack-ng supports it (configure with --with-avx512).

Support for Jemalloc and tcmalloc was added. They used to provide improvements over the system malloc but testing on Ubuntu 16.04 (x86) showed the system malloc is faster in both cases:



Last, but not least for aircrack-ng, it now supports Hashcat HCCAPx files as input file to crack.


Other changes worth noting:

- Airodump-ng adds a new option to override background detection, --background and can now handle GCMP and CCMP-256 encryption.
- dcrack sees a few improvements, mostly internal fixes as well as a few to better handles errors and corner cases
- Documentation improvements: use of hex wordlists, compilation on OSX, experimental tools compilation
- WPE: Logging Response-Identity and display of NETNTLM hash in Hashcat format for HostAPd-WPE and updated building instructions for Freeradius-WPE 3.0.17
- Code reformatted using clang-format. The formatting file has been provided for use with IDE (or through the command line itself using clang-format)
- Typos fixed thanks to codespell
- and much more!

Sunday, April 15, 2018

Aircrack-ng 1.2

It's been way too long since the last stable release.

Compared to the last stable, 1.1, almost 8 years ago, this release has a huge amount of improvements and fixes. The changelog since 1.1 is almost 300 lines long (1200+ commits). Code quality has improved, in parts thanks to Coverity Scan. We now switched to GitHub completely and have a few buildbots (including one for Windows) to test building and run the test suite on a different platforms.

The build system has switched to autotools, which fixes and improves building on a number of different platforms, CPUs and compilers (gcc, clang and Intel).
Aircrack-ng is now a lot faster on recent CPUs (up to 3 times) and a trampoline binary automatically chooses the best executable for your CPU instructions. There is no need to change any of the commands, it is done transparently. Both those changes will make distro package builder's task easier and they won't have to worry about how to build it to be compatible with the most CPUs.

Continuing with Aircrack-ng, it can also output WPA hashes to EWSA and hashcat format for processing with those tools.

There is 802.11 support in airodump-ng with HT40+/HT40- channels and it now displays the rate correctly for 802.11n or 802.11ac Access Points. For those using GPS, it now supports the recent version of GPSd with JSON.

Airmon-ng itself has a number of improvements in chipset/driver detection. The most notables improvements, on top of new chipset/driver detection, is the support for FreeBSD and on Linux, the support for Nexmon driver (monitor mode driver) on the Raspberry Pi 3 (and 0 Wireless) using Kali. Airtun-ng now supports WPA/2.

For the folks following our release candidates, this doesn't bring much compared to rc5, just a few small fixes and adds UTF8 ESSID support in airodump-ng and aireplay-ng. So, if you are already running 1.2rc5, update is merely advised, otherwise, it is highly recommended.


Changelog from rc5:

  • General: Fixed compiling Windows binaries and updated README.md/INSTALLING.
  • General: Fixed commands to install dependencies on Debian/Ubuntu and FreeBSD.
  • General: Added command to install dependencies on Fedora/CentOS/RHEL.
  • General: Removed packages/ directory.
  • General: Added Alpine Linux and Kali Linux buildbots.
  • General: Fixed configure with --with-libpcap-include=/somewhere/include and --with-libpcap-lib=/somewhere/lib.
  • General: Fixed search for ethtool when running as a non-root user.
  • General: Various fixes.
  • Airmon-ng: Fixed mktemp on Alpine Linux.

Tuesday, April 3, 2018

Aircrack-ng 1.2 Release Candidate 5

On top of tons of fixes and improvements everywhere (and on multiple platforms), this release switched to autotools which allows compiling on more platforms. A trampoline binary has been added for Aircrack-ng to automatically select the fastest version for your CPU features. It will also help package maintainers greatly.

A few other notable mentions:
  • Airodump-ng supports setting HT40+/HT40- channels and now displays 802.11n and 802.11ac rates.
  • Created WPA Enterprise WPE patches for HostAPd and Freeradius
  • Support to export to HCCAPx for Hashcat v3.6+
  • Added Airventriloquist-ng, a tool from Caesurus.
  • Airmon-ng supports setting Nexmon devices in/out of monitor mode on Kali


Changelog

  • General: Switching to autotools which allows compiling on more plateforms.
  • General: Updated README.md and INSTALLING files.
  • General: Fixed compilation on a lot of platforms.
  • General: Fixed compilation warnings across platforms and compilers.
  • General: Fixed typos in the tools and in manpages.
  • General: Replace %d/ld with %u/lu for unsigned printf parameters.
  • General: Added option to disable stack protector.
  • General: Improved makefile to get reproducible builds.
  • General: Fixed compilation with OpenSSL 1.1.0.
  • General: Updated radiotap parsing code.
  • General: Updated all URLs to use HTTPS.
  • General: Fixed compilation with libreSSL.
  • General: Added WPS 2.0 test PCAP.
  • General: Do not use stackguard on Windows.
  • General: Fixed warnings on GCC7.
  • General: Improved code quality using Coverity Scan.
  • General: Added badges for Coverity scan and Intel compiler buildbot
  • Aircrack-ng: Use trampoline binary to automatically select fastest executable depending on the CPU
  • Aircrack-ng: Fixed missing include for linecount.
  • Aircrack-ng: Fixed concurrency issues when reading multiple WEP PCAP.
  • Aircrack-ng: Added support for creating HCCAPx file format.
  • Airodump-ng: Get the channel from HT information.
  • Airodump-ng: Detect WPS 2.x.
  • Airodump-ng: Also check current directory for OUI file.
  • Airodump-ng: Fixed writing ESSID to CSV, Kismet CSV and Kismet NetXML files when ESSID gets decloaked and cloaked length was 1.
  • Aireplay-ng: Added deauthentication reason code option.
  • Aireplay-ng: Increase amount of AP to test when running injection test.
  • Airodump-ng: Fixed 802.11a channel hopping list.
  • Airodump-ng: Fix creation of .xor files.
  • Airodump-ng: Added support for HT channels (HT20/HT40-/HT40+).
  • Airodump-ng: Now displaying correct rate for 802.11n or 802.11ac AP.
  • Airmon-ng: Fixed checking for processes.
  • Airmon-ng: Fixed display of "cannot access '/sys/class/ieee80211/': No such file or directory".
  • Airmon-ng: Fixed bashisms.
  • Airmon-ng: Fixed display of specific drivers.
  • Airmon-ng: Fixed display of cards on the sdio bus.
  • Airmon-ng: Now supports nexmon driver on RPi 3 (and 0 Wireless) using Kali Linux.
  • Airmon-ng: Added identification for another realtek chipset and generic Ralink/MT.
  • Airmon-ng: Handle 2 types of rfkill commands and updated unblock text.
  • Airmon-ng: more portable modinfo usage.
  • Airmon-ng: remove grep -P references upon request.
  • Airmon-ng: Do not replace driver name by ?????? when driver is valid.
  • Airgraph-ng: Removed irrelevant comment in README.
  • Airgraph-ng: Handle SSID with double quotes.
  • Airgraph-ng: Fixed parsing OUI file.
  • Airdrop-ng: Updated lorcon2 installation instructions.
  • Besside-ng: Fixed 'wi_read(): No child processes' error.
  • Airdecloak-ng: Fixed segfault due to NULL pointer dereference.
  • osdep: Remove wi_set_channel(1) on open wifi interface (cygwin).
  • osdep: Fixed RAW socket resource leak.
  • Patches: Created WPE patches and documentation for current HostAPd and Freeradius versions.
  • Airodump-ng: Fix incorrect if conditions which always are false.
  • Airodump-ng: Remove useless not NULL check.
  • Airventriloquist: New tool from https://github.com/Caesurus/airventriloquist/
  • dcrack: Fixed indentation.
  • TravisCI: Fixed compilation on OSX.
  • AppVeyor: Added support for AppVeyor, CI for cygwin builds.

Sunday, March 11, 2018

Migration to GitHub

We have been wanting to migrate to GitHub for quite some time. We already had subversion to GitHub synchronization, so some of the work was already done. What was left were tickets.
We now finally migrated completely to GitHub.


It was a lot more complicated than what it sounds because we had tickets on trac and issues + pull requests on GitHub. One of the key migration goals was to replicate the data in such a way
as to near perfectly clone it; all while not bothering our entire user base with GitHub notifications.

Joe Benden did most of the work. The same person who helped us migrating to autotools. I did not expect such a level of professionalism and perfectionism in the work. We probably had a total 10 to 15 test-runs, each taking 1 to 2 days to complete and a lot of discussions to find the best solutions to issues/limitations we came across. Each test run was followed by review and feedback to improve it on the next run.


All the trac tickets are correctly linked to each other and have their attachments. We had some ticket numbers skipped in trac, due to spam some time ago. Luckily, GitHub API allows to skip issues numbers. A lot of fine tuning was done to do it the way we wanted. One of the item was linking tickets that was sometimes done in different ways (#123 or https://trac.aircrack-ng.org/tickets/123 or sometimes hxxp:// trac.aircrack-ng.org/tickets/123). Even URLs were corrected where needed. Along with a bunch of other small details.


Attachment was a big issue since we wanted to keep them and GitHub pretty much only allows text files or pictures in issues. Anything else is out of question. The best solution we could think about was to store them in a repository, GitHub only limiting large attachments which wasn't a problem in this case. On the filesystem, trac doesn't organize them neatly in directories, by names. Instead, it uses some kind of hashing algorithm and it is necessary to look up in the trac database to match them with the tickets and create a script to batch rename them.


Surprisingly, migrated trac tickets look much better than migrated GitHub issues. They look OK but that is due to GitHub API limitations.


One of the decisions was to avoid spamming people with notifications, so all the tickets are created with the Aircrack-ng account. GitHub doesn't allow to set the creation date or the author (there is a mix or email addresses, anonymous author or just username looking reporters) and with almost 2000 tickets, the best solution was to write down the author and date/time in the issues themselves.


Pull requests is tricky, they can't be migrated from one repo to another, period, even when using the API. So, we now have a copy of the existing ones, without link to the code change. So, if anybody who had a PR in the old repository, if you can recreate it in the new one, it would be great. GitHub API has also some kinds of limitation regarding closed PR/issues.


The reason why we decided to create another repository has to do with importing trac tickets and numbering. Matching would have been way too complicated if we appended to the existing repository. On top of that, if it failed somehow, we had no way of going back. It would also have complicated testing significantly.


Finally, while doing the migration, we noticed that the paid accounts don't have as much rate-limiting as the free accounts and the migration went a lot faster than expected. Just a few hours vs 2 days.


We kept the old repository and renamed it aircrack-ng-archive in case we need to look back at some issue/PR history.