Building Python on the NGW100

So for a (hopefully) evil advantage in this robotics competition, I decided to use an NGW100 AVR32-based embedded Linux platform. This board was interesting because it is cheap, and seemingly is powerful enough to let me program the bot in a scripting language. The advantage to using an embedded platform versus a simple microcontroller is that it should be much simpler to program, and allow for cool things such as logging of events and remote control that will help shorten the uninteresting bits of the development cycle.

Anyway, I got the thing and quickly realized that there were no prebuilt packages for python. There are a number of other languages that it does support naturally (such as Lua), but I don’t have any experience with them, and besides Python was what I was aiming for. So, on to cross-compiling Python for the platform.

It turns out that a number of people had already done the heavy lifting in getting Python to cross-compile, including Klaus Reimer (http://www.ailis.de/~k/archives/19-ARM-cross-compiling-howto.html#python), Christopher Lambacher (http://whatschrisdoing.com/blog/2006/10/06/howto-cross-compile-python-25/) and Jo Uthus (http://avr32linux.org/twiki/bin/view/Main/Python). Klaus was the first one (that I could find), however he was targeting Python 2.2.1 and the ARM platform. Christopher modified Klaus’s patch to support for Python 2.5, and Jo rewrote the instructions for the AVR32 platform. What I attempted to do was mash all of these instructions together to get a working build, which appears to be successful (although most likely nowhere near optimized).

If you want, I wrapped up the results of the compiling bit; you can just download this file and skip to step 4:
python-avr.tar

Part 1 – Set up Ubuntu Dapper Drake

I wasn’t too thrilled about this, but Atmel provides a pre-compiled toolchain for Fedora (FC5), OpenSUSE (10.1) and Ubuntu (dapper) only. It is quite possible to build the toolchain by hand, but why?
Update: Atmel provides toolchain packages for more systems, see http://www.atmel.com/dyn/products/tools_card.asp?tool_id=4118 (thanks to Tim)

I decided to cheat and installed dapper drake in a virtual machine (QEMU). This is much slower (I don’t have a CPU that can do KVM), however it let me get a usable environment up quickly and simply. There are some good istructions at https://help.ubuntu.com/community/WindowsXPUnderQemuHowTo, and I won’t repeat those steps here. Just substitute the dapper drake ISO for the Windows one. For reference, the command line I use to start the VM with networking and CDROM on my machine is:

sudo modprobe kqemu
qemu-system-x86_64 -m 384 -net nic -net user -\
  cdrom /home/media/iso/ubuntu-6.06.1-desktop-i386.iso \
  ubuntu-dapper-x86.img

After this, all command are expected to be run in the Dapper Drake environment except as otherwise noted.

Part 2 - Get the build environment set up in Dapper Drake

There are some packages for this, which makes it pretty straightforward. These instructions follow those on http://www.avrfreaks.net/wiki/index.php/Documentation:AVR32_General/Tools_Installation:

Add this Atmel source repository to source.list:

deb http://www.atmel.no/beta_ware/avr32/ubuntu/dapper binary/

Install the BSP:

sudo apt-get update
sudo apt-get install avr32-linux-devel

I ran into the '... but it is not going to be installed' problem, so I also had to add this to my source.list file:

deb http://archive.ubuntu.com/ubuntu dapper main restricted universe multiverse

At this point, the tools should be installed. I also added some other packages so that we will be able to build Python. Unfortunately I didn't take great notes during this time, but the following are a good start:

sudo apt-get install build-essential patch

Part 3 - Cross-compiling Python

Ok, thats all of the setup stuff. Now for the fun bit. First, get the sources for Python 2.5, and apply the patch to add cross-compiling support:

wget http://www.python.org/ftp/python/2.5/Python-2.5.tgz
wget http://whatschrisdoing.com/~lambacck/Python2.5_xcompile.patch
tar -xf Python-2.5.tgz
cd Python-2.5
patch -p1 < ../Python2.5_xcompile.patch

As in everyone else's instructions, the next step is to build the host tools (python uses itself during the build process, so we need to compile a few things so that they can run on the local system):

./configure
make python Parser/pgen
mv python hostpython
mv Parser/pgen Parser/hostpgen
make distclean

The next step is to re-run configure to set up for the cross compile:

CXX=avr32-linux-g++ \
CC=avr32-linux-gcc \
AR=avr32-linux-ar \
RANLIB=avr32-linux-ranlib \
./configure --host=i586-pc-linux-gnu --target=avr32-linux --prefix=$HOME/install

Assuming that this went well, the next step is to build the system:

make CXX=avr32-linux-g++ \
     CC=avr32-linux-gcc \
     AR=avr32-linux-ar \
     RANLIB=avr32-linux-ranlib \
     HOSTPYTHON=./hostpython \
     HOSTPGEN=./Parser/hostpgen \
     BLDSHARED="avr32-linux-gcc -shared"

And then install it:

make install prefix=$HOME/install \
    HOSTPYTHON=./hostpython \
    BLDSHARED="avr32-linux-gcc -shared" \
    CROSS_COMPILE=yes

The only fun bits I got to add were making configure aware that it was targeting a different system, so that it ran the tests against the proper library (this most likely what caused the 'error: stropts.h: No such file or directory' problem that Jo mentions)

Part 4 - Copying to the NGW100
Ok, thanks for staying with me. The installed size of the Python system we just built is over 60MB, so it will need to be placed on an SD card (or possibly USB?). Format the card as ext2 (use mkfs.ext2, be careful), and copy the install directory that should be in your $HOME directory to the card. Put the card into the NGW100. It will be automatically mounted, but Python will expect to be installed in /user/(yourusername)/install/, so the final step (for now) is to re-mount the card in the correct spot:

mkdir /home/(yourusername)
umount /dev/mmcblk0p1
mount /dev/mmcblk0p1 /home/(yourusername)

An improvement over this is probably to install Python to a more sane location (say, /usr/local/), and add it to the path so that it can be used correctly. I'll post an update when I get to this, until then its left as an exercise to the reader :-). Please let me know if you try this.