2008/06/27

WMU-6500FS - GTK+ 2.10.14

GTK+ 2.10.14

[Home page] [List of dependencies]

Build results: [binary] [list files]

Related stuff:
[XCB] [XCB util] [gperf] [cairo] [pango] [libtiff] [fontconfig]

Build sequence:
dev# cd /usr/local/src
dev# wget http://ftp.gnome.org/pub/gnome/sources/gtk+/2.10/gtk+-2.10.14.tar.bz2
dev# tar xjvf gtk+-2.10.14.tar.bz2
dev# ./configure --prefix=/mnt/C/sys
gdkdrawable-x11.c:32:24: cairo-xlib.h: No such file or directory
Not much of a progress. It seems we need to build the cairo library. Here are the details.

Now we can try to re-configure...
dev# ./configure --prefix=/mnt/C/sys/X11/ --with-x --x-includes=/mnt/C/sys/X11/include/ 
--x-libraries=/mnt/C/sys/X11/lib/
configure: error:
*** Checks for TIFF loader failed. You can build without it by passing
*** --without-libtiff to configure but some programs using GTK+ may
*** not work properly
We can build it with the --without-libtiff option or we have to install the TIFF library.

Once it is done we can re-configure and make:
dev# export LDFLAGS=-L/mnt/C/sys/lib
dev# ./configure --prefix=/mnt/C/sys/X11/ --with-x --x-includes=/mnt/C/sys/X11/include/ 
--x-libraries=/mnt/C/sys/X11/lib/
dev# make
...
 gcc -DHAVE_CONFIG_H -I. -I. -I../.. -DG_LOG_DOMAIN=\"Gdk\" -DGDK_COMPILATION 
-I../.. -I../../gdk -I../../gdk -DGDK_PIXBUF_DISABLE_DEPRECATED 
-DG_DISABLE_CAST_CHECKS -pthread -I/usr/include/glib-2.0 
-I/usr/lib/glib-2.0/include -I/usr/include/pango-1.0 -I/usr/include/cairo 
-I/mnt/C/sys/X11/include/ -g -O2 -Wall -MT gdkdrawable-x11.lo -MD -MP -MF 
.deps/gdkdrawable-x11.Tpo -c gdkdrawable-x11.c  -fPIC -DPIC -o 
libs/gdkdrawable-x11.o
gdkdrawable-x11.c:32:24: cairo-xlib.h: No such file or directory
It seems some modules try to directly include the cairo header files as they were placed at an include path, but that is not the case.
dev# export CXXFLAGS=-I/mnt/C/sys/X11/include/cairo
dev# ./configure --prefix=/mnt/C/sys/X11/ --with-x --x-includes=/mnt/C/sys/X11/include/ 
--x-libraries=/mnt/C/sys/X11/lib/
dev# make
In file included from gdkdrawable-x11.c:32:
/mnt/C/sys/X11/include/cairo/cairo-xlib.h:97:3: #error Cairo was not compiled with support for the xlib backend
It seems we have to build the XCB library (Here is the build sequence) and then re-build the cairo library properly.

Now we are ready to proceed with GTK:
dev# ./configure --prefix=/mnt/C/sys/X11/ --with-x --x-includes=/mnt/C/sys/X11/include/ 
--x-libraries=/mnt/C/sys/X11/lib/
...
configure:31390: gcc -o conftest -I/mnt/C/sys/X11/include/cairo -Wall -I/mnt/C/sys/X11/include
-L/mnt/C/sys/lib conftest.c -L/mnt/C/sys/X11/lib -lXext -lXrender -lX11  -lintl >&5
/mnt/C/sys/X11/lib/libxcb-xlib.so.0: undefined reference to `backtrace_symbols'
/mnt/C/sys/X11/lib/libxcb-xlib.so.0: undefined reference to `backtrace'
It seems that xcb is not properly configured, it needs a backtrace support (not provided by uClibc). So we have to rebuild libxcb with the HAVE_BACKTRACE configure hack.

Let's try it again, do not worry, we are almost there ;-)
dev# make clean
dev# ./configure --prefix=/mnt/C/sys/X11/ --with-x --x-includes=/mnt/C/sys/X11/include/ 
--x-libraries=/mnt/C/sys/X11/lib/
dev# make
...
/bin/sh ../libtool --mode=link gcc  -I/mnt/C/sys/X11/include/cairo -Wall  
-L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib -o gtk-query-immodules-2.0 queryimmodules.o 
libgtk-x11-2.0.la ../gdk-pixbuf/libgdk_pixbuf-2.0.la ../gdk/libgdk-x11-2.0.la 
gcc -I/mnt/C/sys/X11/include/cairo -Wall -o .libs/gtk-query-immodules-2.0 
queryimmodules.o  -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib  ./.libs/libgtk-x11-2.0.so 
/usr/local/src/gtk+-2.10.14/gdk/.libs/libgdk-x11-2.0.so -L/mnt/C/sys/X11/lib 
/usr/lib/libatk-1.0.so ../gdk-pixbuf/.libs/libgdk_pixbuf-2.0.so 
../gdk/.libs/libgdk-x11-2.0.so /usr/lib/libpangocairo-1.0.so /usr/lib/libcairo.so 
/usr/lib/libpixman-1.so /usr/lib/libpangoft2-1.0.so /usr/lib/libpango-1.0.so 
...
/usr/local/src/gtk+-2.10.14/gdk/.libs/libgdk-x11-2.0.so: undefined reference to `cairo_xlib_surface_create_for_bitmap'
/usr/local/src/gtk+-2.10.14/gdk/.libs/libgdk-x11-2.0.so: undefined reference to `cairo_xlib_surface_create'
/usr/local/src/gtk+-2.10.14/gdk/.libs/libgdk-x11-2.0.so: undefined reference to `cairo_xlib_surface_set_size'
collect2: ld returned 1 exit status
It seems that the static linker did not find some cairo symbols.
Let's look to gdk to be sure what cairo symbols are referenced:
dev# nm /usr/local/src/gtk+-2.10.14/gdk/.libs/libgdk-x11-2.0.so | grep cairo_xlib_surface_create
         U cairo_xlib_surface_create
         U cairo_xlib_surface_create_for_bitmap
Let's look at the cairo library why they are missing...
dev# nm /mnt/C/sys/X11/lib/libcairo.so | grep cairo_xlib_surface_create
...
0005b7d3 T cairo_xlib_surface_create
0005b834 T cairo_xlib_surface_create_for_bitmap
...
It seems the symbols are there. If they was not there it would be due to incorrect cairo configuration, namely missing --with-x configure script option (see this mail). However I think that in this regard we have the cairo configured properly.

Not sure what is actually going on here, maybe it is the order of libraries passed to static linker (the fact that the libcairo.so goes after the libgdk-x11-2.0.so):
gcc -I/mnt/C/sys/X11/include/cairo -Wall -o .libs/gtk-query-immodules-2.0 queryimmodules.o
-L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib ./.libs/libgtk-x11-2.0.so 
/usr/local/src/gtk+-2.10.14/gdk/.libs/libgdk-x11-2.0.so -L/mnt/C/sys/X11/lib 
/usr/lib/libatk-1.0.so ../gdk-pixbuf/.libs/libgdk_pixbuf-2.0.so 
../gdk/.libs/libgdk-x11-2.0.so /usr/lib/libpangocairo-1.0.so /usr/lib/libcairo.so 
/usr/lib/libpixman-1.so /usr/lib/libpangoft2-1.0.so /usr/lib/libpango-1.0.so 
/usr/lib/libpng12.so /mnt/C/sys/X11/lib/libxcb-render-util.so /mnt/C/sys/X11/lib/libxcb-render.so 
/mnt/C/sys/X11/lib/libpixman-1.so /usr/lib/libfontconfig.so /usr/lib/libfreetype.so 
-lz /usr/lib/libexpat.so /mnt/C/sys/X11/lib/libXinerama.so 
/mnt/C/sys/X11/lib/libXrandr.so /mnt/C/sys/X11/lib/libXext.so 
...
In any case the following hack "fixed" the problem:
dev# make clean
dev# export LDFLAGS="-L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib /mnt/C/sys/X11/lib/libcairo.so"
dev# ./configure --prefix=/mnt/C/sys/X11/ --with-x --x-includes=/mnt/C/sys/X11/include/ 
--x-libraries=/mnt/C/sys/X11/lib/
dev# make
Test installation:

Now we are ready to test the installation:
dev# /mnt/C/sys/X11/bin/gtk-demo
...
(gtk-demo:25684): Pango-CRITICAL **: No fonts found:
This probably means that the fontconfig
library is not correctly configured. You may need to
edit the fonts.conf configuration file. More information
about fontconfig can be found in the fontconfig(3) manual
page and on http://fontconfig.org
...
The application seems as follows:

The fonts are compiled and deployed but not installed correctly, when we build the fontconfig then we can perform the following test:
dev# fc-list
dev#
We have to create a link to /usr/share/ directory to make fonts visible:
dev# ln -s /mnt/C/sys/X11/lib/X11/fonts/ /usr/share/fonts
dev# fc-cache
dev# fc-list
Luxi Serif:style=Regular
LucidaBright:style=Italic
Utopia:style=Bold Italic
Serto Malankara:style=Regular
...
The startup time of the gtk-demo application (as well as for example simple xclock) increased significantly. It seems that number of fonts greatly influences the x-apps' startup times. When we (according to this source) limit the number of linked fonts (e.g. only OTF and TTF) (see alsho this tip), the situation is slightly better:
box# rm /usr/share/fonts
box# mkdir /usr/share/fonts
box# ln -sn $XORG_PREFIX/lib/X11/fonts/OTF /usr/share/fonts/X11-OTF &&
box# ln -sn $XORG_PREFIX/lib/X11/fonts/TTF /usr/share/fonts/X11-TTF
Still the x-client applications are relatively slow, turning the ssh compression helped a bit, but more evaluation and benchmarking is needed...

libxcb-1.1

[XCB dependencies]

Build results: [binary] [list files]

Build sequence: We can download XCB via the git_xorg.sh script (for details see XOrg related post) or we can do it by hand. As it is described here. First we have to download and build the protocol headers and pthread stubs (on linux it should be just stubs and nothing more).
dev# cd /usr/local/src
dev# wget http://xcb.freedesktop.org/dist/xcb-proto-1.1.tar.gz
dev# tar xzvf xcb-proto-1.1.tar.gz
dev# ./configure --prefix=/mnt/C/sys/X11
dev# make
dev# cp xcb-proto.pc /mnt/C/sys/X11/lib/pkgconfig/
dev# cd ..
dev# wget http://xcb.freedesktop.org/dist/libpthread-stubs-0.1.tar.bz2
dev# tar xjzf libpthread-stubs-0.1.tar.bz2
dev# cd libpthread-stubs-0.1
dev# ./configure --prefix=/mnt/C/sys
dev# make
dev# make install
Now we can proceed to build the XCB itself:
dev# cd /usr/local/src
dev# wget http://xcb.freedesktop.org/dist/libxcb-1.1.tar.bz2
dev# tar xjvf libxcb-1.1.tar.bz2
dev# cd libxcb-1.1
In order to configure the it was necessary to set the PKG_CONFIG_PATH properly. Maybe it is a good idea to have it set on every boot...
For now we just set it one-shot:
dev# export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/mnt/C/sys/lib/pkgconfig/:/mnt/C/sys/X11/lib/pkgconfig/
Another problem was with backtrace symbols. During the static linking everything seem ok, but the runtime linker (when an application is launched) complains about missing symbols:
/mnt/C/sys/X11/lib/libxcb-xlib.so.0: undefined reference to `backtrace_symbols'
/mnt/C/sys/X11/lib/libxcb-xlib.so.0: undefined reference to `backtrace'
It seems that backtrace support is not implemented in uClibc 0.9.27. Regarding this ticket it should be present since version 0.9.28.
For now I decided to re-configure the library not to use the backtrace support at all. It is relatively simple - we just backup and alter the configure.ac file as follows:
dev# cat configure.ac | grep backtrace
AC_CHECK_HEADER([execinfo.h], [AC_DEFINE(HAVE_BACKTRACE,1,[Has backtrace*() needed for retrieving stack traces])])
dev# cp configure.ac{,.backup}   
dev# sed -i 's/AC_CHECK_HEADER(\[execinfo.h\],\[AC_DEFINE(HAVE_BACKTRACE/#AC_CHECK_HEADER(\[execinfo.h\],\[AC_DEFINE(HAVE_BACKTRACE/g' configure.ac
dev# cat configure.ac | grep backtrace
#AC_CHECK_HEADER([execinfo.h], [AC_DEFINE(HAVE_BACKTRACE,1,[Has backtrace*() needed for retrieving stack traces])])
One more time we re-configure and rebuild the library:
dev# make clean
dev# ./configure --prefix=/mnt/C/sys/X11
dev# make
dev# make install
Note: Regarding to the missing backtrace support we can also consider deleting the /usr/include/execinfo.h file instead of altering the configuration files.
dev# mv /usr/include/execinfo.h{,.deleted}
I do not know if it is correct solution but it seems logically the file contains just declarations of those backtrace related functions and the fact that is it not present fixes the behavior of configure scripts.
For more info see Problems compiling pdns using uClibc.

gperf-3.0.3

Build results: [binary] [list files]

Build sequence:
dev# cd /usr/local/src
dev# wget http://ftp.gnu.org/gnu/gperf/gperf-3.0.3.tar.gz
dev# tar xzvf gperf-3.0.3.tar.gz
dev# cd gperf-3.0.3
dev# ./configure --prefix=/mnt/C/sys/
dev# make
dev# make install

xcb-util-0.2

Dependencies: [gperf] [xcb]

Build results: [binary] [list files]

Build sequence:
dev# cd /usr/local/src
dev# wget http://xcb.freedesktop.org/dist/xcb-util-0.2.tar.bz2
dev# tar xjvf xcb-util-0.2.tar.bz2
dev# ./configure --prefix=/mnt/C/sys/X11
dev# make
dev# make install

cairo-1.6.4

[Home page]

Dependencies: [xcb]

Build results: [binary] [list files]

Build sequence:
dev# cd /usr/local/src
dev# wget http://cairographics.org/releases/cairo-1.6.4.tar.gz
dev# tar xzvf cairo-1.6.4.tar.gz
dev# cd cairo-1.6.4
Simply calling ./configure --prefix=/mnt/C/sys/X11 does not work with the GTK library, we have to configure the cairo with the Xlib support.
dev# ./configure --prefix=/mnt/C/sys/X11 --with-x -x-includes=/mnt/C/sys/X11/include/ 
--x-libraries=/mnt/C/sys/X11/lib/ --enable-xlib=yes --enable-xlib-xrender=yes --enable-xcb=yes
cairo (version 1.6.4) will be compiled with the following surface backends:
  image:         yes (always builtin)
  Xlib:          yes
  Xlib Xrender:  yes
  Quartz:        no (CoreGraphics framework not found)
  Quartz-image:  no (disabled, use --enable-quartz-image to enable)
  XCB:           yes
  Win32:         no (requires a Win32 platform)
  OS2:           no (disabled, use --enable-os2 to enable)
  PostScript:    yes
  PDF:           yes
  SVG:           yes
  glitz:         no (disabled, use --enable-glitz to enable)
  BeOS:          no (disabled, use --enable-beos to enable)
  DirectFB:      no (disabled, use --enable-directfb to enable)

the following font backends:
  FreeType:      yes
  Win32:         no (requires a Win32 platform)
  Quartz:        no (CoreGraphics framework not found)

the following features:
  PNG functions: yes

and the following debug options:
  gcov support:  no
  test surfaces: no
  ps testing:    no (requires ghostscript)
  pdf testing:   no (requires poppler-glib >= 0.8.0)
  svg testing:   no (requires librsvg-2.0 >= 2.15.0)

using CFLAGS:
-I/usr/include/freetype2 -I/usr/include/libpng12 -I/mnt/C/sys/X11/include -I/mnt/C/sys/XOrg/include -I/mnt/C/sys/XOrg/include -I/mnt/C/sys/XOrg/include/pixman-1 -Wall -Wsign-compare -Werror-implicit-function-declaration -Wpointer-arith -Wwrite-strings 
-Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wnested-externs -Wpacked -Wswitch-enum -Wmissing-format-attribute 
-Wno-unused-parameter -Wno-long-long -Winline -fno-strict-aliasing

*** The XCB backend is still under active development and
*** is included in this release only as a preview. It does not
*** fully work yet and incompatible changes may yet be made
*** to XCB-backend specific API.

dev# make
dev# make install

libtiff 3.8.2

Build results: [binary] [list files]

Build sequence:
dev# cd /usr/local/src
dev# wget ftp://ftp.remotesensing.org/pub/libtiff/tiff-3.8.2.tar.gz
dev# tar xzvf tiff-3.8.2.tar.gz
dev# ./configure --prefix=/mnt/C/sys
dev# make
dev# make install

pango-1.21.3

[Home page]

Dependencies: [cairo]

Build results: [binary] [list files]

Build sequence:
dev# cd /usr/local/src
dev# wget http://ftp.gnome.org/pub/GNOME/sources/pango/1.21/pango-1.21.3.tar.bz2
dev# tar xjvf pango-1.21.3.tar.bz2
dev# cd pango-1.21.3
dev# ./configure --prefix=/mnt/C/sys/X11 --with-x --x-includes=/mnt/C/sys/X11/include/ 
--x-libraries=/mnt/C/sys/X11/lib/
dev# make
dev# make install

fontconfig-2.6.0

[Home page]

Build results: [binary] [list files]

Build sequence: We have to configure it to look at the /mnt/C/sys/X11/etc/fonts/ directory (instead of default /etc/fonts/) since the /etc directory is read-only (we are not even able to create a symlink).
dev# cd /usr/local/src
dev# wget http://fontconfig.org/release/fontconfig-2.6.0.tar.gz
dev# tar xzvf fontconfig-2.6.0.tar.gz
dev# cd fontconfig-2.6.0
dev# ./configure --prefix=/mnt/C/sys/X11 --with-confdir=/mnt/C/sys/X11/etc/fonts/
dev# make 
dev# make install

Read more...

2008/06/26

A curious AIX crash

Few days ago we had to solve an odd crash on AIX platform (powerpc chip). The company I am working in as a developer uses mainly MS Windows as a development platform but supports Linux as well as Unix platforms (namely AIX, HP-UX and Solaris).
The crash was only on AIX and we was not able to replicate it on any other platform. Our Unix experience is not such as it should be so it is always painful process to solve it when such a platform specific bug appears.

What I am going to describe here is too technical and boring for a majority of people, and on the other side maybe too trivial for those UNIX developers more experienced than me, but I hope there could be some (evenly non-practiced) developers out there who could find this information helpful or at least interesting.

The crash we experienced was easily replicable and was always at the same place - there was a totally innocent looking member function, its code is something like the following:
virtual T const& get( unsigned int id ) const
{    <---------------------------- The crash was right here before anything happen
 if ( id < static_cast( m_values.size() ) )
  return get_internal( id );
 throw ExceptionNotFound( LOCARG, id, count() );
 UNREACHABLE_RETURN( T() );
}
The object instance was ok, the id parameter was in range <0, m_values.size() ), there was no apparent reason why it should crash. So it was necessary to sink at the assembly level. The code for this function started as follows (PowerPC Stack Attacks helped us to interpret the code):
0x39927250 (get(unsigned int) const)      7c0802a6       mflr   r0              Function call prologue routine
0x39927254 (get(unsigned int) const+0x4)  93e1fffc        stw   r31,-4(r1)      (seems like the registers are 
0x39927258 (get(unsigned int) const+0x8)  93c1fff8        stw   r30,-8(r1)       associated with individual local
0x3992725c (get(unsigned int) const+0xc)  93a1fff4        stw   r29,-12(r1)      variables)
0x39927260 (get(unsigned int) const+0x10) 9381fff0        stw   r28,-16(r1)
0x39927264 (get(unsigned int) const+0x14) 9361ffec        stw   r27,-20(r1)
0x39927268 (get(unsigned int) const+0x18) 9341ffe8        stw   r26,-24(r1)
0x3992726c (get(unsigned int) const+0x1c) 90010008        stw   r0,0x8(r1)
0x39927270 (get(unsigned int) const+0x20) 9421ed30       stwu   r1,-4816(r1)  -> Here was the place of the segfault
0x39927274 (get(unsigned int) const+0x24) 83e2017c        lwz   r31,0x17c(r2)    
0x39927278 (get(unsigned int) const+0x28) 83c20180        lwz   r30,0x180(r2)    What the coed does is that a new stack frame 
...                                                                              is "allocated" and the stack pointer register 
                                                                                 (r1) is updated

                                                                                 "stwu" instruction stores the current content of 
                                                                                 the r1 register at address computed as (r1-offset) 
                                                                                 and decrements the r1 by offset at the same time
                                                                                 (offset is 4816 bytes in this case)
Initially we thought that due to the stack corruption the the stack pointer (r1) contain incorrect address (pointing to a read-only memory page) and so the write operation to that address failed.
Here is the stack pointer register value:
(dbx) registers
  $r0:0x3d1dd854  $stkp:0x400f5cb0   $toc:0x3a7071dc    $r3:0x40146fb0
  ...
When we looked to the address 0x400f5cb0 we could see an address of the previous frame, at that address was the previous one and over and over. This way was possible to traverse down to the stack and everything seemed consistent. The stack did not seem to be corrupted at all.
(dbx) 0x400f5cb0/4X
0x400f5cb0:  400f5d00 00000000 3d2b2854 00000000      (   80 bytes)
(dbx) 0x400f5d00/4X
0x400f5d00:  400f7010 00000000 3d2b228c 00000000      ( 4880 bytes)
(dbx) 0x400f7010/4X
0x400f7010:  400f70e0 00000000 3d2b410c 00000000      (  208 bytes)
(dbx) 0x400f70e0/4X
0x400f70e0:  400fd560 00000000 3d287294 00000000      (25728 bytes)
(dbx) 0x400fd560/4X
0x400fd560:  401026e0 00000000 3d289bd0 00000000      (20864 bytes)
(dbx) 0x401026e0/4X
0x401026e0:  40102750 00000000 3d28a0cc 00000000      (  112 bytes)
The interesting part was the stack frame size (4816 bytes) - the fact that the memory at the current stkp address seemed to contain a consistent data while the attempt to write to an address computed as 0x400f5cb0-4816 == 0x400f49e0 results in the segmentation violation brought us to an idea that it could be the stack overflow.

The documentation at the IBM Systems Information Center describes the following structure of call stack and related data structures (for whole documentation see this page):
     *      +-----------------------+
     *      | page alignment 2      |
     *      | [8K-4K+PTH_FIXED-a1]  |
     *      +-----------------------+
     *      | pthread ctx [368]     |
     *      +-----------------------+<--- pthread->pt_attr
     *      | pthread attr [112]    |
     *      +-----------------------+ <--- pthread->pt_attr
     *      | pthread struct [960]  |
     *      +-----------------------+ <--- pthread
     *      | pthread stack         |         pthread->pt_stk.st_limit
     *      |   |[96K+4K-PTH_FIXED] |
     *      |   V                   |
     *      +-----------------------+ <--- pthread->pt_stk.st_base
     *      | RED ZONE [4K]         |
     *      +-----------------------+ <--- pthread->pt_guardaddr
     *      | pthread key data [4K] |
     *      +-----------------------+ <--- pthread->pt_data
     *      | page alignment 1 (a1) |
     *      | [<4K]                 |
     *      +-----------------------+
     
     The RED ZONE on this illustration is called the Guardpage.

The decimal number of guard pages to add to the end of the pthread stack is n, which overrides the attribute values 
that are specified at pthread creation time. If the application specifies its own stack, no guard pages are created. 
The default is 0 and n must be a positive value.
The guardpage size in bytes is determined by multiplying n by the PAGESIZE. Pagesize is a system-determined size.
The relatively small default call stack size (96K) supported our hypothesis. We needed just to find a proof that the address being written to is actually the RED ZONE.

Here is the memory dump retrieved from the dbx along with my comments (please note that the address orientation is reversed relatively to previous schema):
(dbx) 0x400f3fe8/2048X
0x400f3fe8:  00000000 00000000 00000000 00000000  ------> Bottom of another thread's call stack
0x400f3ff8:  00000001 40135cac 40135cac 40135cac  ----> 
0x400f4008:  f0286708 f0286708 f0286708 f0286708  ----> Guard page (aka "Red zone")
0x400f4018:  f0286708 f0286708 f0286708 f0286708        Read/write protected memory region
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~   (4096 bytes)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
0x400f49c8:  f0286708 f0286708 f0286708 f0286708
0x400f49d8:  f0286708 f0286708 f0286708 f0286708  <---.  BOOM (Segmentation fault)
0x400f49e8:  f0286708 f0286708 f0286708 f0286708      |  Attempt to write to the 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |  "red zone" (guard page
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |   
0x400f4fe8:  f0286708 f0286708 f0286708 f0286708      |
0x400f4ff8:  f0286708 f0286708 00000000 00000000   ---+-> Top of the current thread's call stack
0x400f5008:  00000000 00000000 00000000 00000000      |   
0x400f5018:  00000000 00000000 00000000 00000000      |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  |
0x400f5988:  00000000 00000000 00000000 00000000      | 
0x400f5c78:  00000000 00000000 400f5cd0 00000000      | 4816 bytes
0x400f5c88:  00000000 00000000 00000000 00000000      |
0x400f5c98:  400fd5b8 401ad028 4019fa58 40165f90    --'
0x400f5ca8:  3d4263e8 3d9981e8 400f5d00 00000000  <---.
0x400f5cb8:  3d2b2854 00000000 401ad028 3d9bd578      |
0x400f5cc8:  00000000 00000000 400f5d20 00000000      |
0x400f5cd8:  39a87794 00000000 00000000 00000000      |
0x400f5ce8:  40165cc4 00000000 00000000 00000000    --'   80 bytes 
0x400f5cf8:  00000000 00000000 400f7010 00000000  <---.
0x400f5d08:  3d2b228c 00000000 401ad028 00000000      |
0x400f5d18:  40165f90 00000000 400f6ff0 00000000      |
0x400f5d28:  39a74c24 00000000 00000000 00000000      |
0x400f5d38:  40165cc0 00000000 00000000 00000000      |  4880 bytes
0x400f5d48:  00000000 00000000 00000000 00000000      |
0x400f5d58:  00000000 00000000 00000000 00000000      | 
0x400f5d68:  00000000 00000000 00000000 00000000      . Linked list of current thread's stack frames
0x400f5d78:  00000000 00000000 00000000 00000000      . 
(dbx)
Those almost 5 kilobytes stack frames were caused by Exception objects residing on stack. Our exception implementation holds relatively big array of wide characters (to avoid heap allocations in case of a failure) and so in some cases the exceptions (though not thrown, just prepared on stack to be thrown in case something fails) consume substantial portion of stack memory and finally cause the stack overflow error.

Just now I am quite happy that we have found and solved this problem and at the same time have learned something new.

It is necessary to a priori know the maximum call stack size and the default values for AIX system are relatively low. Since the architecture has no special support for the call stack, the only available diagnostics is a Guard page (aka Red zone). It is implemented as N read/write protected memory pages mapped above the call stack limit (please, node that regarding the documentation this functionality is disabled by default). A Stack overflow event then (if you are lucky) causes the "normal" segmentation violation - the same as as any other invalid access to a protected memory region and so it is relatively difficult to diagnose for a developers with not much platform-specific experience.

Post note: There is an interesting document about method called "Automatically Extensible Discontinuous Stacks" - a method eliminating the need of the stack size pre-knowledge.

Read more...

2008/06/25

WMU-6500FS - X.Org 7.1 (updated)

Just for fun I am trying to build X.Org and it seems it won't be such a problem to finish it. There are many issues remaining to be solved but at least for now I am able to run couple of simple X-applications.
Update: I have not finished the X server yet, but have built X.Org libraries, so now it is possible to build X client applications.
Build results: [binary (82MB)] [list files]

Following image demonstrates the xcalc application running on the box acting as X-client of the XMing server running on my PC :-)



Dependencies: [m4-1.4.11] [libxslt-1.1.22] [xmlto 0.0.20] [Mesa 7.0.3] [rsh (fake with ssh alias)]
Info sources:
[X Window System Environment]
[Modular Developer's Guide]
[Modular X development using the git trees]
[Embedded Linux Graphics Quick Reference Guide]


Build sequence:
Create source directory and download git_xorg script (will be used to get all xorg modular components from git repository)
dev# cd /usr/local/src
dev# mkdir Xorg
dev# cd Xorg
dev# wget "http://www.x.org/wiki/ModularDevelopersGuide?action=AttachFile&do=get&target=git_xorg.sh" -O git_xorg.sh
dev# chmod a+x git_xorg.sh
dev# echo "do_dir xorg font \"\${font}\"        # this line added to get fonts" >> git_xorg.sh
Customize git_xorg script
dev# nano git_xorg.sh
... you can for example remove all xf86-video-* drivers and related stuff, remove do_dir . xcb "${xcb}", add do_dir xorg font "${font}" in order to download fonts, etc.
dev # diff git_xorg.sh git_xorg.sh.orig
50c50,65
<       xf86-input-vmmouse xf86-input-void"
---
>       xf86-input-vmmouse xf86-input-void xf86-video-apm\
>       xf86-video-ark xf86-video-ast xf86-video-ati\
>       xf86-video-chips xf86-video-cirrus xf86-video-cyrix\
>       xf86-video-dummy xf86-video-fbdev xf86-video-glide\
>       xf86-video-glint xf86-video-i128 xf86-video-i740\
>       xf86-video-impact xf86-video-imstt xf86-video-intel\
>       xf86-video-mga xf86-video-neomagic xf86-video-newport\
>       xf86-video-nsc xf86-video-nv xf86-video-rendition\
>       xf86-video-s3 xf86-video-s3virge xf86-video-savage\
>       xf86-video-siliconmotion xf86-video-sis xf86-video-sisusb\
>       xf86-video-sunbw2 xf86-video-suncg14 xf86-video-suncg3\
>       xf86-video-suncg6 xf86-video-sunffb xf86-video-sunleo\
>       xf86-video-suntcx xf86-video-tdfx xf86-video-tga\
>       xf86-video-trident xf86-video-tseng xf86-video-v4l\
>       xf86-video-vesa xf86-video-vga xf86-video-via\
>       xf86-video-vmware xf86-video-voodoo xf86-video-wsfb"
129c144
< do_dir xorg font "${font}"        # this line added to get fonts
---
> do_dir . xcb "${xcb}"
Now we can start downloading the source:
dev# ./git_xorg.sh
And now we can try to run the build (we have not solved any dependencies yet, I decided to solve it along the way).
The first one is the m4 preprocessor:
dev# PATH=/mnt/C/sys/X11/bin:$PATH ./util/modular/build.sh /mnt/C/sys/X11
    ...
    autom4te: need GNU m4 1.4 or later: /usr/bin/m4
For info how to get and build the m4 preprocessor see this post.
Then there is xcb build error (in case one have not removed it from the git_xorg.sh):
dev# PATH=/mnt/C/sys/X11/bin:$PATH ./util/modular/build.sh /mnt/C/sys/X11
    ...
     configure: error: XCB requires xsltproc.
    /mnt/C/sys/X11/lib/libxcb-xlib.so: undefined reference to `backtrace_symbols'
    /mnt/C/sys/X11/lib/libxcb-xlib.so: undefined reference to `backtrace'
It seems that uClibc does not implement the backtrace related stuff. As a solution we can re-configure the xcb:
dev# cp xcb/libxcb/configure.ac{,.backup}
dev# nano xcb/libxcb/configure.ac
From the configure.ac comment out the following line:
    AC_CHECK_HEADER([execinfo.h], [AC_DEFINE(HAVE_BACKTRACE,1,[Has backtrace*() ...
And then we can continue the build process, the next error is in the libXcomposite:
dev# PATH=/mnt/C/sys/X11/bin:$PATH ./util/modular/build.sh -r xcb/libxcb /mnt/C/sys/X11
    ...
    make[3]: Entering directory `/usr/local/src/Xorg/lib/libXcomposite/man'
    WARNING: configure did not find xmlto, cannot create Xcomposite.man without it
For info how to get and build the xmlto see this post.
The next problem was in xconsole application:
dev# PATH=/mnt/C/sys/X11/bin:$PATH ./util/modular/build.sh -r lib/libXcomposite /mnt/C/sys/X11
...
make[1]: Entering directory `/usr/local/src/Xorg/app/xconsole'
...
In file included from xconsole.c:185:
/usr/include/sys/stropts.h:1:21: stropts.h: No such file or directory
uClibc does not implement streams and so the fact that the stropts.h header file is missing is understandable.
Commenting out the include in xconsole.c file semms to work. For more info about this issue see this source.
After the xconsole.c modification we can try to continue the build process:
dev# PATH=/mnt/C/sys/X11/bin:$PATH ./util/modular/build.sh -r app/xconsole /mnt/C/sys/X11
...
Building app module component xdriinfo...
...
configure: error: cannot find GL library - make sure Mesa or other OpenGL package is installed
For info how to get and build Mesa library see this post.
After that we have to add the new include and library dirs to the environment:
dev# export LDFLAGS="-lGL -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib -lX11"
dev# export CFLAGS=-I/mnt/C/sys/include
And then we can continue.
dev# PATH=/mnt/C/sys/X11/bin:$PATH ./util/modular/build.sh -r app/xdriinfo -m /usr/local/src/Mesa-7.0.3 /mnt/C/sys/X11
...
Building app module component xsm...
...
configure: error: No rsh like program found
For more details about this issue see this message.

At the time I had not even the dropbear/ssh installed I decided to install it and solve the lack of rsh binary with the following fake:
dev# cd /mnt/C
dev# wget http://mgb111.pradnik.net/addons/servers-ssh/dropbear-ssh-sshd-050.3.tgz
dev# tar xzvf dropbear-ssh-sshd-050.3.tgz
dev# which ssh
/mnt/C/sys/bin/ssh
Now we can create rsh names alias for the ssh binary.
dev# ln -s /mnt/C/sys/bin/{s,r}sh
dev# rsh
Dropbear multi-purpose version 0.50
Make a symlink pointing at this binary with one of the following names:
'dropbear' - the Dropbear server
'dbclient' or 'ssh' - the Dropbear client
'dropbearkey' - the key generator
'dropbearconvert' - the key converter
'scp' - secure copy
dev# ls -l /mnt/C/sys/bin/rsh
lrwxrwxrwx  2 root root 18 Jun 25 14:42 /mnt/C/sys/bin/rsh -> /mnt/C/sys/bin/ssh*
dev# ls -l /mnt/C/sys/bin/ssh
lrwxrwxrwx  1 root root 13 Jun 17 14:34 /mnt/C/sys/bin/ssh -> dropbearmulti*
The symlink method does not work. It is because of the fact that dropbear is the multiapplication and so the ssh binary is actually a symlink pointing to the binary called dropbearmulti binary. For such case the szmlink name matters, it can be one of the names listed above.
We have another possibility, we can create a shell script calling ssh along with all its command line arguments:
dev# echo "\
> #!/bin/bash
> ssh \$1 \$2 \$3 \$4 \$5 \$6 \$7 \$8 \$9" > /mnt/C/sys/bin/rsh
dev# cat /mnt/C/sys/bin/rsh
#!/bin/bash
ssh $1 $2 $3 $4 $5 $6 $7 $8 $9
dev# chmod a+x /mnt/C/sys/bin/rsh
dev# rsh
Dropbear client v0.50
Usage: ssh [options] [user@]host [command]
Options are:
...
Now back to the Xorg source directory and continue the building process:
dev# cd /usr/local/src/Xorg
dev# PATH=/mnt/C/sys/X11/bin:$PATH ./util/modular/build.sh -r app/xsm -m /usr/local/src/Mesa-7.0.3 /mnt/C/sys/X11
Building xserver module ...
...
checking for GL... configure: error: Package requirements (glproto >= 1.4.9 gl >= 7.1.0) were not met.
Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively you may set the GL_CFLAGS and GL_LIBS environment variables
to avoid the need to call pkg-config.  See the pkg-config man page for
more details.
For now we can skip the xserver as well as the driver (we are interested in xclients mainly) modules:
dev# PATH=/mnt/C/sys/X11/bin:$PATH ./util/modular/build.sh -r font/util -m /usr/local/src/Mesa-7.0.3 /mnt/C/sys/X11
...
xcb module component util does not exist, skipping.
Wed Jun 25 13:21:59 MDT 2008

***** Skipped components (not available) *****
 xcb/util
For now we are done with X.Org libraries, except the xserver, drivers and xcb. Maybe we come back to that later...

Read more...

2008/06/20

WMU-6500FS - mpd-0.13.2

mpd-0.13.2

[Home page]

Build results: [binary] [list files]

Dependencies: [libao] [libid3tag] [flac] [libmad] [audiofile] [faad2] [libmikmod] [libogg-1.1.3] [libvorbis-1.2.0]

Build sequence:
dev# cd /usr/local/src
dev# wget http://www.musicpd.org/uploads/files/mpd-0.13.2.tar.bz2
dev# tar xjvf mpd-0.13.2.tar.bz2
dev# cd cd mpd-0.13.2
dev# export PKG_CONFIG_PATH=/mnt/C/sys/lib/pkgconfig:$PKG_CONFIG_PATH
dev# ./configure --prefix=/mnt/C/sys --enable-ao --enable-mvp --with-libFLAC=/mnt/C/sys
...
########### MPD CONFIGURATION ############

 Playback Support:
 libao support .................enabled
 OSS support ...................enabled
 ALSA support ..................disabled
 JACK support ..................disabled
 OS X support ..................disabled
 PulseAudio support ............disabled
 Media MVP support .............enabled
 Shout streaming support .......disabled

 File Format Support:
 ID3 tag support ...............enabled
 mp3 support ...................enabled
 Ogg Vorbis support ............enabled
   using tremor.................no
 FLAC support ..................enabled
 OggFLAC support ...............enabled(FLAC 1.1.3)
 Wave file support .............enabled
 MP4/AAC support ...............enabled
 Musepack (MPC) support ........disabled
 MOD support ...................disabled

 Other features:
 libsamplerate support .........disabled
 Zeroconf support ..............disabled

##########################################
Some dependency libraries are still not found succesfully, hopefully I will solve it later...
dev# make
dev# make install

libao-0.8.8

[Home page]

Build results: [binary] [list files]

Build sequence:
dev# cd /usr/local/src
dev# wget http://downloads.xiph.org/releases/ao/libao-0.8.8.tar.gz
dev# tar xzvf libao-0.8.8.tar.gz
dev# cd libao-0.8.8
dev# ./configure --prefix=/mnt/C/sys
dev# make
dev# make install

libid3tag-0.15.1b

[Home page]

Build results: [binary] [list files]

Build sequence:
dev# cd /usr/local/src
dev# wget ftp://ftp.mars.org/pub/mpeg/libid3tag-0.15.1b.tar.gz
dev# tar xzvf libid3tag-0.15.1b.tar.gz
dev# cd libid3tag-0.15.1b
dev# ./configure --prefix=/mnt/C/sys
dev# make
dev# make install

libmad-0.15.1b

[Home page]

Build results: [binary] [list files]

Build sequence:
dev# cd /usr/local/src
dev# wget ftp://ftp.mars.org/pub/mpeg/libmad-0.15.1b.tar.gz
dev# tar xzvf libmad-0.15.1b.tar.gz
dev# cd libmad-0.15.1b
dev# ./configure --prefix=/mnt/C/sys
dev# make
dev# make install

flac-1.2.1

[Home page]

Build results: [binary] [list files]

Build sequence:
dev# cd /usr/local/src
dev# wget "http://downloads.sourceforge.net/flac/flac-1.2.1.tar.gz?modtime=1189961849&big_mirror=0"
dev# tar xzvf flac-1.2.1.tar.gz
dev# cd flac-1.2.1
dev# ./configure --prefix=/mnt/C/sys
dev# make
dev# make install

audiofile-0.2.6

[Home page]

Build results: [binary] [list files]

Build sequence:
dev# cd /usr/local/src
dev# wget http://www.68k.org/~michael/audiofile/audiofile-0.2.6.tar.gz
dev# tar xzvf audiofile-0.2.6.tar.gz
dev# cd audiofile-0.2.6
dev# ./configure --prefix=/mnt/C/sys
dev# make
dev# make install

faad2-2.6.1

[Home page]

Build results: [binary] [list files]

Dependencies: [Perl >= 5.005] [GNU m4 >= 1.4]

Build sequence:
dev# cd /usr/local/src
dev# wget http://downloads.sourceforge.net/faac/faad2-2.6.1.tar.gz
dev# tar xzvf faad2-2.6.1.tar.gz
dev# cd faad2
dev# autoreconf -vif
Perl 5.005 required--this is only version 5.00405, stopped at /usr/share/autoconf/Autom4te/Channels.pm line 59.
BEGIN failed--compilation aborted at /usr/share/autoconf/Autom4te/Channels.pm line 59.
BEGIN failed--compilation aborted at /usr/share/autoconf/Autom4te/ChannelDefs.pm line 20.
BEGIN failed--compilation aborted at /usr/bin/autoreconf line 42.
In order to generate the configure script we have to upgrade perl installation - fortunately we can use JoKeR's binary (version 5.8.8).
dev# cd /mnt/C
dev# wget http://mgb111.pradnik.net/addons/soft-lang/perl-5.8.8-080509.tar.bz2
dev# tar xjvf perl-5.8.8-080509.tar.bz2
The old one is itill in front of the PATH sequence, now we have to disable the previous one:
dev# perl -v
This is perl, version 5.004_05 built for i686-linux
...
# mv /usr/bin/perl{,-5.004_05}
# ln -s {/mnt/C/sys/bin/,/usr/bin/}perl
# perl -v
This is perl, v5.8.8 built for i386-linux-thread-multi
...
Ok, now we are ready to proceed with the build process:
# autoreconf -vif
# ./configure --prefix=/mnt/C/sys
# make
# make install

libmikmod-3.2.0-beta2

[Home page]

Build results: [binary] [list files]

Build sequence:
dev# cd /usr/local/src
dev# wget http://mikmod.raphnet.net/files/libmikmod-3.2.0-beta2.tar.bz2
dev# tar xjvf libmikmod-3.2.0-beta2.tar.bz2
dev# cd libmikmod-3.2.0-beta2
dev# ./configure --prefix=/mnt/C/sys
dev# make
dev# make install

Read more...

2008/06/18

WMU-6500FS - xml stuff

xmlto 0.0.20

Build results: Not actually deployed to the box, just built as an Xorg dependency...

Dependencies: [sgml-base 1.26] [docbook-xsl-1.74] [docbook-xml-4.2]

Build sequence:
I think it is not necessary to deploy xmlto to the box, co will configure it locally (it will not go to the /mnt/C/sys directory).
dev# wget http://cyberelk.net/tim/data/xmlto/stable/xmlto-0.0.20.tar.bz2
dev# tar xjvf xmlto-0.0.20.tar.bz2
dev# cd xmlto-0.0.20
dev# ./configure
dev# make
I/O error : Attempt to load network entity http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl
...
Variable $SGML_CATALOG_FILES not set
It seems there is no support for XML catalogs on the system. According to this source we can fix it in a following way:
dev# mkdir -p /usr/local/etc/xml
dev# xmlcatalog --noout --create /usr/local/etc/xml/catalog
dev# mkdir -p /usr/local/share/docbook/xsl
dev# cd /usr/local/share/docbook/xsl
dev# wget
"http://downloads.sourceforge.net/docbook/docbook-xsl-1.74.0.tar.bz2?modtime=1212403365&big_mirror=0"
-O docbook-xsl-1.74.tar.bz2
dev# tar xjvf docbook-xsl-1.74.tar.bz
I would have liked to install it in /usr/local/etc, but according to the source only /etc works
dev# mkdir -p /etc/xml
dev# xmlcatalog --noout --add 'nextCatalog' '' 'file:///usr/local/share/docbook/docbook-xsl-1.73.2/catalog.xml' --create /etc/xml/catalog
Now it is time to install the catalog:
dev# cd /usr/local/share/docbook/xsl/docbook-xsl-1.74.0/
dev# ./install.sh
WARNING: This install script is meant to be run as a non-root
         user, but you are running it as root.

Are you sure you want to continue? [No] Y
...
Create /root/.resolver/CatalogManager.properties file? [Yes] Y
...
Add /etc/xml/catalog to /root/.resolver/CatalogManager.properties? [Yes] Y
...
Add /usr/local/share/docbook/xsl/docbook-xsl-1.74.0/catalog.xml
to /root/.resolver/CatalogManager.properties file? [Yes] Y
...
To source your shell environment for this distribution, type the
following:
. /usr/local/share/docbook/xsl/docbook-xsl-1.74.0/.profile.incl
dev# . /usr/local/share/docbook/xsl/docbook-xsl-1.74.0/.profile.incl
Now back to the xmlto installation:
dev# cd /usr/local/src/xmlto-0.0.20
dev# make clean
dev# ./configure
dev# make
I/O error : Attempt to load network entity http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd
/usr/local/src/xmlto-0.0.20/./doc/xmlif.xml:4: warning: failed to load external entity "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"
   "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
There is a progress, but we are not done withe XML catalogs yet:
dev# cd /usr/local/share/docbook
dev# wget http://www.oasis-open.org/docbook/xml/4.2/docbook-xml-4.2.zip
There is a bug in the busybox version of unzip - for details see this message.
As a workaround we can pre-create the directories before the actual unzipping:
dev# mkdir -p docbook-xml-4.2/ent
dev# unzip docbook-xml-4.2.zip -d docbook-xml-4.2
Now we can register the new XML catalog:
dev# xmlcatalog --noout --add nextCatalog '' file:///usr/local/share/docbook/docbook-xml-4.2/catalog.xml --create /etc/xml/catalog
And finally finish the xmlto installation:
dev# cd /usr/local/src/xmlto-0.0.20
dev# make clean
dev# ./configure
dev# make
dev# make install

libxslt - 1.1.22

Build results: [binary] [list files]

Dependencies: [libxml2-2.6.30]

Build sequence:
dev# cd /ust/local/src
dev# wget ftp://ftp.gnome.org/pub/GNOME/sources/libxslt/1.1/libxslt-1.1.22.tar.bz2
dev# tar xjvf libxslt-1.1.22.tar.bz2
dev# cd libxslt-1.1.22
dev# ./configure --prefix=/mnt/C/sys
dev# make
dev# make install

libxml2 2.6.30

Build results: [binary] [list files]

Build sequence:
dev# wget ftp://ftp.gnome.org/pub/GNOME/sources/libxml2/2.6/libxml2-2.6.30.tar.bz2
dev# tar xjvf libxml2-2.6.30.tar.bz2
dev# cd libxml2-2.6.30
dev# ./configure --prefix=/mnt/C/sys
dev# make
dev# make install

sgml-base 1.26

Build results: Not actually deployed to the box

Build sequence:
dev# cd /usr/local/src
dev# wget http://ftp.de.debian.org/debian/pool/main/s/sgml-base/sgml-base_1.26.tar.gz
dev# tar xzvf sgml-base_1.26.tar.gz
dev# cd sgml-base-1.26
dev# make
dev# make install

Read more...

WMU-6500FS - file 4.24 (unix command)

Build results: [binary] [list files]

Home page

Build sequence:
dev# wget ftp://ftp.astron.com/pub/file/file-4.24.tar.gz
dev# tar xzvf file-4.24.tar.gz
dev# cd file-4.24
dev# ./configure --prefix=/mnt/C/sys
dev# make
dev# make install

Read more...

WMU-6500FS - m4 1.4.11

Build results: Not actually deployed to the box

Build sequence:
dev# wget http://ftp.gnu.org/gnu/m4/m4-1.4.11.tar.bz2
dev# tar xjvf m4-1.4.11.tar.bz2
dev# cd m4-1.4.11
dev# ./configure
dev# make
dev# make install
Xorg configure looks directly to /usr/bin, it can probably be re-configured, but I do not want to solve it correctly now, here is a workaround I have used:
dev# ln -s /usr/local/bin/m4 /usr/bin

Read more...

2008/06/14

WMU-6500FS - lsof 4.78



Build results: [binary] [files list]

In the Web UI there is a possibility to create/delete/resize/format individual disk partitions (WebUI/Storage configuration/Disk utility) however sometimes it is not possible, e.g. format operation fails with error "Device is busy".
It seems there is some process using the drive (holding some file handles or so). The lsof command (meaning "LiSt Open Files") is a tool which can be used to determine what files are being held by which processes.

Info sources: [Device is busy error] [lsof tool] [Vic Abell's Home Page]

Build sequence:
Download and extract:
dev# wget ftp://ftp.cert.dfn.de/pub/tools/admin/lsof/lsof_4.78.tar.gz
dev# tar xzvf lsof_4.78.tar.gz
dev# cd lsof_4.78
dev# tar xvf lsof_4.78_src.tar
dev# cd lsof_4.78_src
Configure settings
(I left all default values, it is here just for you to see what the possible settings are).
dev# ./Configure linux
Do you want to take inventory (y|n) [y]? y
Do you want to customize (y|n) [y]? y
Customizing ...
Enable HASSECURITY (y|n) [n]?
Disable WARNINGSTATE? (y|n) [n]?
Enable HASKERNIDCK (y|n) [n]?
Do you want to rename machine.h to machine.h.old and replace it with
new_machine.h (y|n) [y]? y
dev# make
In order to find what processes hold a file on the C partition, you can use the following command:
box# lsof | grep mnt/C

Read more...

2008/06/13

WMU-6500FS - cvstrac 2.0.1

Build results: [binary] [files list]
Info sources: [cvstrac site] [cvstrac colored diffs]

Build sequence:
dev# cd /usr/local/src
dev# wget http://www.cvstrac.org/cvstrac-2.0.1.tar.gz
dev# tar xzvf cvstrac-2.0.1.tar.gz
dev# cd cvstrac-2.0.1
dev# cp linux-gcc.mk Makefile
Customize Makefile
(build dir need not be necessary the same as source dir, you have to alter SRCDIR accordingly. Also the INSTALLDIR can be for example set to /mnt/C/sys/var/www/cgi-bin in case you want to run it as CGI)
dev# nano Makefile

SRCDIR = .

INSTALLDIR = /mnt/C/sys/bin
If you have not set the include and lib directories to sys you have to modify also the main.mk:
dev# nano main.mk

XTCC = $(TCC) $(CFLAGS) -I. -I$(SRCDIR) -I/mnt/C/sys/include

$(TCC) -o $(APPNAME) $(OBJ) $(LIBSQLITE) -L/mnt/C/sys/lib
Make the application
dev# make
Error
Undefined reference to sqlite3StrICmp and similar symbols. It seems that 2.0.1 release used a non-public part of SQLite interface and in a recent version of SQLite the referenced functions disappeared.
Fortunately there is a patch 2.0.1-3 solving the problem:
dev# cd ..
dev# wget http://ftp.de.debian.org/debian/pool/main/c/cvstrac/cvstrac_2.0.1-3.diff.gz
dev# gunzip cvstrac_2.0.1-3.diff.gz
dev# patch -p0 < cvstrac_2.0.1-3.diff
dev# cd cvstrac-2.0.1
Or alternatively you can download the development version directly from their CVS:
dev# cvs -d :pserver:anonymous@cvstrac.org:/cvstrac login
dev# cvs -d :pserver:anonymous@cvstrac.org:/cvstrac checkout cvstrac
When prompted for a password, enter "anonymous". For more info see [cvstrac download].

We are ready to build all the three available variants:
dev# make
dev# make install
dev# make APPNAME=svntrac
dev# make install APPNAME=svntrac
dev# make APPNAME=gittrac
dev# make install APPNAME=gittrac
Now we have the application(s) installed and can launch it. Root can neither initialize a project nor launch the server (actually it is possible but you have to launch it in chroot jail), for more info see [installation] and [chroot jail].
dev# su <user>
dev# gittrac init /mnt/C/sys/var/git <project>
dev# gittrac server 8008 /mnt/C/sys/var/git/ <project> &

Read more...

2008/06/10

WMU-6500FS - Boost 1.35

Build results: [binary] [files list]
Note: besides the DSO files this archive contains even all the boost header files and documentation (not actually necessary on the box).

Build sequence:
dev# wget http://downloads.sourceforge.net/boost/boost_1_35_0.tar.bz2?modtime=1206795398&big_mirror=0
dev# tar xjvf boost_1_35_0.tar.bz2
dev# cd boost_1_35_0
dev# ./configure --prefix=/mnt/C/sys
dev# make
dev# make install
The boost library names are more specific (contain boost version and compiler version) than some software demand. I do not know whether there should be libboost_*-mt.so symlinks, maybe I have not configured boost library properly, as a quick fix I have created the links manually:
dev# cd /mnt/C/sys/lib
dev# for i in `ls ./libboost_*-gcc33-mt-1_35.so.1.35.0`; do ln -s `basename $i` ${i/%-gcc33-mt-1_35.so.1.35.0/-mt.so}; done

Read more...

2008/06/09

WMU-6500FS - GIT 1.5.5.3

Build results: [binary] [files list]

Build sequence:
dev# wget http://kernel.org/pub/software/scm/git/git-1.5.5.3.tar.gz
dev# tar xzvf git-1.5.5.3.tar.gz
dev# ./configure --prefix=/mnt/C/sys
dev# make
dev# make install
Deployment hack
(touch some files in order to include them to the package)
dev# find /mnt/C/sys/share/ -path "*git*" -type f -exec touch {} \;
dev# find /mnt/C/sys/lib/ -path "*Git*" -type f -exec touch {} \;
Transfer Subversion DB to GIT
Dependencies: subversion
Info source: [git-svn intro]
box# svnserve -d -r /mnt/C/sys/var/svn/<project>
box# cd /mnt/C/sys/var/git/<project>
box# git svn init -t tags -b branches -T trunk svn://localhost/<project>
(or simply: box# git svn init svn://localhost/<project> )
box# git svn fetch
Error 1
perl: Can't resolve symbol 'apr_initialize'
can be replicated as follows:
box# perl -e "use SVN::Core"
Inspect dependencies:
dev# ldd /mnt/C/sys/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi/auto/SVN/_Core/_Core.so
...
libapr-1.so.0 => /mnt/C/sys/apache2/lib/libapr-1.so.0 (0x00000000)
...
dev# nm /mnt/C/sys/apache2/lib/libapr-1.so.0 | grep apr_initialize
00013c18 T apr_initialize
Everything seem to be ok, maybe the libraries are loaded in a wrong order...
# export LD_PRELOAD=/mnt/C/sys/apache2/lib/libapr-1.so.0
... error 1 fixed, but there is one more ...
Error 2
perl: Can't resolve symbol 'svn_swig_pl_get_current_pool'
can be replicated as follows:
box# perl -e "use SVN::Wc; SVN::Wc::adm_open3(undef, '.', 0, 0);"
Another pre-load needed:
box# export LD_PRELOAD=$LD_PRELOAD:/mnt/C/sys/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi/auto/SVN/_Core/_Core.so
Problem fixed - not saying that the fix is a proper solution, but I use the export only for repo conversion.

Read more...

WMU-6500FS - DidiWiki 0.5



Build results: [binary] [files list]

Build sequence:
dev# wget http://www.openzaurus.org/download/3.5.4/sources/didiwiki-0.5.tar.gz
dev# tar xzvf didiwiki-0.5.tar.gz
dev# cd didiwiki-0.5
dev# ./configure --prefix=/mnt/C/sys
dev# make
dev# make install

Read more...

WMU-6500FS - StreamRipper 1.63.1

libogg-1.1.3

Build results: [binary] [files list]

Build sequence:
dev# wget http://downloads.xiph.org/releases/ogg/libogg-1.1.3.tar.gz
dev# tar xzvf libogg-1.1.3.tar.gz
dev# cd libogg-1.1.3
dev# ./configure --prefix=/mnt/C/sys
dev# make
dev# make install

libvorbis-1.2.0

Build results: [binary] [files list]

Build sequence:
dev# wget http://downloads.xiph.org/releases/vorbis/libvorbis-1.2.0.tar.bz2
dev# tar xjvf libvorbis-1.2.0.tar.bz2
dev# cd libvorbis-1.2.0
dev# ./configure --prefix=/mnt/C/sys
dev# make
dev# make install

StreamRipper 1.63.1

Build results: [binary] [files list]

Build sequence:
dev# wget http://downloads.sourceforge.net/streamripper/streamripper-1.63.1.tar.gz
dev# tar xzvf streamripper-1.63.1.tar.gz
dev# cd streamripper-1.63.1
dev# ./configure --prefix=/mnt/C/sys
dev# make
dev# make install

Read more...

WMU-6500FS - Trac 0.11rc1



Build results: No deployment needed (installation is performed directly to the box).

Dependencies: subversion, python, apache & mod_python, SQLite

Info sources: [Trac installation guide] [Subversion-Trac howto] [Trac & ModPython]

setuptools

box# cd /mnt/C
box# wget http://peak.telecommunity.com/dist/ez_setup.py
box# python ez_setup.py

Genshi

box# python /mnt/C/sys/bin/easy_install --prefix=/mnt/C/sys Genshi

Trac

box# python /mnt/C/sys/bin/easy_install --prefix=/mnt/C/sys Trac
Prepare project
box# mkdir /mnt/C/sys/var/trac
box# mkdir /mnt/C/sys/var/trac/<project>
box# trac-admin /mnt/C/sys/var/trac/<project> initenv
... and fill in all project info (point it to project's subversion erpository e.g. /mnt/C/sys/var/svn/<project> etc.)

Test standalone trac http daemon
box# tracd --port 8000 /mnt/C/sys/var/trac/<project>
box# lynx http://localhost:8000
Administration
Edit apache configuration
box# nano /mnt/C/sys/apache2/conf/httpd.conf
Add following:
<Location /projects/<project>>
  SetHandler mod_python
  PythonInterpreter main_interpreter
  PythonHandler trac.web.modpython_frontend
  PythonOption TracEnv /mnt/C/sys/var/trac/<project>
  PythonOption TracUriRoot /projects/<project>
</Location>

<Location /projects/<project>/login>
  AuthType Basic
  AuthName "<project>"
  AuthUserFile /mnt/C/sys/var/trac/<project>/.htpasswd
  Require valid-user
</Location>
Create password file
box# htpasswd -c /mnt/C/sys/var/trac/<project>/.htpasswd <user>
box# trac-admin /mnt/C/sys/var/trac/<project> permission add admin TRAC_ADMIN
Initialize web administration
box# nano /mnt/C/sys/var/trac/<project>/conf/trac.ini
Add following:
[components]
webadmin.* = enabled

Read more...

WMU-6500FS - Subversion 1.4.6

Configured with python and perl language bindings.



Build results: [binary] [files list]
Info sources: [Installing svn - quick guide] [svn with apache]

Here you can find the following info:
One thing you do need, though, is SSL capability. So you must use the --with-ssl option to configure and configure must find the Openssl library installed. If you don't need a language other than English, you can simplify things a little more with the --disable-nls option.
Without --disable-nls svn binary had locales related problems (undefined __ctype_* symbols)
dev# wget http://subversion.tigris.org/downloads/subversion-deps-1.4.6.tar.gz
dev# tar xvzf subversion-deps-1.4.6.tar.gz
dev# wget http://subversion.tigris.org/downloads/subversion-1.4.6.tar.gz
dev# tar xvzf subversion-1.4.6.tar.gz
dev# cd subversion-1.4.6
dev# ./configure --prefix=/mnt/C/sys --enable-shared --without-neon 
--with-swig --with-ssl --disable-nls --with-apxs=/mnt/C/sys/apache2/bin/apxs 
--with-apr=/mnt/C/sys/apache2 --with-apr-util=/mnt/C/sys/apache2 
--with-python=/mnt/C/sys/bin/python2.5 --with-perl=/mnt/C/sys/lib/perl5/5.8.8 --enable-maintainer-mode
dev# make
Generate and build swig python bindings
dev# make swig-py
Libtool error: unrecognized option '-DLINUX=2'
=> used swig version 1.3.27 instead of 1.3.29, rebuilt both swig and svn and problem was fixed.
Libtool error: command i386-linux-gcc not found
=> following hack helped to solve it
dev# ln -s /usr/bin/i386-linux-uclibc-gcc i386-linux-gcc
Generate and build swig perl bindings
dev# make swig-pl
Install
dev# make install
dev# make install-swig-py
dev# make install-swig-pl
Make sure that whatever directory the bindings got installed is accessible. You can use PYTHONPATH variable or the following approach:
dev# echo /mnt/C/sys/lib/svn-python > /mnt/C/sys/lib/python2.5/site-packages/subversion.pth
Test installation
You can try subversion tests (but for me it did not succeed completely and I have not solved it yet):
dev# make check-swig-py
dev# make check-swig-pl
At least following should succeed:
dev# python -c "from svn import client"
dev# cd tools/examples
dev# python svnlook.py <some-svn-repository>
Hack for the deployment (in order to be included in generated tarball) - without it some files are older and do not get to the resulting package:
dev# find /mnt/C/sys/lib/perl5/ -path *SVN* -type f -exec touch {} \;

Read more...

WMU-6500FS - HTop 0.8

Build results: [binary] [files list]



Build sequence:
dev# wget http://mesh.dl.sourceforge.net/sourceforge/htop/htop-0.8.tar.gz 
dev# tar xzvf htop-0.8.tar.gz
dev# cd htop-0.8
In order to be able to configure it on chroot-ed environment, we have to fake the /proc filesystem
dev# echo > /proc/stat
dev# echo > /proc/meminfo
dev# ./configure --prefix=/mnt/C/sys
Build error
It seems to me there is an inconsistency in DEBUG/ASSERT definition. There is a function called just in asserts (not necessary disabled in release build) but defined just in case of DEBUG build.
Quick fix: In HashTable.c comment out all asserts with HashTable_isConsistent
dev# make
dev# make install

Read more...

WMU-6500FS - Apache 2.2.8

Apache 2.2.8

Build results: [binary] [files list]

Build sequence
dev# wget http://apache.mirror.superhosting.cz/httpd/httpd-2.2.8.tar.bz2
dev# tar xvjf httpd-2.2.8.tar.bz2
dev# cd httpd-2.2.8
dev# ./configure --prefix=/mnt/C/sys/apache2 --enable-dav --enable-so --enable-maintainer-mode --enable-ssl
dev# make
dev# make install
Configuration
Info sources: [Optimizing apache]
First, we have to configure box to run apache after reboot: ... in file "/mnt/C/sys/etc/env-ng":
# if 1 then installed apache 2.2 version will be running
APACHE22=1
Then we can try to modify default apache behavior regarding to number of processes and threads (due to limited RAM of the box):
In file /usr/links/storage-C/sys/apache2/conf/extra/httpd-mpm.conf I have changed the following settings:
StartServers          1
MinSpareServers       1
MaxSpareServers       1
MaxClients            1
MaxRequestsPerChild 5000

mod_python 3.3.1

Apache module for python implemented handlers
Build results: [binary] [files list]
Info sources: [Getting ModPython Working] [Trac ModPython]

Build sequence
dev# wget http://apache.mirror.superhosting.cz/httpd/modpython/mod_python-3.3.1.tgz
dev# tar xvzf mod_python-3.3.1.tgz
dev# cd mod_python-3.3.1/
dev# ./configure --prefix=/mnt/C/sys --with-apxs=/mnt/C/sys/apache2/bin/apxs 
--with-python=/mnt/C/sys/bin/python2.5
found /usr/bin/flex, we'll use this. Use --with-flex to specify another. 
checking flex version... configure: WARNING: Flex version 2.5.4 found. 
Version 2.5.31 or greater is required. You can generally ignore this 
warning unless you need to regenerate psp_parser.c from psp_parse.l. 
If you do need regenerate psp_parser.c, use --with-flex to specify the 
location of the correct flex version. See the README for more information.

dev# make
dev# sudo make install
Create a "/mnt/C/sys/var/www/test/.htaccess" file in your subdirectory and place in it:
AddHandler mod_python .py
PythonHandler mptest
PythonDebug On
Configure www directory:
box# cd /mnt/C/sys/var/www
box# mkdir test
( box# chown  test )
box# cd test
box# echo "AddHandler mod_python .py
> PythonHandler mptest
> PythonDebug On" > .htaccess
Modify apache configuration:
box# nano /mnt/C/sys/apache2/conf/httpd.conf
In httpd.conf change following:
  User nobody
  Group nobody

  DocumentRoot "/mnt/C/sys/var/www"

  <Directory "/mnt/C/sys/var/www">
    ...
  </Directory>

  ScriptAlias /cgi-bin/ "/mnt/C/sys/var/cgi-bin/"

  <Directory "/mnt/C/sys/var/cgi-bin">
    ...
  </Directory>
... and add following (at Load modules section):
  LoadModule python_module      modules/mod_python.so
... and add following (after the root directory definition):
  <Directory "/mnt/C/sys/var/www/test">
    AllowOverride FileInfo
  </Directory>
Create test python handler "/mnt/C/sys/apache2/htdocs/test/mptest.py":
box# echo "from mod_python import apache
>
> def handler(req):
>     req.log_error('handler')
>     req.content_type = 'text/plain'
>     req.send_http_header()
>     req.write('It is working!!!\n')
>     return apache.OK
> " > /mnt/C/sys/apache2/htdocs/test/mptest.py
Test configuration (following command should show page containing string "It is working!!!"):
box# lynx localhost/test/mptest.py

Read more...

WMU-6500FS - SWIG 1.3.27

Build results: Actually not deployed to the box, needed just for subversion python & perl bindings.

Subversion documentation recommends version 1.3.27, while the latest version is currently 1.3.35. Read ./subversion/bindings/swig/INSTALL in order to build the bindings.

"In particular, pay attention to the version of SWIG which can be required (1.3.0 comes with pre-generated wrappers, though). Do not install 1.3.28 or 1.3.29. 1.3.27 seems to work fine (Although not in all cases. The subversion users mailing lists suggest going back to 1.3.25, which may be necessary)."

Build sequence:
Selected Berlin mirror for download...
dev# cd /usr/local/src
dev# wget http://dfn.dl.sourceforge.net/sourceforge/swig/swig-1.3.27.tar.gz
dev# tar xzvf swig-1.3.27.tar.gz
dev# cd swig-1.3.27
dev# ./configure --with-python=/mnt/C/sys/bin/python2.5
dev# make
dev# make install

Read more...

WMU-6500FS - SQLLite 3.5.8

Build result: [binary] [files list]
Alternatively you can download the JoKeR's one [here]

Build sequence
dev# wget http://www.sqlite.org/sqlite-amalgamation-3.5.8.tar.gz
dev# tar xzvf sqlite-amalgamation-3.5.8.tar.gz
dev# cd sqlite-3.5.8/
To avoid some problems with testThreadLockingBehavior - finishing with SIG32 - disable thread safety. Without this settings I have experienced a (silent) crash during the CVSTrac initialization (someone is experiencing the same).
dev# ./configure --prefix=/mnt/C/sys --enable-threadsafe=no
dev# make
dev# make install

Read more...

WMU-6500FS - Deployment

For easy and convenient deployment I have created a simple script creating an appropriate package based on a timestamp. It deploys all the files newer than a ".start_time" file created just before an installation process. The script is very simple, just in case one is lazy to create his own, here is the [link] to download it.

I got used to following sequence of operations whenever I am installing a new piece of software:
  1. download software package
  2. dev# cd /usr/local/src
    dev# wget <sw-src-uri>
    dev# tar xzvf <sw-src-package>
    
  3. configure and build
  4. dev# cd <sw-src-dir>
    dev# ./configure --prefix=/mnt/C/sys <possibly-other-options>
    dev# make
    
  5. create timestamp, install and create resulting binary package
  6. dev# echo > /mnt/C/.start_time
    dev# make install
    dev# cd /mnt/C
    dev# ./filopack.sh <package-name>
    
  7. upload resulting package to the box
  8. deb# scp <dev-root>/mnt/C/<package-name>.tgz root@storage:/mnt/C/
    
  9. install the resulting package on the box
  10. box# cd /mnt/C
    box# tar xzvf <package-name>.tgz
    

Script usage examples

Help and usage
dev# ./filopack.sh -h
usage: filopack.sh [OPTIONS] <package-name>

 Based on previously created timestamp (.start_time file) generates
 archive <package-name>.<fmt> along with file list <package-name>.lst.

 Available options:
         -o <fmt> ... archive format ('tar', 'tgz', 'tbz2', 'lst' currently supported)
                 ('lst' just generates .lst file and prints filenames)
         -p ... prefix path ('sys/' by default)
         -f ... filter ('-newer .start_time' by default, '' performs no filtering)
         -r ... reuse the previously generated filename list (.lst file)
         -v ... prints version
         -h ... prints this help message
Deployment of the file (unix command) - tar.bz2 archive is created by default.
dev# echo > /mnt/C/.start_time
dev# make install
dev# cd /mnt/C
dev# ./filopack.sh file-4.24
-rwxr-xr-x root/root     39355 2008-06-16 14:46:50 sys/bin/file
-rwxr-xr-x root/root    285972 2008-06-16 14:46:49 sys/lib/libmagic.so.1.0.0
lrwxrwxrwx root/root         0 2008-06-16 14:46:49 sys/lib/libmagic.so.1 -> libmagic.so.1.0.0
lrwxrwxrwx root/root         0 2008-06-16 14:46:49 sys/lib/libmagic.so -> libmagic.so.1.0.0
-rwxr-xr-x root/root       803 2008-06-16 14:46:49 sys/lib/libmagic.la
-rw-r--r-- root/root    396680 2008-06-16 14:46:49 sys/lib/libmagic.a
-rw-r--r-- root/root     16176 2008-06-16 14:46:51 sys/share/man/man1/file.1
-rw-r--r-- root/root      7555 2008-06-16 14:46:51 sys/share/man/man3/libmagic.3
-rw-r--r-- root/root     16504 2008-06-16 14:46:51 sys/share/man/man4/magic.4
-rw-r--r-- root/root   1627584 2008-06-16 14:46:50 sys/share/file/magic.mgc
-rw-r--r-- root/root      3547 2008-06-16 14:46:50 sys/include/magic.h
Deployment of the same package - this time the file list is reused and gzip archive is used fir resulting archive.
dev# ./filopack.sh -o tgz -r file-4.24
-rwxr-xr-x root/root     39355 2008-06-16 14:46:50 sys/bin/file
-rwxr-xr-x root/root    285972 2008-06-16 14:46:49 sys/lib/libmagic.so.1.0.0
lrwxrwxrwx root/root         0 2008-06-16 14:46:49 sys/lib/libmagic.so.1 -> libmagic.so.1.0.0
lrwxrwxrwx root/root         0 2008-06-16 14:46:49 sys/lib/libmagic.so -> libmagic.so.1.0.0
-rwxr-xr-x root/root       803 2008-06-16 14:46:49 sys/lib/libmagic.la
-rw-r--r-- root/root    396680 2008-06-16 14:46:49 sys/lib/libmagic.a
-rw-r--r-- root/root     16176 2008-06-16 14:46:51 sys/share/man/man1/file.1
-rw-r--r-- root/root      7555 2008-06-16 14:46:51 sys/share/man/man3/libmagic.3
-rw-r--r-- root/root     16504 2008-06-16 14:46:51 sys/share/man/man4/magic.4
-rw-r--r-- root/root   1627584 2008-06-16 14:46:50 sys/share/file/magic.mgc
-rw-r--r-- root/root      3547 2008-06-16 14:46:50 sys/include/magic.h
Archive of the Mesa libraries is created (timestamp filtering is disabled and prefix pattern is used for the file selection).
dev# ./filopack.sh -p 'sys/lib/lib[OG]*' -f '' Mesa-7.0.3
lrwxrwxrwx root/root         0 2008-06-16 14:07:09 sys/lib/libGL.so -> libGL.so.1
lrwxrwxrwx root/root         0 2008-06-16 14:07:09 sys/lib/libGL.so.1 -> libGL.so.1.5.070003
-rwxr-xr-x root/root  20101208 2008-06-16 14:07:09 sys/lib/libGL.so.1.5.070003
-rw-r--r-- root/root         0 2008-06-16 14:37:35 sys/lib/libGL.so.1.lst
lrwxrwxrwx root/root         0 2008-06-16 14:07:24 sys/lib/libGLU.so -> libGLU.so.1
lrwxrwxrwx root/root         0 2008-06-16 14:07:24 sys/lib/libGLU.so.1 -> libGLU.so.1.3.070003
-rwxr-xr-x root/root   2875754 2008-06-16 14:07:24 sys/lib/libGLU.so.1.3.070003
lrwxrwxrwx root/root         0 2008-06-16 14:07:24 sys/lib/libGLw.so -> libGLw.so.1
lrwxrwxrwx root/root         0 2008-06-16 14:07:24 sys/lib/libGLw.so.1 -> libGLw.so.1.0.0
-rwxr-xr-x root/root    127359 2008-06-16 14:07:24 sys/lib/libGLw.so.1.0.0
lrwxrwxrwx root/root         0 2008-06-16 14:07:10 sys/lib/libOSMesa.so -> libOSMesa.so.6
lrwxrwxrwx root/root         0 2008-06-16 14:07:10 sys/lib/libOSMesa.so.6 -> libOSMesa.so.6.5.3
-rwxr-xr-x root/root    252524 2008-06-16 14:07:10 sys/lib/libOSMesa.so.6.5.3
This time the whole X11 distribution is archived (all files in given sub-tree and no timestamp filtering).
dev# ./filopack.sh -p sys/X11R7/ -f '' XOrg.7.1
...

Read more...

WMU-6500FS - Build environment

  1. M$ Windows machine with running CoLinux/Debian 4.0 installation (just minimal console setup, installing packages as needed) - in this document shell commands from this system are prefixed deb#
  2. uClibc development system - Prebuilt by JoKeR - download available [here] (tried to build my own, but did not succeed even with JoKeR's config) stored on 'deb#' system and placed in chroot jail - shell commands from this system are prefixed dev#
  3. WMU-6500FS box upgraded with JoKeR's firmware version C009M Advanced. Currently there is a newer version available - shell commands from this system are prefixed box#
The initial preparation of the environment consists of following steps:
  1. Download the archive with JoKeR's pre-built development system (assuming you want to extract it to your home directory):
      
    deb# cd ~
    deb# wget http://mgb111.pradnik.net/addons/uclibc-basic.tar.bz2
    
  2. Extracting the archive:
      
    deb# tar xjvf uclibc-basic.tar.bz2
    ...
    ./uclibc-basic/uclibc-env/dev/<device>
    tar: ./uclibc-basic/uclibc-env/dev/<device>: Cannot mknod: Operation not permitted
    ...
    tar: Error exit delayed from previous errors
    
    We ended up with some permition denied errors. We need to have root rights in order to create /dev nodes (special files created with the mknod command). The problem is soved when we untar the archive as superuser:
      
    deb# sudo tar xjvf uclibc-basic.tar.bz2
    Password: ...
    
    ... now the extraction finished with no errors.
  3. Now we are ready to log-in to the newly created chroot-ed environment:
      
    deb# chroot uclibc-basic/uclibc-env/ /bin/su -
    -bash: chroot: command not found
    
    We need superuser rights in order to use the chroot command:
      
    deb# sudo chroot uclibc-basic/uclibc-env/ /bin/su -
    Password: ...
    dev#
    
    ... so now we are sucessfully logged in the new environment.
Tip:
I am using Putty connection manager for accessing all the systems and I have found it very convenient. In 'PCM' it is possible to create a command sequence executed just after the login - so for the dev# login I just setup the following command sequence:
  
deb# cd ~
deb# chroot uclibc-basic/uclibc-env/ /bin/su -
dev# export PATH=$PATH:/mnt/C/sys/bin
dev# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/mnt/C/sys/lib:/mnt/C/sys/apache2/lib

Read more...

WMU-6500FS Configuration

Info sources:
[WMU-6500FS product page]
[Original firmware]
[JoKeR's alternative firmware]
[JoKeR's pre-built software]
[Macsat forum]

Related info:
[Howto build the firmware]
[Original fw source]
[WMU-6000 FS build toolchain]
[C library for embedded Linux systems]
[C++ standard library for embedded systems]

Prerequisites

  1. JoKeR's firmware: [FW C009-M Advanced]
  2. First access to device after the (successful) firmware upgrade:

  3. ... use a telnet client (e.g. Putty) to connect to the device:
    box# cd /mnt/C
    box# ls
    
    ... there should be no sys/ directory at the time
    box# wget http://mgb111.pradnik.net/minimal-config.tar
    box# tar xvf minimal-config.tar
    
  4. Configuration
  5. Consider following settings in file "/mnt/C/sys/etc/env-ng":
    # include /etc/rc.nextgen to startup, if RCNG=0, 
    # none of bottom options will work 
    #(excluding CHBACKUP, NOTELNET, NOLOGIN!)
    RCNG=1
    # change root home dir from readonly flash to disk
    ROOTDIR=1
    # change root shell to bash (if exist)
    ROOTSHELL=1
    # change users shell to bash (if exist)
    USERSHELL=1
    
    # if 1 dropbear sshd will start (must exist, and configured!)
    DROPBEAR=1
    
    # if 1 then may use extra shell startup scripts 
    # /mnt/C/sys/etc/rc.d/rc.* and different style /mnt/C/sys/etc/start.d/* 
    # for links to executables.
    RCPLUS=1
    
    # if 1 then kernel module will be unloaded
    NOWIFI=1
    
    ... and creation of following files:
    /mnt/C/sys/etc/rc.d/rc.kill
    ### kill/remove unused#
    killall -9 dhcpd amit_avs amit_tftpd amit_db 2>/dev/null &
    route del -net 224.0.0.0 netmask 240.0.0.0 dev br0
    
    /mnt/C/sys/etc/rc.d/rc.cfg
    ### replace system configs
    # hostname mgb111.host.net
    # cat /mnt/C/sys/etc/smb.conf > /var/config/samba/smb.conf
    # cat /mnt/C/sys/etc/resolv.conf > /etc/resolv.conf
    # cat /mnt/C/sys/etc/hosts > /etc/hosts
    
    # place user dirs at /home directory
    sed -i 's|/tmp/Virtual/|/home/|g' /var/config/passwd
    
  6. X client-server configuration
  7. We need an X server running and accessible for our X client applications. There are plenty of possible configurations, I use the XMing server. For details see XMing notes and XMing intallation guide.

    Download locations: [XMing mesa] [XMing fonts]
    Once we install and run the X server we have to configure X clients properly:
    box# export DISPLAY=<X-server-host-IP>:0
    
    Now we can try to launch one:
    box# /mnt/C/sys/X11/bin/xterm
    Xlib: connection to "<X-server-host-IP>:0.0" refused by server
    Xlib: No protocol specified
    xterm Xt error: Can't open display: <X-server-host-IP>:0
    
    There is a configuration problem. When we look at the XMing log, we see:
    AUDIT: ... Xming: client 4 rejected from IP <X-server-host-IP>
    
    As a quick solution, we can disable the access control:
    Note: -ac is unsafe on public networks, but makes things easier for beginners on private ones.
    C:\Program Files\XMing\Xming.exe :0 -clipboard -multiwindow -ac
    
    ... and problem is solved.

    To make the DISPLAY and other X related settings permanent we have to add the following lines top the /mnt/C/sys/etc/profile:
    # executable library and package paths (both X-based and console apps)
    export PATH=$PATH:/mnt/C/sys/X11/bin/
    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/mnt/C/sys/lib/:/mnt/C/sys/X11/lib/
    export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/mnt/C/sys/lib/pkgconfig/:/mnt/C/sys/X11/lib/pkgconfig/
    # X server IP (for X-client apps)
    export DISPLAY=<X-server-host-IP>:0
    
  8. Soft & libs installation
  9. Essential and other libs and utils
    box# cd /mnt/C
    box# wget http://mgb111.pradnik.net/addons/libs-essential-20080502.tar.bz2
    box# tar xvjf libs-essential-20080502.tar.bz2
    box# wget http://mgb111.pradnik.net/addons/util-linux.tar
    box# tar xvf util-linux.tar
    box# wget http://mgb111.pradnik.net/addons/soft-system/coreutils-5.96-2.tar.gz
    box# tar xzvf coreutils-5.96-2.tar.gz
    box# wget http://mgb111.pradnik.net/addons/nfs-utils.tar
    box# tar xvf nfs-utils.tar
    box# wget http://mgb111.pradnik.net/addons/busybox2-extended.tar.gz
    box# tar xzvf busybox2-extended.tar.gz
    box# wget http://mgb111.pradnik.net/addons/modules-kernel.tar
    box# tar xvf modules-kernel.tar
    box# wget http://mgb111.pradnik.net/addons/modules-kernel-extra.tar
    box# tar xvf modules-kernel-extra.tar
    
    SSL
    box# cd /mnt/C
    box# wget http://mgb111.pradnik.net/addons/openssl-0.9.tar
    box# tar xvf openssl-0.9.tar
    
    Bash
    box# cd /mnt/C
    box# wget http://mgb111.pradnik.net/addons/soft-shells/bash-3.2.33-3.tar.gz
    box# tar xzvf bash-3.2.33-3.tar.gz
    
    File viewers and editors
    box# cd /mnt/C
    box# wget http://mgb111.pradnik.net/addons/soft-system/less-381.tar
    box# tar xvf less-381.tar
    box# wget http://mgb111.pradnik.net/addons/soft-editors/nano-2.0.7-2.tar
    box# tar xvf nano-2.0.7-2.tar
    box# wget http://mgb111.pradnik.net/addons/soft-editors/zile-2.2.57.tar
    box# tar xvf zile-2.2.57.tar
    
    Archivers/Unpackers
    box# cd /mnt/C
    box# wget http://mgb111.pradnik.net/addons/soft-archive-unpack/unzip-5.52.tar
    box# tar xvf unzip-5.52.tar
    box# wget http://mgb111.pradnik.net/addons/soft-archive-unpack/rar-3.0.tar
    box# tar xvf rar-3.0.tar
    box# wget http://mgb111.pradnik.net/addons/soft-archive-unpack/p7zip_4.57.tar
    box# tar xvf p7zip_4.57.tar
    
    Midnight commander
    box# cd /mnt/C
    box# wget http://mgb111.pradnik.net/addons/soft-file-managers/mc-20070623.tar.gz
    box# wget http://mgb111.pradnik.net/addons/mc-4.6.1.tar
    box# tar xvf mc-4.6.1.tar
    box# wget http://mgb111.pradnik.net/addons/mcedit.tar
    box# tar xvf mcedit.tar
    
    AWK
    box# cd /mnt/C
    box# wget http://mgb111.pradnik.net/addons/soft-system/gawk-3.1.6.tar
    box# tar xvf gawk-3.1.6.tar
    
    HDParm
    box# cd /mnt/C
    box# wget http://mgb111.pradnik.net/addons/soft-system/hdparm-8.6.tar
    box# tar xvf hdparm-8.6.tar
    
    SSH server/client (dropbear)
    Info source: [Public Key Authentication Howto]
    box# cd /mnt/C
    box# wget http://mgb111.pradnik.net/addons/servers-ssh/dropbear-050.3-n-0.tgz
    box# tar xvzf dropbear-050.3-n-0.tgz
    box# dropbearkey -t rsa -f /mnt/C/sys/etc/dropbear_rsa_host_key
    Will output 1024 bit rsa secret key to '/mnt/C/sys/etc/dropbear_rsa_host_key'
    Generating key, this may take a while...
    Public key portion is:
    ssh-rsa AAAA... <user>@<hostname>
    Fingerprint: md5 ... 
    
    ... after reboot you can use SSH access (e.g. via Putty with SSH protocol) to connect to the device

Communication with the box

As far as some communication channel can be established (e.g. via TCP/IP in this case) the fact that input/output devices are not present does not matter so much. There is a whole variety of ways how to communicate with such a device.

For example following (mostly client/server) configurations are possible:
(by building X.Org libraries I am just trying to enable the point 3 - I am interested mainly in X client part - not in server part - but do not know the exact dependencies and so I am building all)
  1. Web browser as a thin client connecting to a HTTP daemon running on the box (examples: built-in webUI, cvstrac, ...)
  2. Telnet/SSH client running on separate machine and connecting to the box with running SSH daemon (example: dropbear + mc)
  3. X-client application running on the box and connecting to X-server running elsewhere (e.g. on a PC). Box serves as an application server and X client, PC serves as X server and app client (examples: x applications)
  4. VNC viewer connecting to VNC server (along with X server) running on the box (examples: X applications) - not sure it this is possible with Airlive.
  5. Some peripherals connected to the box (e.g. via USB). I tried to search the Web and have found some description how to setup USB mouse and keyboard and that there is a USB2SVGA converter and a guide how to set it up. I have no idea what the real possibilities are for the Airlive device and whether it is useful and practical but I think at least in principle it is possible.

Read more...