2008/12/22

WMU-6500FS - dropbear-0.52

Introduction

It seems to me that dropbear ssh daemon from JoKeR's package is configured so it does not support public key authentication. It is possible that I am doing something wrong but I was trying hard to make it work and did not succeed.
Finally I decided to build dropbear on my own, since the publickey authentication feature is essential for me (I am using the box for hosting of git repositories and I really do not want to type my password every time I commit a code).

Build result

[binary] [file list]

Installation

box# cd /mnt/C
box# ./filopack.sh --download dropbear-0.52
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Downloading package dropbear-0.52 from http://filodej.ic.cz ...
connected!

Length: 502 [text/plain]
connected!

Length: 177,869 [application/x-tar]
box# ./filopack.sh --install dropbear-0.52
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Sure to unpack dropbear-0.52 locally at /mnt/C (y/n)?y
sys/bin/dbclient
sys/bin/dropbearmulti
sys/bin/dropbearkey
sys/bin/dropbearconvert
sys/bin/scp
sys/sbin/dropbear
box# killall dropbear
box# dropbear
In this post I am going to describe how I built the dropbear package and diagnosed and solved (or worked around) problems which arose during the whole process.

JoKeR's dropbear

Let's try co connect to the box:
If ssh daemon (dropbear in our case) is not running on the box, the error message will likely look as follows:
deb# ssh root@storage
ssh: connect to host storage port 22: Connection refused
If everything is ok, it should look similar to following:
deb# ssh root@storage
root@storage's password: < type a password > 

  [    FW 4.00b4.C009-M[NG]    ]
  [ http://mgb111.pradnik.net/ ]

box#
Now we are trying to get rid of typing the password. We use the RSA public authentication for it.

Publickey authentication

If we do not have a RSA key generated yet, we can generate it as follows:
deb# cd ~
deb# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/filodej/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/filodej/.ssh/id_rsa.
Your public key has been saved in /home/filodej/.ssh/id_rsa.pub.
The key fingerprint is:
89:...:b4... filodej@debian
Deploy the public key to the box:
deb# scp .ssh/id_rsa.pub root@storage:/mnt/C/sys/root/.ssh/
root@storage's password:
id_rsa.pub                                                              100%  393     0.4KB/s   00:00
Now, on the box, append the public key to the list of authorized keys:
box# cd /mnt/C/sys/root/.ssh/
box# cat id_rsa.pub >> authorized_keys
From now on, the ssh client should not ask for anything (more precisely, it should not ask for password, it should ask for publickey passphrase if we did not leave it empty - for someone it can look similar as password, but it is completely different thing).

Let's try:
deb# ssh root@storage
root@storage's password:

Simple diagnostics

Ooops, it seems we did not succeed. Let's do some diagnostics (we can use either of "-v", "-vv", "-vvv" verbose modes, one "v" is sufficiend here):
dev# ssh -v root@storage
OpenSSH_4.3p2 Debian-9, OpenSSL 0.9.8c 05 Sep 2006
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to storage [192.168.1.104] port 22.
debug1: Connection established.
debug1: identity file /home/filodej/.ssh/identity type -1
debug1: identity file /home/filodej/.ssh/id_rsa type 1
debug1: identity file /home/filodej/.ssh/id_dsa type -1
debug1: Remote protocol version 2.0, remote software version dropbear_0.50
debug1: no match: dropbear_0.50
...
debug1: Host 'storage' is known and matches the RSA host key.
debug1: Found key in /home/filodej/.ssh/known_hosts:4
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: password
debug1: Next authentication method: password
root@storage's password:
Let's try to connect to another machine and compare the trace:
deb# ssh -v root@notasw
OpenSSH_4.3p2 Debian-9, OpenSSL 0.9.8c 05 Sep 2006
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to notasw [192.168.1.103] port 22.
debug1: Connection established.
debug1: identity file /home/filodej/.ssh/identity type -1
debug1: identity file /home/filodej/.ssh/id_rsa type 1
debug1: identity file /home/filodej/.ssh/id_dsa type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_4.7p1 Debian-8ubuntu1.2
debug1: match: OpenSSH_4.7p1 Debian-8ubuntu1.2 pat OpenSSH*
...
debug1: Host 'notasw' is known and matches the RSA host key.
debug1: Found key in /home/filodej/.ssh/known_hosts:8
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Trying private key: /home/filodej/.ssh/identity
debug1: Offering public key: /home/filodej/.ssh/id_rsa
debug1: Authentications that can continue: publickey,password
debug1: Trying private key: /home/filodej/.ssh/id_dsa
debug1: Next authentication method: password
root@notasw's password:
We can diff the logs to easily see the differences:
deb# ssh -v root@storage 2> storage.txt
root@storage's password:
<Ctrl-D>
deb# ssh -v root@notasw 2> notas.txt
root@notasw's password:
<Ctrl-D>
deb# diff storage.txt notas.txt
4c4
< debug1: Connecting to storage [192.168.1.104] port 22.
---
> debug1: Connecting to notasw [192.168.1.103] port 22.
9,10c9,10
< debug1: Remote protocol version 2.0, remote software version dropbear_0.50
< debug1: no match: dropbear_0.50
---
> debug1: Remote protocol version 2.0, remote software version OpenSSH_4.7p1 Debian-8ubuntu1.2
> debug1: match: OpenSSH_4.7p1 Debian-8ubuntu1.2 pat OpenSSH*
23,26c23,28
...
< debug1: Host 'storage' is known and matches the RSA host key.
< debug1: Found key in /home/filodej/.ssh/known_hosts:4
---
...
> debug1: Host 'notasw' is known and matches the RSA host key.
> debug1: Found key in /home/filodej/.ssh/known_hosts:8
33c35,40
< debug1: Authentications that can continue: password
---
> debug1: Authentications that can continue: publickey,password
> debug1: Next authentication method: publickey
> debug1: Trying private key: /home/filodej/.ssh/identity
> debug1: Offering public key: /home/filodej/.ssh/id_rsa
> debug1: Authentications that can continue: publickey,password
> debug1: Trying private key: /home/filodej/.ssh/id_dsa
It seems that the JoKeR's daemon is not configured for publickey authentication at all. Ok, let's build our own.

Build process

First we have to download, extract and configure the build:
dev# cd /usr/local/src/
dev# wget http://matt.ucc.asn.au/dropbear/dropbear-0.52.tar.

bz2
dev# tar xjvf dropbear-0.52.tar.bz2 
dev# cd dropbear-0.52
dev# ./configure --prefix /mnt/C/sys
...
configure: Now edit options.h to choose features.
As the message suggests, now we can set up some additional build options:
dev# nano options.h

change:
  #define RSA_PRIV_FILENAME "/etc/dropbear/dropbear_rsa_host_key"
to:
  #define RSA_PRIV_FILENAME "/mnt/C/sys/etc/dropbear_rsa_host_key"

and possibly comment out:
  #define DROPBEAR_DSS
or change:
  #define DSS_PRIV_FILENAME "/etc/dropbear/dropbear_dss_host_key"
to:
  #define DSS_PRIV_FILENAME "/mnt/C/sys/etc/dropbear_dss_host_key"

leave following unchanged:
  #define ENABLE_SVR_PASSWORD_AUTH
  /* PAM requires ./configure --enable-pam */
  /*#define ENABLE_SVR_PAM_AUTH*/
  #define ENABLE_SVR_PUBKEY_AUTH

  /* Wether to ake public key options in authorized_keys file into account */
  #ifdef ENABLE_SVR_PUBKEY_AUTH
  #define ENABLE_SVR_PUBKEY_OPTIONS
  #endif

  #define ENABLE_CLI_PASSWORD_AUTH
  #define ENABLE_CLI_PUBKEY_AUTH
  #define ENABLE_CLI_INTERACT_AUTH
... and make:
dev# make PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp"
...
It compiled successfully, but I have realized that I prefer to link the dependencies statically:
dev# make PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp" clean
dev# make STATIC=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp"
...
/mnt/C/sys/lib/gcc/i386-linux-uclibc/4.1.2/../../../../i386-linux-uclibc/bin/ld: cannot find -lz
collect2: ld returned 1 exit status
... the error means that linker is unable to find a static version of zlib library.
When we investigate that we see taht there is just a dynamic version of the library on the system:
dev# ls /lib/libz*
/lib/libz.so.1  /lib/libz.so.1.2.2
dev# ls /mnt/C/sys/lib/libz*
/mnt/C/sys/lib/libz*: No such file or directory
We could pick the sources and build a static version of zlib library, but for now I decided to re-configure the dropbear build and disable the zlib support:
dev# make STATIC=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp" clean
dev# ./configure --prefix /mnt/C/sys --disable-zlib STATIC=1 
dev# make STATIC=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp"
... now it compiled without problems, we can try to install it and upload to the box:
dev# make STATIC=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp" install
...
dev# scp /mnt/C/sys/sbin/dropbear root@storage:/tmp/
Now we can start the brand new version of dropbear daemon on the box:
box# killall dropbear
box# /tmp/dropbear
And try to log in over ssh client (from my debian system):
deb# ssh -vvv root@storage -i ~/.ssh/id_rsa.pub
...
debug2: key: /home/filodej/.ssh/id_rsa.pub (0xb7f7b480)
debug2: key:  (0xb7f7c8e0)
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Trying private key: /home/filodej/.ssh/identity
debug1: Offering public key: /home/filodej/.ssh/id_rsa
debug1: Authentications that can continue: publickey,password
debug1: Trying private key: /home/filodej/.ssh/id_dsa
debug1: Next authentication method: password
root@storage's password:
debug1: Authentications that can continue: publickey,password
Permission denied, please try again.
root@storage's password:
debug1: Authentications that can continue: publickey,password
Permission denied, please try again.
root@storage's password:
It seems we have done some progress here, but still are not able to authenticate via publickey. And even if we are using the correct password, permission is denied.

List of valid shells

Let's restart the daemon, but now not forking to the background and logging to the stderr rather that syslog:
box# dropbear -E -F
[4456] Nov 30 23:59:30 Not backgrounding
[4462] Nov 30 23:59:56 Child connection from 192.168.1.103:49013
[4462] Dec 01 00:00:08 user 'root' has invalid shell, rejected
Ok, there is something wrong with the shell settings, let's investigate that:
box# cat /var/config/passwd
root: ... :0:0:root:/mnt/C/sys/root:/mnt/C/sys/bin/bash
...
box# ls /mnt/C/sys/bin/bash
/mnt/C/sys/bin/bash
User root has /mnt/C/sys/bin/bash in his settings, the file exists and is at correct place. Let's look at the list of valid shells:
box# cat /etc/shells
/bin/sh
/sbin/nologin
/bin/nologin
/bin/ash
/sbin/ash
/bin/false
/sbin/false
/mnt/C/sys/bash
/bin/bash
/usr/bin/bash
The path /mnt/C/sys/bash is wrong since the bash executable is in /mnt/C/sys/bin directory, but the /usr/bin/bash path should be ok, since the /usr/bin is just a symlink to /mnt/C/sys/bin:
box# ls -l /usr/bin
lrwxrwxrwx    1 root     root           14 May 21  2008 /usr/bin -> /mnt/C/sys/bin
We have to look at the problem in more detail. Since there is no verbose mode available in dropbear by default, we can enable it in code (I got the inspiration from this post):
dev# nano debug.h
    uncomment:
        #define DEBUG_TRACE
Now when we rebuilt and re-deployed the dropbear, we can run in in verbose mode:
box# dropbear -v -E -F 
...
TRACE (4848): shell is /mnt/C/sys/bin/bash
TRACE (4848): test shell is '/bin/sh'
TRACE (4848): test shell is '/sbin/nologin'
TRACE (4848): test shell is '/bin/nologin'
TRACE (4848): test shell is '/bin/ash'
TRACE (4848): test shell is '/sbin/ash'
TRACE (4848): test shell is '/bin/false'
TRACE (4848): test shell is '/sbin/false'
TRACE (4848): test shell is '/mnt/C/sys/bash'
TRACE (4848): test shell is '/bin/bash'
TRACE (4848): test shell is '/usr/bin/bash'
As can be seen in the svr-auth.c source code:
00273       while ((listshell = getusershell()) != NULL) {
00274             TRACE(("test shell is '%s'", listshell))
00275             if (strcmp(listshell, usershell) == 0) {
00276                   /* have a match */
00277                   goto goodshell;
00278             }
00279       }
... the shell path is compared as plain strings and so /usr/bin/bash and /mnt/C/sys/bin/bash are two different things.

Since the /etc/shells file is placed on read-only file system we have to make a change in /var/config/passwd file:
box# sed -i 's|/mnt/C/sys/bin/|/usr/bin/|g' /var/config/passwd
Since it is overwritten on every boot, we have to make the correction in our rc.cfg file, to make it permanent:
nano /mnt/C/sys/etc/rc.d/rc.cfg
   add the following line:
    sed -i 's|/mnt/C/sys/bin/|/usr/bin/|g' /var/config/passwd 
Now when we try to connect again, we can see that the problem is solved:
[394] Dec 01 01:32:25 pubkey auth succeeded for 'root' with key md5 

3a:...:05 from 192.168.1.103:44521

Read only file system

There is another problem:
[394] Dec 01 01:32:25 chown(/dev/ttyp2, 0, 0) failed: Read-only file system
[394] Dec 01 01:32:26 exit after auth (root): chmod(/dev/ttyp2, 0622) failed: Read-only file system
The problem seems to be in the file sshpty.c in function pty_setowner but I am not sure what exactly is wrong. We can try to debug it remotely in order to find out what is going on. One option in debug.h can be handy - we can disable the forking the sub-processes - then there is just one dropbear process to attach.
/* To debug with GDB it is easier to run with no forking of child processes.
   You will need to pass "-F" as well. */
/* #define DEBUG_NOFORK */
Let's rebuild it with debug info:
dev# nano Makefile
...
CFLAGS=-I. -I$(srcdir) -I$(srcdir)/libtomcrypt/src/headers/ $(CPPFLAGS) -I/mnt/C/sys/include 

-I/mnt/C/sys/X11/include -g
...
Rebuild and redeploy the dropbear binary.

Now we can run the dropbear process on the box:
box# dropbear -E -F
box# [4498] Dec 21 23:50:01 Running in background
...
On another console connected to the box we start the gdbserver:
box# gdbserver colinux:2345 --attach 4498
Attached; pid = 4498
Listening on port 2345
...
Now on the mirror system we can start the gdb:
dev# gdb dropbear
GNU gdb 6.3
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-linux-uclibc"...Using host libthread_db library "/lib/libthread_db.so.1".

(gdb) target remote 192.168.1.104:2345
Remote debugging using 192.168.1.104:2345
0x0806d7ad in fast_s_mp_sqr (a=0xbfffe348, b=0xbfffe348) at bn_fast_s_mp_sqr.c:73
73               _W += ((mp_word)*tmpx++)*((mp_word)*tmpy--);
(gdb) break pty_setowner
Breakpoint 1 at 0x805bfcc: file sshpty.c, line 364.
(gdb) cont
Continuing.
Now we are ready to connect over ssh:
dev# ssh root@storage
root@storage's password:
... and debugger stops on the breakpoint:
Breakpoint 1, pty_setowner (pw=0x80dd480, tty_name=0x80e1578 "/dev/ttyp3") at sshpty.c:364
364             grp = getgrnam("tty");
(gdb) step
365             if (grp) {
(gdb) step
369                     gid = pw->pw_gid;
(gdb) step
370                     mode = S_IRUSR | S_IWUSR | S_IWGRP | S_IWOTH;
(gdb) step
378             if (stat(tty_name, &st)) {
(gdb) step
382             dropbear_log(LOG_ERR, "stat: %u", (unsigned int)st.st_uid );
(gdb) print st
$1 = {st_dev = 7936, __pad1 = 45352, __st_ino = 363, st_mode = 8630, st_nlink = 1, st_uid = 0, st_gid = 5,
  st_rdev = 771, __pad2 = 0, st_size = 0, st_blksize = 4096, st_blocks = 0, st_atime = 1206970720,
  st_atimensec = 134760895, st_mtime = 1206970720, st_mtimensec = 134691188, st_ctime = 1206970720,
  st_ctimensec = 135124096, st_ino = 363}
(gdb) print tty_name
$2 = 0x80e1578 "/dev/ttyp3"
(gdb) next
383             if (st.st_uid != pw->pw_uid || st.st_gid != gid) {
(gdb) print *pw
$3 = {pw_name = 0x80dd380 "root", pw_passwd = 0x80dd385 "...", pw_uid = 0, pw_gid = 0, pw_gecos = 

0x80dd3a4 "root", pw_dir = 0x80dd3a9 "/mnt/C/sys/root", pw_shell = 0x80dd3b9 "/usr/bin/bash"}
...
(gdb) step
401 if ((st.st_mode & (S_IRWXU|S_IRWXG|S_IRWXO)) != mode) {
(gdb) print /o st.st_mode
$4 = 020666
(gdb) step
402                     if (chmod(tty_name, mode) < 0) {
(gdb) step
403                             if (errno == EROFS &&
(gdb) step
409                                     dropbear_exit("chmod(%.100s, 0%o) failed: %.100s",
Ok, we have analyzed the situation. The pty_setowner function is trying to initialize the device file representing the terminal we are trying to connect to. Device file is on read only file system and so the attribute changes do not succeed.

If changing the tty owner does not succeed, the code at (1) activates more relaxed mode in case the file system is read only and tty is owned by root or the uids match. In our case the relaxed mode is activated and the server just writes the warning chown(/dev/ttyp2, 0, 0) failed: Read-only file system and continues.

Then there is code changing tty read mode. This code tries to prevent unauthorized users from reading tty content. In order to get just a warning, the reading by group and reading by others must be prohibited. In our case the st.st_mode is 020666 (read/write by everyone) and so we get the error chmod(/dev/ttyp2, 0622) failed: Read-only file system.

 /*
  * Change owner and mode of the tty as required.
  * Warn but continue if filesystem is read-only and the uids match/
  * tty is owned by root.
  */
 if (stat(tty_name, &st)) {
  dropbear_exit("pty_setowner: stat(%.101s) failed: %.100s",
    tty_name, strerror(errno));
 }

 if (st.st_uid != pw->pw_uid || st.st_gid != gid) {
  if (chown(tty_name, pw->pw_uid, gid) < 0) {
   if (errno == EROFS &&
       (st.st_uid == pw->pw_uid || st.st_uid == 0)) {
(1)    dropbear_log(LOG_ERR,
     "chown(%.100s, %u, %u) failed: %.100s",
      tty_name, (unsigned int)pw->pw_uid, (unsigned int)gid,
      strerror(errno));
   } else {
    dropbear_exit("chown(%.100s, %u, %u) failed: %.100s",
        tty_name, (unsigned int)pw->pw_uid, (unsigned int)gid,
        strerror(errno));
   }
  }
 }

 if ((st.st_mode & (S_IRWXU|S_IRWXG|S_IRWXO)) != mode) {

  if (chmod(tty_name, mode) < 0) {
   if (errno == EROFS &&
(2)       (st.st_mode & (S_IRGRP | S_IROTH)) == 0) {
    dropbear_log(LOG_ERR,
     "chmod(%.100s, 0%o) failed: %.100s",
     tty_name, mode, strerror(errno));
   } else {
    dropbear_exit("chmod(%.100s, 0%o) failed: %.100s",
        tty_name, mode, strerror(errno));
   }
  }
 }
Some discussions (for example here and here) recommended to configure build in order to disable openpty:
dev# make STATIC=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp" clean
dev# ./configure --prefix /mnt/C/sys --disable-zlib --disable-openpty STATIC=1 
dev# make STATIC=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp"
dev# make STATIC=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp" install
But it did not help in this case.

Nowadays I do not know a better workaround in order to make it work than change the dropbear_exit to dropbear_log. I have found a discussion about this on Web and the following rationale seems reasonable:
Ah. If it's a single user system, the easiest solution might just be to disable the chown() and chmod() call - it's not like there're any other users who might be sniffing :)
(sshpty.c is where you'll find those chown/chmods).
... but be careful, you were warned ;-)

Minor issues

There were also two another problems (just warnings). The first was wtmp problem:
[2398] Dec 01 12:13:04 wtmp_write: problem writing /var/log/wtmp: No such file or directory
It was solved by adding the --disable-wtmp to the configure command line.

The second was logging problem:
dev# ssh root@storage
root@storage's password:
[2719] Dec 01 13:25:53 lastlog_perform_login: Couldn't stat /var/log/lastlog: No such file or directory
[2719] Dec 01 13:25:53 lastlog_openseek: /var/log/lastlog is not a file or directory!

  [    FW 4.00b4.C009-M[NG]    ]
  [ http://mgb111.pradnik.net/ ]
We are not alone on this (see this post or OpenWrt ticket, also this archive served as a source of information).

It was solved by adding the --disable-lastlog to the configure command line.

Final build

As a result we have the following configure command line:
dev# ./configure --prefix /mnt/C/sys --disable-zlib --disable-openpty --disable-wtmp --disable-lastlog
Now we can modify the Makefile in order to generate code optimized for size:
dev# nano Makefile
...
CFLAGS=-I. -I$(srcdir) -I$(srcdir)/libtomcrypt/src/headers/ $(CPPFLAGS) -I/mnt/C/sys/include -I/mnt/C/sys/X11/include -Os
If we want to create multi-application instead of individual applications, we can use MULTI variable on the make command line:
dev# make STATIC=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp" clean
dev# make STATIC=1 MULTI=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp"
We can strip the sources to spare some space:
dev# make STATIC=1 MULTI=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp" strip
And are ready to install and create the package:
dev# make STATIC=1 MULTI=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp" install
dev# cd /mnt/C
dev# ./filopack.sh --pack dropbear-0.52

Test

Start the dropbear server (logging to strerr, not forking to background):
box# dropbear -E -F
[902] Dec 22 02:14:49 Not backgrounding
This is output when I was connecting from a machine without a registered public key:
[906] Dec 22 02:14:52 Child connection from 192.168.1.102:44241
[906] Dec 22 02:15:02 password auth succeeded for 'root' from 192.168.1.102:44241
[906] Dec 22 02:15:02 chown(/dev/ttyp3, 0, 0) failed: Read-only file system
[906] Dec 22 02:15:02 !!! Filodej is warning you. Be sure you are the only user !!!
[906] Dec 22 02:15:02 chmod(/dev/ttyp3, 0622) failed: Read-only file system
[906] Dec 22 02:15:47 chown /dev/ttyp3 0 0 failed: Read-only file system
[906] Dec 22 02:15:47 chmod /dev/ttyp3 0666 failed: Read-only file system
[906] Dec 22 02:15:47 exit after auth (root): Exited normally
This is output when I was connecting from a machine with registered public key:
[913] Dec 22 02:15:59 Child connection from 192.168.1.102:44242
[913] Dec 22 02:16:06 pubkey auth succeeded for 'root' with key md5 f1:...:3b from 192.168.1.102:44242
[913] Dec 22 02:16:06 chown(/dev/ttyp3, 0, 0) failed: Read-only file system
[913] Dec 22 02:16:06 !!! Filodej is warning you. Be sure you are the only user !!!
[913] Dec 22 02:16:06 chmod(/dev/ttyp3, 0622) failed: Read-only file system
[913] Dec 22 02:16:21 chown /dev/ttyp3 0 0 failed: Read-only file system
[913] Dec 22 02:16:21 chmod /dev/ttyp3 0666 failed: Read-only file system
[913] Dec 22 02:16:21 exit after auth (root): Exited normally

Read more...

2008/12/20

WMU-6500FS - xterm 237

GUI dependencies: [X.Org]

Build result

[binary] [file list]

Installation

box# cd /mnt/C
box# ./filopack.sh --download xterm-237
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Downloading package xterm-237 from http://filodej.ic.cz ...
connected!

Length: 502 [text/plain]
connected!

Length: 177,869 [application/x-tar]
box# ./filopack.sh --install xterm-237
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Sure to unpack xterm-237 locally at /mnt/C (y/n)?y
sys/X11/share/pixmaps/xterm-color_32x32.xpm
sys/X11/share/pixmaps/xterm-color_48x48.xpm
sys/X11/share/pixmaps/xterm_32x32.xpm
sys/X11/share/pixmaps/xterm_48x48.xpm
sys/X11/man/man1/xterm.1
sys/X11/man/man1/resize.1
sys/X11/man/man1/uxterm.1
sys/X11/man/man1/koi8rxterm.1
sys/X11/lib/X11/app-defaults/XTerm
sys/X11/lib/X11/app-defaults/XTerm-color
sys/X11/lib/X11/app-defaults/UXTerm
sys/X11/lib/X11/app-defaults/KOI8RXTerm
sys/X11/bin/xterm
sys/X11/bin/resize
sys/X11/bin/uxterm
sys/X11/bin/koi8rxterm
box# xterm

Make sure you have the environment variables set properly (you can make it permanent e.g. by editing the /mnt/C/sys/etc/profile; for details see the X Configuration post):

PATH variable can possibly contain the /mnt/C/sys/X11/bin/ path:
box# export PATH=$PATH:/mnt/C/sys/X11/bin/
LD_LIBRARY_PATH must (among others) contain the /mnt/C/sys/X11/lib path:
box# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/mnt/C/sys/X11/lib/
DISPLAY environment variable properly set to a display of a machine running xserver:
box# export DISPLAY=<machine>:<display-number>
... or in case you are using putty, X11 forwarding enabled:

Build procedure

First we have to initialize the package timestamp.
dev# cd /mnt/C
dev# ./filopack.sh --init xterm-237 
Configuration file .filopack/.config file found and used
Timestamp written to file .filopack/xterm-237.ts
Now we can download and extract the source.
dev# cd /usr/local/src
dev# wget ftp://invisible-island.net/xterm/xterm-237.tgz
...
Length: 860,424 (unauthoritative)

100%[==========================================================>] 860,424       14.59K/s    ETA 00:00

14:18:14 (17.14 KB/s) - `xterm-237.tgz' saved [860424]

dev# tar xzvf xterm-237.tgz 
...
dev# cd xterm-237
We configure the build in order to have access to XOrg header files and libraries:
dev#  ./configure --x-includes=/mnt/C/sys/X11/include/ --x-libraries=/mnt/C/sys/X11/lib/ --prefix=/mnt/C/sys/X11
...
... and make and install the source:
dev# make
...
dev# make install
...
We can test the installation:
dev# /mnt/C/sys/X11/bin/xterm
Warning: This program is an suid-root program or is being run by the root user.
The full text of the error or warning message cannot be safely formatted
in this environment. You may get a more descriptive message by running the
program as a non-root user or by removing the suid bit on the executable.
/mnt/C/sys/X11/bin/xterm Xt error: Can't open display: %s
/mnt/C/sys/X11/bin/xterm:  DISPLAY is not set
Oops, I forgot to set the display variable on the mirror machine. Let's do it now (note that the IP address 192.168.2.1 is specific to my network configuration):
dev# export DISPLAY=192.168.2.1:0
dev# /mnt/C/sys/X11/bin/xterm
xterm: Error 18, errno 2: No such file or directory
Reason: spawn: open() failed on ptsname
It seems there is still some error. Let's investigate it with strace:
dev# strace -f /mnt/C/sys/X11/bin/xterm 2> strace.txt

<Ctrl-C>

dev# tail -n 120 strace.txt
[pid  8119] close(8 <unfinished ...>
[pid  8120] <... setsid resumed> )      = 8120
[pid  8119] <... close resumed> )       = 0
[pid  8120] setpgid(0, 0 <unfinished ...>
[pid  8119] close(5 <unfinished ...>
[pid  8120] <... setpgid resumed> )     = -1 EPERM (Operation not permitted)
[pid  8119] <... close resumed> )       = 0
[pid  8120] ioctl(4, TIOCSPTLCK <unfinished ...>
[pid  8119] read(7,  <unfinished ...>
[pid  8120] <... ioctl resumed> , [0])  = 0
[pid  8120] ioctl(4, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
[pid  8120] ioctl(4, TIOCGPTN, [2])     = 0
[pid  8120] open("/dev/pts/2", O_RDWR)  = -1 ENOENT (No such file or directory)
[pid  8120] write(2, "xterm", 5xterm)        = 5
[pid  8120] write(2, ": Error ", 8: Error )     = 8
[pid  8120] write(2, "18", 218)           = 2
[pid  8120] write(2, ", errno ", 8, errno )     = 8
[pid  8120] write(2, "2", 12)            = 1
[pid  8120] write(2, ": ", 2: )           = 2
[pid  8120] write(2, "No such file or directory", 25No such file or directory) = 25
[pid  8120] write(2, "\n", 1
)           = 1
[pid  8120] write(2, "Reason: ", 8Reason: )     = 8
[pid  8120] write(2, "spawn: open() failed on ptsname", 31spawn: open() failed on ptsname) = 31
[pid  8120] write(2, "\n", 1
)           = 1
[pid  8120] ioctl(4, TCFLSH, 0x1)       = 0
[pid  8120] chown("/dev/pts/2", 0, 0)   = -1 ENOENT (No such file or directory)
[pid  8120] chmod("/dev/pts/2", 0666)   = -1 ENOENT (No such file or directory)
[pid  8120] close(4)                    = 0
[pid  8120] munmap(0xb7c10000, 33608)   = 0
[pid  8120] munmap(0xb7c0b000, 17560)   = 0
[pid  8120] _exit(18)                   = ?
Process 8120 detached
<... read resumed> "", 1048)            = 0
--- SIGCHLD (Child exited) @ 0 (0) ---
...
xterm is unable to open the pseudo terminal device, it should be OK when deployed to the box (there is no full device mapping on my mirror system; it is just chroot-ed to some sub-directory of hosting system).

Let's create the package:
dev# ./filopack.sh --pack xterm-237
Configuration file .filopack/.config file found and used
-rw-r--r-- root/root      1381 2008-12-20 14:27:33 sys/X11/share/pixmaps/xterm-color_32x32.xpm
-rw-r--r-- root/root      2710 2008-12-20 14:27:33 sys/X11/share/pixmaps/xterm-color_48x48.xpm
-rw-r--r-- root/root      2110 2008-12-20 14:27:33 sys/X11/share/pixmaps/xterm_32x32.xpm
-rw-r--r-- root/root      2586 2008-12-20 14:27:33 sys/X11/share/pixmaps/xterm_48x48.xpm
-rw-r--r-- root/root    174641 2008-12-20 14:27:32 sys/X11/man/man1/xterm.1
-rw-r--r-- root/root      2678 2008-12-20 14:27:33 sys/X11/man/man1/resize.1
-rw-r--r-- root/root      2967 2008-12-20 14:27:33 sys/X11/man/man1/uxterm.1
-rw-r--r-- root/root      3188 2008-12-20 14:27:33 sys/X11/man/man1/koi8rxterm.1
-rw-r--r-- root/root      6853 2008-12-20 14:27:33 sys/X11/lib/X11/app-defaults/XTerm
-rw-r--r-- root/root      4324 2008-12-20 14:27:33 sys/X11/lib/X11/app-defaults/XTerm-color
-rw-r--r-- root/root      2109 2008-12-20 14:27:33 sys/X11/lib/X11/app-defaults/UXTerm
-rw-r--r-- root/root       747 2008-12-20 14:27:33 sys/X11/lib/X11/app-defaults/KOI8RXTerm
-rwxr-xr-x root/root    334759 2008-12-20 14:27:32 sys/X11/bin/xterm
-rwxr-xr-x root/root     11157 2008-12-20 14:27:32 sys/X11/bin/resize
-rwxr-xr-x root/root      2088 2008-12-20 14:27:32 sys/X11/bin/uxterm
-rwxr-xr-x root/root      2163 2008-12-20 14:27:32 sys/X11/bin/koi8rxterm
... and deploy the package to the box:
dev# ./filopack.sh --upload xterm-237
Configuration file .filopack/.config file found and used
Sure to upload xterm-237 to root@storage:/mnt/C (y/n)?y
root@storage's password:
root@storage's password:
Now on the box, we can install it:
box# [root@storage C]# ./filopack.sh --install xterm-237
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Module xterm-237 not found in package list - use 'tar.bz2' extension
Sure to unpack xterm-237 locally at /mnt/C (y/n)?y
sys/X11/share/pixmaps/xterm-color_32x32.xpm
sys/X11/share/pixmaps/xterm-color_48x48.xpm
sys/X11/share/pixmaps/xterm_32x32.xpm
sys/X11/share/pixmaps/xterm_48x48.xpm
sys/X11/man/man1/xterm.1
sys/X11/man/man1/resize.1
sys/X11/man/man1/uxterm.1
sys/X11/man/man1/koi8rxterm.1
sys/X11/lib/X11/app-defaults/XTerm
sys/X11/lib/X11/app-defaults/XTerm-color
sys/X11/lib/X11/app-defaults/UXTerm
sys/X11/lib/X11/app-defaults/KOI8RXTerm
sys/X11/bin/xterm
sys/X11/bin/resize
sys/X11/bin/uxterm
sys/X11/bin/koi8rxterm
And test the installation:
box# xterm
Cannot chown /dev/ttyp4 to 0,0: Read-only file system
Cannot chmod /dev/ttyp4 to 620 currently 666: Read-only file system
Cannot chown /dev/ttyp4 to 0,0: Read-only file system
The problem here is that xterm is trying to change the rights for the terminal device it is using (/dev/ttyp4 in this case) in order to prevent a read access of an unauthorized person (octal value 620 means that only root can read and write the terminal and users from the same group can just write). However, since the file system is read-only the chmod operation does not succeed and so the terminal device stays readable for everyone.
I was dealing with the same issue during the dropbear build.

Anyway, in case you are using the box on a trusted network with no one "sniffing around" and have it properly secured from the outside world then the warning should not mean any harm. Maybe it could be solved by a change in configuration (or by a slight change in source code), but I am not going to solve it now.


Read more...

2008/12/08

WMU-6500FS - built sw -Python 2.5.2

[Homepage]

[files] [binary]

First I briefly write about the initial build:
dev# cd /usr/local/src
dev# wget http://python.org/ftp/python/2.5.2/Python-2.5.2.tgz
dev# tar xvzf Python-2.5.2.tgz
dev# cd Python-2.5.2
dev# ./configure --prefix=/mnt/C/sys --enable-shared
dev# make
There was an error with missing 'yperr_string' symbol (I am not sure whether it was during the build or when I launched python console). The symbol was needed by the nis extension module. I do not know how on earth I would utilize an Interface to Sun's NIS (Yellow pages) so as a workaround I simply disabled it.
dev# make
dev# make install
dev# python
Another failure - error while loading shared libraries: libpython2.5.so.1.0 (it is located in /mnt/C/sys/lib).
As a solution, if there was a dynamic loader configuration on the box (/etc/ld.so.conf.d) we could do following:
dev# echo /usr/local/lib > /etc/ld.so.conf.d/python2.5.conf
dev# ldconfig
As there is not, we can use the LD_LIBRARY_PATH variable instead:
dev# nano /mnt/C/sys/etc/rc-local
Add following:
    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/mnt/C/sys/lib
Now everything works ok.

After some time I realized that there is a readline extension module missing on the system. This module is needed for interactive console applications, and particularly is needed for the deluge console:
box# deluge --ui=null
[INFO    ] 21:58:04 main:93 Deluge ui 1.0.5
[DEBUG   ] 21:58:04 main:94 options: {'config': None, 'logfile': None, 'ui': 'null'}
[DEBUG   ] 21:58:04 main:95 args: []
[DEBUG   ] 21:58:05 configmanager:44 ConfigManager started..
[INFO    ] 21:58:05 main:98 Starting ui..
[DEBUG   ] 21:58:05 ui:44 UI init..
[DEBUG   ] 21:58:05 configmanager:88 Getting config 'ui.conf'
[DEBUG   ] 21:58:05 config:47 Config created with filename: ui.conf
[DEBUG   ] 21:58:05 config:48 Config defaults: {'default_ui': 'gtk'}
[INFO    ] 21:58:05 ui:68 Starting NullUI..
[DEBUG   ] 21:58:06 client:54 CoreProxy init..
Traceback (most recent call last):
  File "/usr/bin/deluge", line 8, in <module>
    load_entry_point('deluge==1.0.5', 'console_scripts', 'deluge')()
  File "/mnt/C/sys/lib/python2.5/site-packages/deluge-1.0.5-py2.5-linux-i686.egg/deluge/main.py", line 99, in start_ui
    UI(options, args)
  File "/mnt/C/sys/lib/python2.5/site-packages/deluge-1.0.5-py2.5-linux-i686.egg/deluge/ui/ui.py", line 69, in __init__
    from deluge.ui.null.deluge_shell import NullUI
  File "/mnt/C/sys/lib/python2.5/site-packages/deluge-1.0.5-py2.5-linux-i686.egg/deluge/ui/null/deluge_shell.py", line 29, in <module>
    import readline
ImportError: No module named readline
We can replicate this problem directly from the python console:
dev# python
Python 2.5.2 (r252:60911, May  3 2008, 16:19:27)
[GCC 3.3.6] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import readline
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named readline
>>>
So let's look at the old log of python's configure script to see why the readline extension package was not built:
dev# cat config.log | grep readline
configure:20980: checking for readline in -lreadline
configure:21015: gcc -pthread -o conftest -g -O2   conftest.c -lreadline  -lpthread -ldl  -lutil >&5
/usr/bin/ld: cannot find -lreadline
| char readline ();
| return readline ();
configure:21126: checking for rl_callback_handler_install in -lreadline
configure:21161: gcc -pthread -o conftest -g -O2   conftest.c -lreadline  -lpthread -ldl  -lutil >&5
/usr/bin/ld: cannot find -lreadline
configure:21254: checking for rl_pre_input_hook in -lreadline
configure:21289: gcc -pthread -o conftest -g -O2   conftest.c -lreadline  -lpthread -ldl  -lutil >&5
/usr/bin/ld: cannot find -lreadline
configure:21325: checking for rl_completion_matches in -lreadline
configure:21360: gcc -pthread -o conftest -g -O2   conftest.c -lreadline  -lpthread -ldl  -lutil >&5
/usr/bin/ld: cannot find -lreadline
ac_cv_lib_readline_readline=no
ac_cv_lib_readline_rl_callback_handler_install=no
ac_cv_lib_readline_rl_completion_matches=no
ac_cv_lib_readline_rl_pre_input_hook=no
We can see that the library simply was not on the system and so the python configure script decided to exclude it's python wrapper from the build.

Before we proceed, we have to build the GNU readline and create a new package for it. How it was done is described in this post.

Now we have the libreadline.so library on the system and so can proceed to re-configure and re-build the python package. First we create a new timestamp file for python package and touch all the old files (in order to be included in the package). We can use the filopack script for that purpose:
dev# cd /mnt/C
dev# ./filopack.sh --init python-2.5.2
dev# ./filopack.sh --touch python-2.5.2
Configuration file .filopack/.config file found and used
Sure to touch all files of the python-2.5.2 locally at /mnt/C (y/n)?y
...
Now we can move to the python source directory and re-configure the build:
dev# cd /usr/local/src/Python-2.5.2
dev# ./configure --prefix=/mnt/C/sys --enable-shared
...
Let's check whether the readline package was found and included to the build:
dev# cat config.log | grep readline
configure:20980: checking for readline in -lreadline
configure:21015: gcc -pthread -o conftest -I/mnt/C/sys/include -I/mnt/C/sys/X11/include  -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib conftest.c -lreadline  -lpthread -ldl  -lutil >&5
/mnt/C/sys/lib/libreadline.so: undefined reference to `tgetnum'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tgoto'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tgetflag'
/mnt/C/sys/lib/libreadline.so: undefined reference to `BC'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tputs'
/mnt/C/sys/lib/libreadline.so: undefined reference to `PC'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tgetent'
/mnt/C/sys/lib/libreadline.so: undefined reference to `UP'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tgetstr'
...
ac_cv_lib_readline_readline=no
ac_cv_lib_readline_rl_callback_handler_install=no
ac_cv_lib_readline_rl_completion_matches=no
ac_cv_lib_readline_rl_pre_input_hook=no
The situation changed, the readline library is (presumably) no longer missing, but still there are some missing symbols and thus the configure test failed.

After a short googling (for query "libreadline.so: undefined reference to BC PC UP") I found this page.
It seems that the missing symbols should be present in ncurses library.

Let's look what symbols are at the library in our system:
dev# nm /lib/libncurses.so | grep "tgetnum\|tgoto\|tgetflag\|BC\|PC\|UP"
0003a544 B BC
0003a770 B PC
0003a540 B UP
000248e1 T tgetflag
00024958 T tgetnum
00024b0c T tgoto
It means that the symbols are there, but the python configuration script did not manage to find the library.

For quick feedback for our experiments we can create a dummy C file and use the same compiler command line as the configure script did:
dev# echo "int main() { return 0; }" > conftest.c
dev# gcc -pthread -o conftest -I/mnt/C/sys/include -I/mnt/C/sys/X11/include  -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib conftest.c -lreadline  -lpthread -ldl  -lutil
/mnt/C/sys/lib/gcc/i386-linux-uclibc/4.1.2/../../../../i386-linux-uclibc/lib/crt1.o: In function `_start':
crt1.S:(.text+0x27): undefined reference to `main'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tgetnum'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tgoto'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tgetflag'
/mnt/C/sys/lib/libreadline.so: undefined reference to `BC'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tputs'
/mnt/C/sys/lib/libreadline.so: undefined reference to `PC'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tgetent'
/mnt/C/sys/lib/libreadline.so: undefined reference to `UP'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tgetstr'
collect2: ld returned 1 exit status
We see that the linker ended up with the same error. Now we can try to fix the command line. There is missing ncurses library on the command line; what about adding it explicitly?
dev# gcc -pthread -o conftest -I/mnt/C/sys/include -I/mnt/C/sys/X11/include  -L/mnt/C/sys/lib
 -L/mnt/C/sys/X11/lib conftest.c -lncurses -lreadline  -lpthread -ldl  -lutil
The test build now works, so now we can remove the temporary C-file and re-configure the python build:
dev# rm conftest.c

dev# LIBS="-lncurses" ./configure --prefix=/mnt/C/sys --enable-shared
...
Let's make sure that readline package passed the configure test:
 
dev# cat config.log | grep readline
configure:20980: checking for readline in -lreadline
configure:21015: gcc -pthread -o conftest -I/mnt/C/sys/include -I/mnt/C/sys/X11/include  -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib conftest.c -lreadline  -lpthread -ldl -lncurses -lutil >&5
configure:21126: checking for rl_callback_handler_install in -lreadline
configure:21161: gcc -pthread -o conftest -I/mnt/C/sys/include -I/mnt/C/sys/X11/include  -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib conftest.c -lreadline  -lreadline -lpthread -ldl -lncurses -lutil >&5
configure:21254: checking for rl_pre_input_hook in -lreadline
configure:21289: gcc -pthread -o conftest -I/mnt/C/sys/include -I/mnt/C/sys/X11/include  -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib conftest.c -lreadline  -lreadline -lpthread -ldl -lncurses -lutil >&5
configure:21325: checking for rl_completion_matches in -lreadline
configure:21360: gcc -pthread -o conftest -I/mnt/C/sys/include -I/mnt/C/sys/X11/include  -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib conftest.c -lreadline  -lreadline -lpthread -ldl -lncurses -lutil >&5
ac_cv_lib_readline_readline=yes
ac_cv_lib_readline_rl_callback_handler_install=yes
ac_cv_lib_readline_rl_completion_matches=yes
ac_cv_lib_readline_rl_pre_input_hook=yes
Yes, everything is ok now, we can build the source and install the binaries:
dev# make
...
/mnt/C/sys/include/sqlite3.h: version 3.5.8
INFO: Can't locate Tcl/Tk libs and/or headers
building 'readline' extension
gcc -pthread -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -I/usr/local/src/Python-2.5.2/./Include -I/mnt/C/sys/include -I. -IInclude -I./Include -I/usr/local/include -I/usr/local/src/Python-2.5.2/Include -I/usr/local/src/Python-2.5.2 -c /usr/local/src/Python-2.5.2/Modules/readline.c -o build/temp.linux-i686-2.5/usr/local/src/Python-2.5.2/Modules/readline.o
gcc -pthread -shared -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes build/temp.linux-i686-2.5/usr/local/src/Python-2.5.2/Modules/readline.o -L/usr/lib/termcap -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib -L/usr/local/lib -L. -lreadline -lncurses -lpython2.5 -o build/lib.linux-i686-2.5/readline.so
building 'nis' extension
gcc -pthread -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -I/usr/local/src/Python-2.5.2/./Include -I/mnt/C/sys/include -I. -IInclude -I./Include -I/usr/local/include -I/usr/local/src/Python-2.5.2/Include -I/usr/local/src/Python-2.5.2 -c /usr/local/src/Python-2.5.2/Modules/nismodule.c -o build/temp.linux-i686-2.5/usr/local/src/Python-2.5.2/Modules/nismodule.o
gcc -pthread -shared -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes build/temp.linux-i686-2.5/usr/local/src/Python-2.5.2/Modules/nismodule.o -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib -L/usr/local/lib -L. -lnsl -lpython2.5 -o build/lib.linux-i686-2.5/nis.so
*** WARNING: renaming "nis" since importing it failed: dynamic module does not define init function (initnis)
running build_scripts

dev# make install
...
Now, let's test the new installation:
dev# python
Python 2.5.2 (r252:60911, Dec  8 2008, 07:26:01)
[GCC 4.1.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import readline
This is fine; and what about the deluge console:
dev# deluge --ui=null
[INFO    ] 07:28:18 main:93 Deluge ui 1.0.5
[DEBUG   ] 07:28:18 main:94 options: {'config': None, 'logfile': None, 'ui': 'null'}
[DEBUG   ] 07:28:18 main:95 args: []
[DEBUG   ] 07:28:18 configmanager:44 ConfigManager started..
[INFO    ] 07:28:18 main:98 Starting ui..
[DEBUG   ] 07:28:18 ui:44 UI init..
[DEBUG   ] 07:28:18 configmanager:88 Getting config 'ui.conf'
[DEBUG   ] 07:28:18 config:47 Config created with filename: ui.conf
[DEBUG   ] 07:28:18 config:48 Config defaults: {'default_ui': 'gtk'}
[INFO    ] 07:28:18 ui:68 Starting NullUI..
[DEBUG   ] 07:28:18 client:54 CoreProxy init..
Welcome to deluge-shell. Type 'help' to see a list of available commands.
> help

Available commands:
        * add: Add a torrent
        * config-set: Change a configuration setting
        * configs: Show configuration values
        * connect: Connect to a new deluge server.
        * del: Remove a torrent
        * exit: Exit from the client.
        * halt: Shutdown the deluge server.
        * help: Show help
        * info: Show information about the torrents
        * pause: Pause a torrent
        * quit: Exit from the client.
        * resume: Resume a torrent
        * rm: Remove a torrent
> quit

Thanks
Everything is now ok, we can create a new python package:
dev# cd /mnt/C
dev# ./filopack.sh --pack python-2.5.2
...
I realized that there are many files which are not a necessary part of the resulting package. Thhose are files with .pyc and .pyo extensions - just in time compiled python files. So I removed them from the list and re-created the package:
dev# ./filopack.sh -R --pack python-2.5.2
...
I have uploaded the resulting package on the web site. Now we can fix the problem on the box.

First let's look at the summary of installed files:
box# cd /mnt/C
box# ./filopack.sh --summary
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)

Filodej package summary:
Package name                          downloaded [#installed/#total]
--------------------------------------------------------------------
...
python-2.5.2                              no           [3525/3648]
...
... so the python package is installed; let's remove it:
box# ./filopack.sh --remove python-2.5.2
Configuration file .filopack/.config file found and used
Sure to remove python-2.5.2 locally at /mnt/C (y/n)?y
...
Now we can download and install both readline-5.2:
box# ./filopack.sh --download readline-5.2
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Downloading package readline-5.2 from http://filodej.ic.cz ...
connected!

Length: 573 [text/plain]
connected!

Length: 265,962 [application/x-tar]

box# ./filopack.sh --install readline-5.2
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Sure to unpack readline-5.2 locally at /mnt/C (y/n)?y
sys/lib/libreadline.a
sys/lib/libhistory.a
sys/lib/libhistory.so.5
sys/lib/libhistory.so.5.2
sys/lib/libhistory.so
sys/lib/libreadline.so.5.2
sys/lib/libreadline.so.5
sys/lib/libreadline.so
sys/man/man3/readline.3
sys/man/man3/history.3
sys/include/readline/rltypedefs.h
sys/include/readline/chardefs.h
sys/include/readline/history.h
sys/include/readline/keymaps.h
sys/include/readline/readline.h
sys/include/readline/rlconf.h
sys/include/readline/rlstdc.h
sys/include/readline/tilde.h
sys/info/readline.info
sys/info/rluserman.info
sys/info/history.info
... and python-2.5.2 package:
box# ./filopack.sh --download python-2.5.2
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Downloading package python-2.5.2 from http://filodej.ic.cz ...
connected!

Length: 134,205 [text/plain]
connected!

Length: 12,984,464 [application/x-tar]

box# ./filopack.sh --install python-2.5.2
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Sure to unpack python-2.5.2 locally at /mnt/C (y/n)?y
...
After the installation everything seemed ok:
box# python
Python 2.5.2 (r252:60911, Dec  8 2008, 07:26:01)
[GCC 4.1.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import readline
>>>
... but surprisingly one more problem arose when I ran the deluge console:
box# deluged.sh

box# deluge --ui=null
[INFO    ] 16:56:44 main:93 Deluge ui 1.0.5
[DEBUG   ] 16:56:44 main:94 options: {'config': None, 'logfile': None, 'ui': 'null'}
[DEBUG   ] 16:56:44 main:95 args: []
[DEBUG   ] 16:56:45 configmanager:44 ConfigManager started..
[INFO    ] 16:56:45 main:98 Starting ui..
[DEBUG   ] 16:56:45 ui:44 UI init..
[DEBUG   ] 16:56:45 configmanager:88 Getting config 'ui.conf'
[DEBUG   ] 16:56:45 config:47 Config created with filename: ui.conf
[DEBUG   ] 16:56:45 config:48 Config defaults: {'default_ui': 'gtk'}
[INFO    ] 16:56:45 ui:68 Starting NullUI..
[DEBUG   ] 16:56:46 client:54 CoreProxy init..
Welcome to deluge-shell. Type 'help' to see a list of available commands.
Traceback (most recent call last):
  File "/usr/bin/deluge", line 8, in 
    load_entry_point('deluge==1.0.5', 'console_scripts', 'deluge')()
  File "/mnt/C/sys/lib/python2.5/site-packages/deluge-1.0.5-py2.5-linux-i686.egg/deluge/main.py", line 99, in start_ui
    UI(options, args)
  File "/mnt/C/sys/lib/python2.5/site-packages/deluge-1.0.5-py2.5-linux-i686.egg/deluge/ui/ui.py", line 70, in __init__
    ui = NullUI(args)
  File "/mnt/C/sys/lib/python2.5/site-packages/deluge-1.0.5-py2.5-linux-i686.egg/deluge/ui/null/deluge_shell.py", line 470, in __init__
    readline.read_init_file()
IOError: [Errno 2] No such file or directory
When we look at the python readline documentation we see the following:
readline.read_init_file([filename])
    Parse a readline initialization file. The default filename is the last filename used.
It seems that there is a readline initialization file missing on the box (probably it was outside the /mnt/C/sys directory tree and so was not transferred to the box).

Let's try to replicate the problem directly from python console:
box# python
Python 2.5.2 (r252:60911, Dec  8 2008, 07:26:01)
[GCC 4.1.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import readline
>>> readline.read_init_file()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IOError: [Errno 2] No such file or directory
When I tried it on the colinux system, everything was ok:
dev# python
Python 2.5.2 (r252:60911, Dec  8 2008, 07:26:01)
[GCC 4.1.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import readline
>>> readline.read_init_file()
>>>
Hopefully the system call trace will help:
box# strace -o readline-init.txt python
Python 2.5.2 (r252:60911, Dec  8 2008, 07:26:01)
[GCC 4.1.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import readline
>>> readline.read_init_file()
Traceback (most recent call last):
  File "<stdin>", line 1, in 
IOError: [Errno 2] No such file or directory
>>>
Let's look at the end of the trace log:
box# tail -n 100 readline-init.txt
...
rt_sigprocmask(SIG_BLOCK, [INT], [RTMIN], 8) = 0
ioctl(0, SNDCTL_TMR_STOP or TCSETSW, {B38400 opost isig icanon echo ...}) = 0
rt_sigprocmask(SIG_SETMASK, [RTMIN], NULL, 8) = 0
rt_sigaction(SIGWINCH, {SIG_DFL}, {0x40139066, [], SA_RESTORER|SA_RESTART, 0x401c45a8}, 8) = 0
time(NULL)                              = 1228752774
stat("/mnt/C/sys/root/.inputrc", 0xbffff4ec) = -1 ENOENT (No such file or directory)
stat("/etc/inputrc", 0xbffff4ec)        = -1 ENOENT (No such file or directory)
write(2, "Traceback (most recent call last"..., 35) = 35
...
write(2, "  File \"<stdin>\", line 1, in <mo"..., 38) = 38
write(2, "IOError", 7)                  = 7
write(2, ": ", 2)                       = 2
write(2, "[Errno 2] No such file or direct"..., 35) = 35
write(2, "\n", 1)                       = 1
ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
rt_sigprocmask(SIG_BLOCK, [INT], [RTMIN], 8) = 0
ioctl(0, TIOCGWINSZ, {ws_row=31, ws_col=84, ws_xpixel=0, ws_ypixel=0}) = 0
ioctl(0, TIOCSWINSZ, {ws_row=31, ws_col=84, ws_xpixel=0, ws_ypixel=0}) = 0
ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(0, SNDCTL_TMR_STOP or TCSETSW, {B38400 opost isig -icanon -echo ...}) = 0
rt_sigprocmask(SIG_SETMASK, [RTMIN], NULL, 8) = 0
rt_sigaction(SIGWINCH, {0x40139066, [], SA_RESTORER|SA_RESTART, 0x401c45a8}, {SIG_DFL}, 8) = 0
write(1, ">>> ", 4)                     = 4
select(1, [0], NULL, NULL, NULL)        = 1 (in [0])
rt_sigprocmask(SIG_BLOCK, NULL, [RTMIN], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [RTMIN], 8) = 0
read(0, "\4", 1)                        = 1
rt_sigprocmask(SIG_BLOCK, [INT], [RTMIN], 8) = 0
ioctl(0, SNDCTL_TMR_STOP or TCSETSW, {B38400 opost isig icanon echo ...}) = 0
rt_sigprocmask(SIG_SETMASK, [RTMIN], NULL, 8) = 0
rt_sigaction(SIGWINCH, {SIG_DFL}, {0x40139066, [], SA_RESTORER|SA_RESTART, 0x401c45a8}, 8) = 0
write(2, "\n", 1)                       = 1
rt_sigaction(SIGINT, {SIG_DFL}, {0x40139066, [], SA_RESTORER, 0x401c45a8}, 8) = 0
munmap(0x402ad000, 16188)               = 0
munmap(0x402b1000, 216100)              = 0
_exit(0)                                = ?
So we found that neither /etc/inputrc nor /mnt/C/sys/root/.inputrc is present on the box and thus the readline initialization complains.

As a solution I choosen to copy the initialization file to root's user directory and re-build the readline package:
dev# cp /etc/inputrc /mnt/C/sys/root/.inputrc
dev# echo "sys/root/.inputrc" >> .filopack/readline-5.2.lst
dev# ./filopack.sh -R --pack readline-5.2
After the re-deployment to the box everything works ok:
dev# deluge --ui=null
[INFO    ] 17:22:53 main:93 Deluge ui 1.0.5
[DEBUG   ] 17:22:53 main:94 options: {'config': None, 'logfile': None, 'ui': 'null'}
[DEBUG   ] 17:22:53 main:95 args: []
[DEBUG   ] 17:22:53 configmanager:44 ConfigManager started..
[INFO    ] 17:22:53 main:98 Starting ui..
[DEBUG   ] 17:22:53 ui:44 UI init..
[DEBUG   ] 17:22:53 configmanager:88 Getting config 'ui.conf'
[DEBUG   ] 17:22:53 config:47 Config created with filename: ui.conf
[DEBUG   ] 17:22:53 config:48 Config defaults: {'default_ui': 'gtk'}
[INFO    ] 17:22:53 ui:68 Starting NullUI..
[DEBUG   ] 17:22:55 client:54 CoreProxy init..
Welcome to deluge-shell. Type 'help' to see a list of available commands.
> help

Available commands:
        * add: Add a torrent
        * config-set: Change a configuration setting
        * configs: Show configuration values
        * connect: Connect to a new deluge server.
        * del: Remove a torrent
        * exit: Exit from the client.
        * halt: Shutdown the deluge server.
        * help: Show help
        * info: Show information about the torrents
        * pause: Pause a torrent
        * quit: Exit from the client.
        * resume: Resume a torrent
        * rm: Remove a torrent

> connect


> info

*** ID: 33820db6dd5e5928d23bc811bbac2f4ae94cb882
*** Name: ubuntu-8.10-desktop-i386.iso
*** Path: torrentfiles
*** Completed: 0.0 KiB/698.8 MiB
*** Status: Paused


> resume

Usage: resume  [ ...]


> resume 33820db6dd5e5928d23bc811bbac2f4ae94cb882


> info

*** ID: 33820db6dd5e5928d23bc811bbac2f4ae94cb882
*** Name: ubuntu-8.10-desktop-i386.iso
*** Path: torrentfiles
*** Completed: 2.1 MiB/698.8 MiB
*** Status: Downloading
*** Download Speed: 60.9 KiB/s
*** Upload Speed: 0.0 KiB/s
*** ETA: 3h 15m


> exit

Thanks
box#
That's all for now.

Read more...

WMU-6500FS - built sw - readline-5.2

[Homepage]

[files] [binary]

First we initialize the package timestamp:
dev# cd /mnt/C/
dev# ./filopack.sh --init readline-5.2
Configuration file .filopack/.config file found and used
Timestamp written to file .filopack/readline-5.2.ts
Now we can download and extract the source to the /usr/local/src directory:
dev# cd /usr/local/src
dev# wget ftp://ftp.gnu.org/gnu/readline/readline-5.2.tar.gz
--06:08:23--  ftp://ftp.gnu.org/gnu/readline/readline-5.2.tar.gz
           => `readline-5.2.tar.gz'
Resolving ftp.gnu.org... 140.186.70.20
Connecting to ftp.gnu.org[140.186.70.20]:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD /gnu/readline ... done.
==> PORT ... done.    ==> RETR readline-5.2.tar.gz ... done.
Length: 2,037,705 (unauthoritative)

100%[===============================================================================================================================>] 2,037,705      4.38K/s    ETA 00:00

06:20:28 (2.75 KB/s) - `readline-5.2.tar.gz' saved [2037705]

dev# tar xzvf readline-5.2.tar.gz
...
Configuration and build process is straightforward:
dev# cd readline-5.2
dev# ./configure --prefix=/mnt/C/sys
...
dev# make
...
dev# make install
...
Now we go back to the /mnt/C directory and finish the package creation:
dev# cd /mnt/C/
dev# ./filopack.sh --pack readline-5.2
Configuration file .filopack/.config file found and used
-rw-r--r-- root/root    308780 2008-12-08 06:26:38 sys/lib/libreadline.a
-rw-r--r-- root/root     37294 2008-12-08 06:26:38 sys/lib/libhistory.a
lrwxrwxrwx root/root         0 2008-12-08 06:26:38 sys/lib/libhistory.so.5 -> libhistory.so.5.2
-r-xr-xr-x root/root     34696 2008-12-08 06:26:38 sys/lib/libhistory.so.5.2
lrwxrwxrwx root/root         0 2008-12-08 06:26:38 sys/lib/libhistory.so -> libhistory.so.5
-r-xr-xr-x root/root    249866 2008-12-08 06:26:38 sys/lib/libreadline.so.5.2
lrwxrwxrwx root/root         0 2008-12-08 06:26:38 sys/lib/libreadline.so.5 -> libreadline.so.5.2
lrwxrwxrwx root/root         0 2008-12-08 06:26:38 sys/lib/libreadline.so -> libreadline.so.5
-rw-r--r-- root/root     38459 2008-12-08 06:26:38 sys/man/man3/readline.3
-rw-r--r-- root/root     21769 2008-12-08 06:26:38 sys/man/man3/history.3
-rw-r--r-- root/root      2977 2008-12-08 06:26:38 sys/include/readline/rltypedefs.h
-rw-r--r-- root/root      4684 2008-12-08 06:26:38 sys/include/readline/chardefs.h
-rw-r--r-- root/root     10207 2008-12-08 06:26:38 sys/include/readline/history.h
-rw-r--r-- root/root      3529 2008-12-08 06:26:38 sys/include/readline/keymaps.h
-rw-r--r-- root/root     34515 2008-12-08 06:26:38 sys/include/readline/readline.h
-rw-r--r-- root/root      2305 2008-12-08 06:26:38 sys/include/readline/rlconf.h
-rw-r--r-- root/root      1503 2008-12-08 06:26:38 sys/include/readline/rlstdc.h
-rw-r--r-- root/root      3167 2008-12-08 06:26:38 sys/include/readline/tilde.h
-rw-r--r-- root/root    192320 2008-12-08 06:26:38 sys/info/readline.info
-rw-r--r-- root/root     72955 2008-12-08 06:26:38 sys/info/rluserman.info
-rw-r--r-- root/root     58583 2008-12-08 06:26:38 sys/info/history.info
One more thing we have to do is to copy the readline initialization file to the root's directory, add it to the .lst file and re-build the package:
dev# cp /etc/inputrc /mnt/C/sys/root/.inputrc
dev# echo "sys/root/.inputrc" >> .filopack/readline-5.2.lst
dev# ./filopack.sh -R --pack readline-5.2

Read more...

2008/12/04

Questions and answers


This post will contain answers for some questions, which can be interesting for others and which are more elaborated so that their scope exceeds a limit reasonable for ordinary comments.

Deluge installation, missing pkg_resources package

Q:

Hi Filodej,
thanks a lot for your reply. I got stuck when i was building the Apache files.
I don't know much about linux, but i'll try and i'll let you know if i get Deluge running

I'm planning to do this:

1.- Flash WMU box with Joker Firmware (already done w/o problems)

2.- Uncompress the binary files that you already build inside the WMU box (i'm not gonna use the chroot-ed system with uClibc library under Kubuntu)
For example:
tar xvzf binary_file

where binary_file are the following dependencies

X.Org
Gtk+
boost
python
pycairo
pygtk
atk
glade
pygobject
pyxdg
DBus
DBus-glib
DBus-python

I'll follow the order you set in your blog

3.- Modify configuration files or set environment variables (if any)

And that's all. I don't know if i can do this, or if i'm missing something. Hope this works! 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Hi Filodej,

i tried what i told you before. I uncompress the binaries and i got an error like "can't load library 'librt.so.0'"
So, i uncompress libs-essential-20080502.tar.bz2 and that fixed the error. But now, when i try to run Deluge i get this error:


[root@Nas_Storage site-packages]# deluge
Traceback (most recent call last):
File "/usr/bin/deluge", line 5, in < module >
from pkg_resources import load_entry_point
ImportError: No module named pkg_resources

I couldn't see that error on your blog and couldn't find any clue to help me solve this, do you have any idea about how to dix this?
Thanks a lot, and sorry for my qustions

Edgard 

A:

The pkg_resources is a part of python setuptools package, you have to install it. I have install it during the Trac installation and so did not encounter this dependency, sorry for that.

When I try it on a relatively clean debian installation (with python 2.5 already installed), I get the same error:
debian:~# python2.5
Python 2.5 (release25-maint, Jul 20 2008, 20:47:25)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkg_resources
Traceback (most recent call last):
  File "", line 1, in 
ImportError: No module named pkg_resources
>>>
In order to install the setuptools package we can follow the installation instructions or as described in the Trac installation (setuptools section) use the ez_setup.py script:

First, we have to download it:
debian:~# cd /tmp
debian:/tmp# wget http://peak.telecommunity.com/dist/ez_setup.py
--04:10:59--  http://peak.telecommunity.com/dist/ez_setup.py
           => `ez_setup.py'
Resolving peak.telecommunity.com... 209.190.5.234
Connecting to peak.telecommunity.com|209.190.5.234|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 9,716 (9.5K) [text/plain]

100%[=======================================================================================================>] 9,716         24.97K/s

04:11:00 (24.97 KB/s) - `ez_setup.py' saved [9716/9716]
Now we can try to launch it:
debian:/tmp# python2.5 ez_setup.py
Downloading http://pypi.python.org/packages/2.5/s/setuptools/setuptools-0.6c9-py2.5.egg
error: invalid Python installation: unable to open /usr/include/python2.5/pyconfig.h (No such file or directory)
It seems we are missing some python C headers, we have to install the package with python 2.5 header files:
debian:/tmp# apt-get install python2.5-dev
Reading package lists... Done
Building dependency tree... Done
The following NEW packages will be installed:
  python2.5-dev
0 upgraded, 1 newly installed, 0 to remove and 36 not upgraded.
Need to get 0B/1676kB of archives.
After unpacking 5349kB of additional disk space will be used.
Selecting previously deselected package python2.5-dev.
(Reading database ... 25024 files and directories currently installed.)
Unpacking python2.5-dev (from .../python2.5-dev_2.5-5+etch1_i386.deb) ...
Setting up python2.5-dev (2.5-5+etch1) ...
Now try it again:
debian:/tmp# python2.5 ez_setup.py
Downloading http://pypi.python.org/packages/2.5/s/setuptools/setuptools-0.6c9-py2.5.egg
Processing setuptools-0.6c9-py2.5.egg
Copying setuptools-0.6c9-py2.5.egg to /usr/lib/python2.5/site-packages
Adding setuptools 0.6c9 to easy-install.pth file
Installing easy_install script to /usr/bin
Installing easy_install-2.5 script to /usr/bin

Installed /usr/lib/python2.5/site-packages/setuptools-0.6c9-py2.5.egg
Processing dependencies for setuptools==0.6c9
Finished processing dependencies for setuptools==0.6c9
Now we can test whether the installation was sucessful:
debian:/tmp# python2.5
Python 2.5 (release25-maint, Jul 20 2008, 20:47:25)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkg_resources
>>>

GCC package is huge, is it really necessary?

Q:

Anonymous said... 
    ... 
    I tried to download 'gcc' from you site but the files seems to be corrupted. Do you have nay ideas ?

    below is a screen-shot from my machine:

    $ tar xjvf gcc-4.1.2-sjlj.tar.bz2.tar
    sys/bin/c++
    sys/bin/cpp
    sys/bin/g++
    sys/bin/gcc
    sys/bin/gccbug
    sys/bin/gcov
    sys/bin/i386-linux-uclibc-c++
    sys/bin/i386-linux-uclibc-g++
    sys/bin/i386-linux-uclibc-gcc
    sys/bin/i386-linux-uclibc-gcc-4.1.2
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/README
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/float.h
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/iso646.h
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/stdarg.h
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/stdbool.h
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/stddef.h
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/varargs.h
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/mmintrin.h
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/mm3dnow.h
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/xmmintrin.h
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/emmintrin.h
    tar: Decompression failed
    tar: Unable to read all data
    tar: Unable to read all data

    regards,
    Christos

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Anonymous said...

    oops...sorry file was corrupted during download - apologies.
    Christos

A:

The truth is that the gcc package is huge (~25MB) though there are just few libraries for deluge. I am going to address this issue so that I cut just a few relevant libraries out of the huge gcc package and provide a new package (~1MB) for it. Following text is by no means necessary for the successful installation, I am just going to describe the process of cutting out relevant dependencies for interested...

First we can list the installed packages. It is possible to use the filopack script for it:
box# cd /mnt/C
box# ./filopack --summary
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)

Filodej package summary:
Package name                          downloaded [#installed/#total]
--------------------------------------------------------------------
...
gcc-4.1.2-sjlj                            no           [ 546/ 549]
...
Now we can uninstall the package:
box# ./filopack.sh --remove gcc-4.1.2-sjlj
Configuration file .filopack/.config file found and used
Sure to remove gcc-4.1.2-sjlj locally at /mnt/C (y/n)?y
Let's make sure the package is actually removed:
box# ./filopack.sh --summary
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)

Filodej package summary:
Package name                          downloaded [#installed/#total]
--------------------------------------------------------------------
...
gcc-4.1.2-sjlj                            no           [   0/ 549]
...
Ok, now when we try to run deluge, we get the "missing library" error message:
box# deluged.sh
/usr/bin/python: can't load library 'libstdc++.so.6'
It seems we need at least the libstdc++.so.6 library. Let's find all the related files:
box# cat .filopack/gcc-4.1.2-sjlj.lst | grep  libstdc++
sys/lib/libstdc++.a
sys/lib/libstdc++.so
sys/lib/libstdc++.so.6.0.8
sys/lib/libstdc++.so.6
sys/lib/libstdc++.la
We actually do not need the static libraries (unless we have a plan to build something directly on the box), so we are interested just in so files:
box# cat .filopack/gcc-4.1.2-sjlj.lst | grep  libstdc++*.so
sys/lib/libstdc++.so
sys/lib/libstdc++.so.6.0.8
sys/lib/libstdc++.so.6
Let's make a new lst file representing the new package deluge-1.0.5-gcclibs we are going to create. This file will contain a list of all the relevant libraries:
box# cat .filopack/gcc-4.1.2-sjlj.lst | grep  libstdc++*.so > .filopack/deluge-1.0.5-gcclibs.lst
... and extract just those files from the gcc-4.1.2-sjlj package:
box# cat .filopack/gcc-4.1.2-sjlj.lst | grep libstdc++*.so | xargs tar -xjvf gcc-4.1.2-sjlj.tar.bz2
sys/lib/libstdc++.so
sys/lib/libstdc++.so.6.0.8
sys/lib/libstdc++.so.6
When we try to start deluge daemon again:
box# deluged.sh
/usr/bin/python: Can't resolve symbol '_Unwind_SjLj_Register'
... it seems there are still some missing libraries (or more likely some missing symbols in an older version of the same library). Let's look at the dependencies more closer:
box# LD_TRACE_LOADED_OBJECTS=1 deluged.sh
        /usr/lib/libssl.so.0.9.7 => /usr/lib/libssl.so.0.9.7 (0x40007000)
        /usr/lib/libboost_filesystem-gcc41-mt-1_35.so.1.35.0 => /usr/lib/libboost_filesystem-gcc41-mt-1_35.so.1.35.0 (0x40030000)
        libpython2.5.so.1.0 => /mnt/C/sys/lib//libpython2.5.so.1.0 (0x4003f000)
        libpthread.so.0 => /mnt/C/sys/lib//libpthread.so.0 (0x40166000)
        libdl.so.0 => /lib/libdl.so.0 (0x40178000)
        libutil.so.0 => /mnt/C/sys/lib//libutil.so.0 (0x4017b000)
        libm.so.0 => /lib/libm.so.0 (0x4017d000)
        libc.so.0 => /lib/libc.so.0 (0x4018b000)
        libcrypto.so.0.9.7 => /mnt/C/sys/lib//libcrypto.so.0.9.7 (0x4021f000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x402f4000)
        libboost_system-gcc41-mt-1_35.so.1.35.0 => /mnt/C/sys/lib//libboost_system-gcc41-mt-1_35.so.1.35.0 (0x402fc000)
        librt.so.0 => /mnt/C/sys/lib//librt.so.0 (0x40300000)
        libstdc++.so.6 => /mnt/C/sys/lib//libstdc++.so.6 (0x40302000)
        ld-uClibc.so.0 => /lib/ld-uClibc.so.0 (0x40000000)
Here is a list of libraries present in the gcc package:
dev# cat .filopack/gcc-4.1.2-sjlj.lst | grep "\.so"
sys/lib/libgcc_s.so
sys/lib/libgcc_s.so.1
sys/lib/libstdc++.so
sys/lib/libstdc++.so.6.0.8
sys/lib/libstdc++.so.6
sys/lib/libmudflap.so.0.0.0
sys/lib/libmudflap.so.0
sys/lib/libmudflap.so
sys/lib/libmudflapth.so.0.0.0
sys/lib/libmudflapth.so.0
sys/lib/libmudflapth.so
sys/lib/libssp.so.0.0.0
sys/lib/libssp.so.0
sys/lib/libssp.so
The libgcc_s looks like a good candidate. We can look for the missing symbol:
box# nm /lib/libgcc_s.so.1
nm: /lib/libgcc_s.so.1: no symbols

box# nm --dynamic /lib/libgcc_s.so | grep Unwind
00003ebc T _Unwind_Backtrace
00003e98 T _Unwind_DeleteException
00002800 T _Unwind_FindEnclosingFunction
00004e2c T _Unwind_Find_FDE
00003c50 T _Unwind_ForcedUnwind
000027dc T _Unwind_GetCFA
00002838 T _Unwind_GetDataRelBase
00003f60 T _Unwind_GetGR
00003f84 T _Unwind_GetIP
000027e8 T _Unwind_GetLanguageSpecificData
000027f4 T _Unwind_GetRegionStart
00002844 T _Unwind_GetTextRelBase
00003a08 T _Unwind_RaiseException
00003d00 T _Unwind_Resume
00003dc8 T _Unwind_Resume_or_Rethrow
00003f70 T _Unwind_SetGR
00003f90 T _Unwind_SetIP
The /lib/libgcc_s.so is an "old" version of the library being replaced by the new one contained in the gcc package. Apparently the symbol '_Unwind_SjLj_Register' is missing in the list, but there are similar symbols.
Let's list the libgcc related files:
box# cat .filopack/gcc-4.1.2-sjlj.lst | grep libgcc_s*.so
sys/lib/libgcc_s.so
sys/lib/libgcc_s.so.1
Now we can extract the files from the package tarball:
box# cat .filopack/gcc-4.1.2-sjlj.lst | grep libgcc_s*.so | xargs tar -xjvf gcc-4.1.2-sjlj.tar.bz2
sys/lib/libgcc_s.so
sys/lib/libgcc_s.so.1
Make sure that the symbol is actually there:
box# nm --dynamic /mnt/C/sys/lib/libgcc_s.so | grep Unwind
00005361 T _Unwind_Backtrace
00005345 T _Unwind_DeleteException
0000522e T _Unwind_FindEnclosingFunction
00004e4b T _Unwind_Find_FDE
000051d6 T _Unwind_GetCFA
00005235 T _Unwind_GetDataRelBase
000051c5 T _Unwind_GetGR
000051fb T _Unwind_GetIP
0000521a T _Unwind_GetLanguageSpecificData
00005227 T _Unwind_GetRegionStart
0000523c T _Unwind_GetTextRelBase
000051e7 T _Unwind_SetGR
00005209 T _Unwind_SetIP
000054a5 T _Unwind_SjLj_ForcedUnwind
000054ef T _Unwind_SjLj_RaiseException
0000511c T _Unwind_SjLj_Register
00005452 T _Unwind_SjLj_Resume
0000557d T _Unwind_SjLj_Resume_or_Rethrow
000051ba T _Unwind_SjLj_Unregister
Now we can append the files to the lst file.
box# cat .filopack/gcc-4.1.2-sjlj.lst | grep libgcc_s*.so >> .filopack/deluge-1.0.5-gcclibs.lst
And test the deluge again:
[root@storage C]# deluged.sh
... now the daemon starts without problems.
The last step is to create the deluge-1.0.5-gcclibs package.
box# cat .filopack/deluge-1.0.5-gcclibs.lst
sys/lib/libstdc++.so
sys/lib/libstdc++.so.6.0.8
sys/lib/libstdc++.so.6
sys/lib/libgcc_s.so
sys/lib/libgcc_s.so.1
We can use the --pack action of the filopack script with -R option in order to reuse the (already created) lst file:
box# ./filopack.sh -R --pack deluge-1.0.5-gcclibs
Configuration file .filopack/.config file found and used
tar: unrecognized option `--files-from=.filopack/deluge-1.0.5-gcclibs.lst'
BusyBox v1.90-JKR (2008.04.09-06:53 CET) multi-call binary

Usage: tar -[czjZxtvO] [-f TARFILE] [-C DIR] [FILE(s)] ...

Create, extract, or list files from a tar file.

Options:
        c               create
        x               extract
        t               list

Archive format selection:
        z               Filter the archive through gzip
        j               Filter the archive through bzip2
        Z               Filter the archive through compress

File selection:
        f               name of TARFILE or "-" for stdin
        O               extract to stdout
        C               change to directory DIR before operation
        v               verbosely list files processed

... oops, it seems that I have different versions of tar on my box and on my dev# system (Chroot-ed environment on top of the Colinux with Debian), there the --files-from works ok.
Let's make sure. First find the tar on the box:
box# which tar
/bin/tar
box# ls -l /bin/tar
lrwxrwxrwx 1 root root 7 2008-05-21 13:40 /bin/tar -> busybox
And now on the dev# system:
dev# tar --version
tar (GNU tar) 1.15.1

dev# which tar
/bin/tar

dev# ls -l /bin/tar
-rwxr-xr-x  1 root root 166880 Apr 16  2008 /bin/tar*
As a solution I have updated the filopack script to be compliant with the busybox tar version. I changed the following statement:
tar $PACK_CMD $PACKAGE.$EXT --files-from $CONFIG_DIR/$PACKAGE.lst 2>&1 | tee $CONFIG_DIR/$PACKAGE.err
to the following:
cat $CONFIG_DIR/$PACKAGE.lst | xargs tar $PACK_CMD $PACKAGE.$EXT 2>&1 | tee $CONFIG_DIR/$PACKAGE.err
.. the only thing I am not sure of is whether the number of command line argument is somewhat limited; such limitation could cause some problems for packages with many files.
Let's try it directly:
box# ./filopack.sh -R --pack deluge-1.0.5-gcclibs
Configuration file .filopack/.config file found and used
tar: Creating bzip2 compressed archives is not currently supported.
Equivalent command:
cat .filopack/deluge-1.0.5-gcclibs.lst | xargs tar cjvf deluge-1.0.5-gcclibs.tar.bz2

Bzip compression is not supported, so try the tgz:
box# ./filopack.sh -R -o tgz --pack deluge-1.0.5-gcclibs
Configuration file .filopack/.config file found and used
sys/lib/libgcc_s.so.1
sys/lib/libgcc_s.so
sys/lib/libstdc++.so.6
sys/lib/libstdc++.so.6.0.8
sys/lib/libstdc++.so
Equivalent command:
cat .filopack/deluge-1.0.5-gcclibs.lst | xargs tar czvf deluge-1.0.5-gcclibs.tgz

... this works. We have successfully created the package.
Note: Finally I created the bz2 version on my mirror system.
dev# ./filopack.sh -R --pack deluge-1.0.5-gcclibs
...
So, that's all for now. I have uploaded the package and now it is publicly available:
box# ./filopack.sh --packages
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Available packages:
...
deluge-1.0.5
deluge-1.0.5-deplibs
...
You can download it to your box:
box# ./filopack.sh --download deluge-1.0.5-gcclibs
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Downloading package deluge-1.0.5-gcclibs from http://filodej.ic.cz ...
connected!

Length: 113 [text/plain]
connected!

Length: 1,191,263 [application/x-tar]
And install it as follows:
box# ./filopack.sh --install deluge-1.0.5-gcclibs
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Sure to unpack deluge-1.0.5-gcclibs locally at /mnt/C (y/n)?y
sys/lib/libstdc++.so
sys/lib/libstdc++.so.6.0.8
sys/lib/libstdc++.so.6
sys/lib/libgcc_s.so
sys/lib/libgcc_s.so.1

Unresolved 'PyUnicodeUCS2_DecodeUTF8' symbol

Q:

$ deluged.sh
/usr/bin/python: Can't resolve symbol '_Unwind_SjLj_Register'
$
$
$ LD_PRELOAD="/usr/lib/libssl.so.0.9.7 /usr/lib/libboost_filesystem-gcc41-mt-1_35.so.1.35.0 /usr/lib/libgcc_s.so" deluged -d
[DEBUG ] 08:53:55 configmanager:44 ConfigManager started..
[INFO ] 08:53:55 daemon:44 Deluge daemon 1.0.5
[DEBUG ] 08:53:55 daemon:45 options: {'logfile': None, 'config': None, 'port': None, 'donot': True}
[DEBUG ] 08:53:55 daemon:46 args: []
/usr/bin/python: can't resolve symbol 'PyUnicodeUCS2_DecodeUTF8'
$

A:

Likely this symbol should be defined in python library. Let's look if it is the case:
box# nm /mnt/C/sys/lib/libpython2.5.so.1.0 | grep DecodeUTF8
00073f08 T PyUnicodeUCS4_DecodeUTF8
00073f30 T PyUnicodeUCS4_DecodeUTF8Stateful
So it seems there is a UCS4 vs. UCS2 mismatch on your system. When I googled PyUnicodeUCS4 PyUnicodeUCS2 I got the following explanation:
You are using a version of Python that uses a 4-byte representation for Unicode characters, but some C extension module you are importing was compiled using a Python that uses a 2-byte representation for Unicode characters (the default).
So it seems to be exactly the case on your system. Look at the python console just to make sure:
box# python
Python 2.5.2 (r252:60911, May  3 2008, 16:19:27)
[GCC 3.3.6] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print sys.maxunicode > 65535 and 'UCS4 build' or 'UCS2 build'
UCS4 build
>>>
Now we have to find out what is the python extension to be blamed, but unfortunately I cannot do it on my own since it seems there is no such problem on my box. At least I can try to help you a bit how to investigate that.

You can use strace (system calls and signals tracer) for that. Here is the a short explanation how I built it and here you can directly download the package, or possibly if you are using the filopack system you can simply write:
$ cd /mnt/C
$ ./filopack.sh --download strace-4.5.15
$ ./filopack.sh --install strace-4.5.15
... and (hopefully) you are prepared.

To replicate a scenario similar to yours, I can run the deluge daemon without the LD_PRELOAD prefix, in such case I am also missing some symbols:
box# /usr/bin/deluged
box# /usr/bin/python: can't resolve symbol '__cxa_pure_virtual'
So now I can try to trace all the system calls:
box# strace /usr/bin/deluged
...
stat64("/mnt/C/sys/root/.config/deluge", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0
fork()                                  = 17025
_exit(0)                                = ?
Process 17020 detached
box# /usr/bin/python: can't resolve symbol '__cxa_pure_virtual'
It seems that the loader problem was in another instance of deluged process. Let's look at strace help to see what we can do about it:
box# strace -h
usage: strace [-dffhiqrtttTvVxx] [-a column] [-e expr] ... [-o file]
              [-p pid] ... [-s strsize] [-u username] [-E var=val] ...
              [command [arg ...]]
   or: strace -c [-e expr] ... [-O overhead] [-S sortby] [-E var=val] ...
              [command [arg ...]]
-c -- count time, calls, and errors for each syscall and report summary
-f -- follow forks, -ff -- with output into separate files
-F -- attempt to follow vforks, -h -- print help message
-i -- print instruction pointer at time of syscall
-q -- suppress messages about attaching, detaching, etc.
-r -- print relative timestamp, -t -- absolute timestamp, -tt -- with usecs
-T -- print time spent in each syscall, -V -- print version
-v -- verbose mode: print unabbreviated argv, stat, termio[s], etc. args
-x -- print non-ascii strings in hex, -xx -- print all strings in hex
-a column -- alignment COLUMN for printing syscall results (default 40)
-e expr -- a qualifying expression: option=[!]all or option=[!]val1[,val2]...
   options: trace, abbrev, verbose, raw, signal, read, or write
-o file -- send trace output to FILE instead of stderr
-O overhead -- set overhead for tracing syscalls to OVERHEAD usecs
-p pid -- trace process with process id PID, may be repeated
-s strsize -- limit length of print strings to STRSIZE chars (default 32)
-S sortby -- sort syscall counts by: time, calls, name, nothing (default time)
-u username -- run command as username handling setuid and/or setgid
-E var=val -- put var=val in the environment for command
-E var -- remove var from the environment for command
Now we can give it another try, this time with the -f option:
box# strace -f /usr/bin/deluged
...
...
close(7)                                = 0
munmap(0x40006000, 4096)                = 0
open("./libbz2.so.1.0", O_RDONLY)       = -1 ENOENT (No such file or directory)
open("/mnt/C/sys/lib//libbz2.so.1.0", O_RDONLY) = 7
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40006000
read(7, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0$\21\0\000"..., 4096) = 4096
old_mmap(NULL, 57344, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40e57000
old_mmap(0x40e57000, 52556, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 7, 0) = 0x40e57000
old_mmap(0x40e64000, 3712, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 7, 0xd000) = 0x40e64000
close(7)                                = 0
munmap(0x40006000, 4096)                = 0
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40006000
write(2, "", 0)                         = 0
write(2, "/usr/bin/python", 15/usr/bin/python)         = 15
write(2, ": can\'t resolve symbol \'", 24: can't resolve symbol ') = 24
write(2, "__cxa_pure_virtual", 18__cxa_pure_virtual)      = 18
write(2, "\'\n", 2'
)                     = 2
munmap(0x40006000, 4096)                = 0
_exit(1)                                = ?
Process 17039 detached
Most likely the libbz2.so module is requiring the unresolved __cxa_pure_virtual symbol.

Back to your problem, once we know the particular python extension requiring the 2 byte unicode version of the DecodeUTF8 method we can try to re-build or re-install it appropriately...

Part II.

Dear Filodej,

you are right! I have followed your instructions and indeed this is the problem. 
I hope that the console output after executing strace commandes would be found 
usefull for you.
Here is the complete output from executing strace, and here for the strace -f.

Thank you in advance, for helping me out :-) hope others will benefit as well.

Regards,
Christos
Good, so the relevent part of your trace is here:
$ strace -f /usr/bin/deluged
execve("/usr/bin/deluged", ["/usr/bin/deluged"], [/* 31 vars */]) = 0
...
ioctl(9, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbffeddc8) = -1 ENOTTY (Inappropriate ioctl for device)
fstat64(9, {st_mode=S_IFREG|0755, st_size=411371, ...}) = 0
open("/usr/lib/python2.5/site-packages/_xmlplus/parsers/pyexpat.so", O_RDONLY) = 10
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40006000
read(10, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0$7\0\000"..., 4096) = 4096
old_mmap(NULL, 180224, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4059c000
old_mmap(0x4059c000, 167720, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 10, 0) = 0x4059c000
old_mmap(0x405c5000, 9872, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 10, 0x29000) = 0x405c5000
close(10)                               = 0
munmap(0x40006000, 4096)                = 0
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40006000
write(2, "", 0)                         = 0
write(2, "/usr/bin/python", 15/usr/bin/python)         = 15
write(2, ": can\'t resolve symbol \'", 24: can't resolve symbol ') = 24
write(2, "PyUnicodeUCS2_DecodeUTF8", 24PyUnicodeUCS2_DecodeUTF8) = 24
write(2, "\'\n", 2'
)                     = 2
munmap(0x40006000, 4096)                = 0
_exit(1)                                = ?
Process 2748 detached
I wonder where did you get the pyexpat library since I did not build the PyXML as part of my python package. Let's look at the JoKeR's package (it seems as a good candidate):
box# wget http://mgb111.pradnik.net/addons/soft-lang/Python-25-PyXML-084.tar.gz
connected!

Length: 17,194,098 [application/x-gzip]

box# tar tzf Python-25-PyXML-084.tar.gz | grep pyexpat
./sys/lib/python2.5/site-packages/_xmlplus/parsers/pyexpat.so
./sys/lib/python2.5/site-packages/_xmlplus/sax/drivers/drv_pyexpat.py
./sys/lib/python2.5/site-packages/_xmlplus/sax/drivers/drv_pyexpat.pyc
./sys/lib/python2.5/site-packages/_xmlplus/sax/drivers2/drv_pyexpat.py
./sys/lib/python2.5/site-packages/_xmlplus/sax/drivers2/drv_pyexpat.pyc
./sys/lib/python2.5/test/output/test_pyexpat
./sys/lib/python2.5/test/test_pyexpat.py
./sys/lib/python2.5/test/test_pyexpat.pyc
./sys/lib/python2.5/test/test_pyexpat.pyo
./sys/lib/python2.5/lib-dynload/pyexpat.so
Bingo! You probably did not remove the JoKeR's installation and just extract my python package over it.
Since I have built python with different unicode settings that JoKeR (it is nothing like Microsoft's intended incompatibilities, I swear I had no idea about it at the time ;-) now you are experiencing the missing symbol(s).

Just to make sure I am right in my assumption, I try to replicate your problem on my box. Let's extract the _xmlplus directory from JoKeR's package:
box# tar xzvf Python-25-PyXML-084.tar.gz ./sys/lib/python2.5/site-packages/_xmlplus/*
...
Now I can try to do some python xml parsing and see what happens:
box# strace -o st.txt /usr/bin/python
Python 2.5.2 (r252:60911, May  3 2008, 16:19:27)
[GCC 3.3.6] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import xml.dom.minidom
>>> xml.dom.minidom.parseString( 'test')
/usr/bin/python: can't resolve symbol 'PyUnicodeUCS2_DecodeUTF8'
Great, now I am in the same trouble as you ;-)
Let's look at the trace log:
box# tail -n 20 st.txt
stat64("/usr/lib/python2.5/site-packages/_xmlplus/parsers/pyexpat", 0xbfff4a4c) = -1 ENOENT (No such file or directory)
open("/usr/lib/python2.5/site-packages/_xmlplus/parsers/pyexpat.so", O_RDONLY|O_LARGEFILE) = 5
ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfff49e8) = -1 ENOTTY (Inappropriate ioctl for device)
fstat64(5, {st_mode=S_IFREG|0755, st_size=411371, ...}) = 0
open("/usr/lib/python2.5/site-packages/_xmlplus/parsers/pyexpat.so", O_RDONLY) = 6
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40006000
read(6, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0$7\0\000"..., 4096) = 4096
old_mmap(NULL, 180224, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40310000
old_mmap(0x40310000, 167720, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 6, 0) = 0x40310000
old_mmap(0x40339000, 9872, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 6, 0x29000) = 0x40339000
close(6)                                = 0
munmap(0x40006000, 4096)                = 0
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40006000
write(2, "", 0)                         = 0
write(2, "/usr/bin/python", 15)         = 15
write(2, ": can\'t resolve symbol \'", 24) = 24
write(2, "PyUnicodeUCS2_DecodeUTF8", 24) = 24
write(2, "\'\n", 2)                     = 2
munmap(0x40006000, 4096)                = 0
_exit(1)                                = ?
and try to launch the deluge daemon:
box# deluged.sh
box# /usr/bin/python: can't resolve symbol 'PyUnicodeUCS2_DecodeUTF8'
Ok, now I am pretty sure this is exactly your situation. Let's remove the _xmlplus package again, we cannot use it anyway, since it is built incompatibly with the rest of the python framework:
box# rm -d -r ./sys/lib/python2.5/site-packages/_xmlplus/
Now let's try the xml parsing and launch the deluge daemon again:
box# python
Python 2.5.2 (r252:60911, May  3 2008, 16:19:27)
[GCC 3.3.6] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import xml.dom.minidom
>>> xml.dom.minidom.parseString( '<a>test</a>')
<xml.dom.minidom.Document instance at 0x4027940c>
>>>

box# deluged.sh
box# 
Everything works again.

The removal of the _xmlplus pyhon package should help in your case as well, but note that there could be other incompatible files. The ultimate solution would be to remove both python packages (mine and JoKeR's) and install just mine again. In case you needed the PyXML package (which seems unlikely to me, since it does not work for you anyway) I could sometimes build one compatible with 4-byte unicode.

Unresolved 'g_assertion_message_expr' symbol

Q:

Hello again,

I have erased everything from my box in order to perform a clean installation based on your instructions. 
Unfortunately, this didn't go as it was expected.
Initially I installed Deluge and all of the dependencies, apart from setuptools. 
This was due to the fact that I couldn't run python cause there was a library .so-something missing 
- unfortunately I did not write down the name.
In order to fix this (and hopefully it was fixed) I downloaded & installed Joker's "minimal-config" & "libs-essential".

Now when I run:
LD_PRELOAD="/usr/lib/libssl.so.0.9.7 /usr/lib/libboost_filesystem-gcc41-mt-1_35.so.1.35.0 /usr/lib/libgcc_s.so" deluged -d

I am getting the following error:
/usr/bin/python: can't resolve symbol 'g_assertion_message_expr'
...
regards,
Christos

A:

First I googled a bit and found that the g_assertion_message_expr symbol should be present in the libglib-2.0.so library. It seemed little bit weird that you are experiencing such error, since this library is listed in my list of dependencies. It is clear that it needs a deeper analysis...

First let's make sure the libglib library is really present on the system:
box# ls -l /mnt/C/sys/lib/libglib*
-rwxr-xr-x 1 root root     878 2008-06-30 00:06 /mnt/C/sys/lib/libglib-2.0.la
lrwxrwxrwx 1 root root      23 2008-12-03 23:04 /mnt/C/sys/lib/libglib-2.0.so -> libglib-2.0.so.0.1702.0
lrwxrwxrwx 1 root root      23 2008-12-03 23:04 /mnt/C/sys/lib/libglib-2.0.so.0 -> libglib-2.0.so.0.1702.0
-rwxr-xr-x 1 root root 2631967 2008-04-17 12:34 /mnt/C/sys/lib/libglib-2.0.so.0.1400.0
rwxr-xr-x 1 root root 1131687 2008-06-30 00:06 /mnt/C/sys/lib/libglib-2.0.so.0.1702.0
... as can be seen there are two of them but just one of them is linked. This fact could be related to your problem.

We can learn what is present in my glib package:
box# ./filopack.sh --download glib-2.17.2                                 
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Downloading package glib-2.17.2 from http://filodej.ic.cz ...
connected!

Length: 21,692 [text/plain]
connected!

Length: 1,947,202 [application/x-tar]

box# tar tjvf glib-2.17.2.tar.bz2 | grep libglib
-rwxr-xr-x 0/0   1131687 2008-06-30 00:06:55 sys/lib/libglib-2.0.so.0.1702.0
lrwxrwxrwx 0/0         0 2008-06-30 00:06:55 sys/lib/libglib-2.0.so.0 -> libglib-2.0.so.0.1702.0
lrwxrwxrwx 0/0         0 2008-06-30 00:06:55 sys/lib/libglib-2.0.so -> libglib-2.0.so.0.1702.0
-rwxr-xr-x 0/0       878 2008-06-30 00:06:55 sys/lib/libglib-2.0.la
So we see that the 2.17.2 version comes from my package. What about the other one?

You write that you have downloaded & installed Joker's "minimal-config" & "libs-essential", so let's look at that:
box# wget http://mgb111.pradnik.net/addons/libs-essential-20080502.tar.bz2
connected!

Length: 13,454,268 [application/x-bzip2]

box# tar tjvf libs-essential-20080502.tar.bz2 | grep libglib
-rwxr-xr-x 0/0       834 2008-04-17 12:34:55 ./sys/lib/libglib-2.0.la
lrwxrwxrwx 0/0         0 2008-05-10 20:41:07 ./sys/lib/libglib-2.0.so -> libglib-2.0.so.0.1400.0
lrwxrwxrwx 0/0         0 2008-05-10 20:41:07 ./sys/lib/libglib-2.0.so.0 -> libglib-2.0.so.0.1400.0
-rwxr-xr-x 0/0   2631967 2008-04-17 12:34:55 ./sys/lib/libglib-2.0.so.0.1400.0
... ok, so another one (2.14.0 version) is from JoKeR's package.

Let's dig a little bit deeper, what about symbols?
box# nm --dynamic /mnt/C/sys/lib/libglib-2.0.so.0.1702.0 | grep g_assertion_message_expr
0006409b T g_assertion_message_expr
box# nm --dynamic /mnt/C/sys/lib/libglib-2.0.so.0.1400.0 | grep g_assertion_message_expr
so it seems that the g_assertion_message_expr symbol was introduced right between 14.0 and 17.2 versions (or possibly the lack of it in the older version is caused by different build configuration). In any case the problem was caused by the fact that you extracted JoKeR's essential-libs package AFTER my glib package and thus effectively downgraded the glib from 2.17.2 version down to 2.14.0.

Sorry for that complication, I built my system on top of JoKeR's minimal-config and libs-essential and I mention it in the post about the box configuration; but it is sure my fault that I did not mention it explicitly among the deluge dependencies. I am doing it now to prevent further confusion.

That's all for now. You can now redirect the symlinks (libglib-2.0.so and libglib-2.0.so.0) by hand or better re-install my glib package, supposedly it should fix your problem. As can be seen such subtle details like installation order can sometimes make a big difference :-)

Missing bash shell

Q:

...
Anyway, I then ran the "./filopack.sh --download deluge-1.0.5" line, and I got this: "-ash: ./filopack.sh: not found." Now, that's not good, right?

Any idea what it is that I'm doing wrong please?
...

A:

In a bash session with bash shell installed on the system the filopack script is running fine:
box# ./filopack.sh
    
Stupid packaging system (version 0.3.7)

Usage:
 
  filopack.sh [OPTIONS] [] []

Basic usage:
  ...
When we look at the first line of the script there is bash shell specified as an interpreter:
box# head -n1 ./filopack.sh
#!/bin/bash
Let's look where is the bash shell installed:
box# which bash
/usr/bin/bash
The /bin directory is on a read-only file system, but /bin/bash is just a symlink to /usr/bin/bash:
box# ls -l /bin/bash 
lrwxrwxrwx 1 root root 19 2008-05-21 13:40 /bin/bash -> /mnt/C/sys/bin/bash
Let's try to "delete" the bash shell for a moment:
box# mv /usr/bin/bash{,.del}
Now when we try to run the filopack script:
 
box# ./filopack.sh           
-bash: ./filopack.sh: /bin/bash: bad interpreter: No such file or directory
The bash shell (we are still running the bash session even if the bash executable is removed) is sufficiently verbose in its error messages so the cause of the problem is obvious. Let's try the same in ash shell:
box# ash
box$ cd /mnt/C/
box$ ./filopack.sh
ash: ./filopack.sh: not found
Ok now we have replicated your problem. You have to install the bash shell in order to run the filopack script.

Read more...