Questions and answers

This post will contain answers for some questions, which can be interesting for others and which are more elaborated so that their scope exceeds a limit reasonable for ordinary comments.

Deluge installation, missing pkg_resources package

Q:

Hi Filodej,
thanks a lot for your reply. I got stuck when i was building the Apache files.
I don't know much about linux, but i'll try and i'll let you know if i get Deluge running

I'm planning to do this:

1.- Flash WMU box with Joker Firmware (already done w/o problems)

2.- Uncompress the binary files that you already build inside the WMU box (i'm not gonna use the chroot-ed system with uClibc library under Kubuntu)
For example:
tar xvzf binary_file

where binary_file are the following dependencies

X.Org
Gtk+
boost
python
pycairo
pygtk
atk
glade
pygobject
pyxdg
DBus
DBus-glib
DBus-python

I'll follow the order you set in your blog

3.- Modify configuration files or set environment variables (if any)

And that's all. I don't know if i can do this, or if i'm missing something. Hope this works! 

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Hi Filodej,

i tried what i told you before. I uncompress the binaries and i got an error like "can't load library 'librt.so.0'"
So, i uncompress libs-essential-20080502.tar.bz2 and that fixed the error. But now, when i try to run Deluge i get this error:


[root@Nas_Storage site-packages]# deluge
Traceback (most recent call last):
File "/usr/bin/deluge", line 5, in < module >
from pkg_resources import load_entry_point
ImportError: No module named pkg_resources

I couldn't see that error on your blog and couldn't find any clue to help me solve this, do you have any idea about how to dix this?
Thanks a lot, and sorry for my qustions

Edgard 

A:

The pkg_resources is a part of python setuptools package, you have to install it. I have install it during the Trac installation and so did not encounter this dependency, sorry for that.

When I try it on a relatively clean debian installation (with python 2.5 already installed), I get the same error:

debian:~# python2.5
Python 2.5 (release25-maint, Jul 20 2008, 20:47:25)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkg_resources
Traceback (most recent call last):
  File "", line 1, in 
ImportError: No module named pkg_resources
>>>

In order to install the setuptools package we can follow the installation instructions or as described in the Trac installation (setuptools section) use the ez_setup.py script:

First, we have to download it:

debian:~# cd /tmp
debian:/tmp# wget http://peak.telecommunity.com/dist/ez_setup.py
--04:10:59--  http://peak.telecommunity.com/dist/ez_setup.py
           => `ez_setup.py'
Resolving peak.telecommunity.com... 209.190.5.234
Connecting to peak.telecommunity.com|209.190.5.234|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 9,716 (9.5K) [text/plain]

100%[=======================================================================================================>] 9,716         24.97K/s

04:11:00 (24.97 KB/s) - `ez_setup.py' saved [9716/9716]

Now we can try to launch it:

debian:/tmp# python2.5 ez_setup.py
Downloading http://pypi.python.org/packages/2.5/s/setuptools/setuptools-0.6c9-py2.5.egg
error: invalid Python installation: unable to open /usr/include/python2.5/pyconfig.h (No such file or directory)

It seems we are missing some python C headers, we have to install the package with python 2.5 header files:

debian:/tmp# apt-get install python2.5-dev
Reading package lists... Done
Building dependency tree... Done
The following NEW packages will be installed:
  python2.5-dev
0 upgraded, 1 newly installed, 0 to remove and 36 not upgraded.
Need to get 0B/1676kB of archives.
After unpacking 5349kB of additional disk space will be used.
Selecting previously deselected package python2.5-dev.
(Reading database ... 25024 files and directories currently installed.)
Unpacking python2.5-dev (from .../python2.5-dev_2.5-5+etch1_i386.deb) ...
Setting up python2.5-dev (2.5-5+etch1) ...

Now try it again:

debian:/tmp# python2.5 ez_setup.py
Downloading http://pypi.python.org/packages/2.5/s/setuptools/setuptools-0.6c9-py2.5.egg
Processing setuptools-0.6c9-py2.5.egg
Copying setuptools-0.6c9-py2.5.egg to /usr/lib/python2.5/site-packages
Adding setuptools 0.6c9 to easy-install.pth file
Installing easy_install script to /usr/bin
Installing easy_install-2.5 script to /usr/bin

Installed /usr/lib/python2.5/site-packages/setuptools-0.6c9-py2.5.egg
Processing dependencies for setuptools==0.6c9
Finished processing dependencies for setuptools==0.6c9

Now we can test whether the installation was sucessful:

debian:/tmp# python2.5
Python 2.5 (release25-maint, Jul 20 2008, 20:47:25)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkg_resources
>>>

GCC package is huge, is it really necessary?

Q:

Anonymous said... 
    ... 
    I tried to download 'gcc' from you site but the files seems to be corrupted. Do you have nay ideas ?

    below is a screen-shot from my machine:

    $ tar xjvf gcc-4.1.2-sjlj.tar.bz2.tar
    sys/bin/c++
    sys/bin/cpp
    sys/bin/g++
    sys/bin/gcc
    sys/bin/gccbug
    sys/bin/gcov
    sys/bin/i386-linux-uclibc-c++
    sys/bin/i386-linux-uclibc-g++
    sys/bin/i386-linux-uclibc-gcc
    sys/bin/i386-linux-uclibc-gcc-4.1.2
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/README
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/float.h
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/iso646.h
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/stdarg.h
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/stdbool.h
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/stddef.h
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/varargs.h
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/mmintrin.h
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/mm3dnow.h
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/xmmintrin.h
    sys/lib/gcc/i386-linux-uclibc/4.1.2/install-tools/include/emmintrin.h
    tar: Decompression failed
    tar: Unable to read all data
    tar: Unable to read all data

    regards,
    Christos

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Anonymous said...

    oops...sorry file was corrupted during download - apologies.
    Christos

A:

The truth is that the gcc package is huge (~25MB) though there are just few libraries for deluge. I am going to address this issue so that I cut just a few relevant libraries out of the huge gcc package and provide a new package (~1MB) for it. Following text is by no means necessary for the successful installation, I am just going to describe the process of cutting out relevant dependencies for interested...

First we can list the installed packages. It is possible to use the filopack script for it:

box# cd /mnt/C
box# ./filopack --summary
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)

Filodej package summary:
Package name                          downloaded [#installed/#total]
--------------------------------------------------------------------
...
gcc-4.1.2-sjlj                            no           [ 546/ 549]
...

Now we can uninstall the package:

box# ./filopack.sh --remove gcc-4.1.2-sjlj
Configuration file .filopack/.config file found and used
Sure to remove gcc-4.1.2-sjlj locally at /mnt/C (y/n)?y

Let's make sure the package is actually removed:

box# ./filopack.sh --summary
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)

Filodej package summary:
Package name                          downloaded [#installed/#total]
--------------------------------------------------------------------
...
gcc-4.1.2-sjlj                            no           [   0/ 549]
...

Ok, now when we try to run deluge, we get the "missing library" error message:

box# deluged.sh
/usr/bin/python: can't load library 'libstdc++.so.6'

It seems we need at least the libstdc++.so.6 library. Let's find all the related files:

box# cat .filopack/gcc-4.1.2-sjlj.lst | grep  libstdc++
sys/lib/libstdc++.a
sys/lib/libstdc++.so
sys/lib/libstdc++.so.6.0.8
sys/lib/libstdc++.so.6
sys/lib/libstdc++.la

We actually do not need the static libraries (unless we have a plan to build something directly on the box), so we are interested just in so files:

box# cat .filopack/gcc-4.1.2-sjlj.lst | grep  libstdc++*.so
sys/lib/libstdc++.so
sys/lib/libstdc++.so.6.0.8
sys/lib/libstdc++.so.6

Let's make a new lst file representing the new package deluge-1.0.5-gcclibs we are going to create. This file will contain a list of all the relevant libraries:

box# cat .filopack/gcc-4.1.2-sjlj.lst | grep  libstdc++*.so > .filopack/deluge-1.0.5-gcclibs.lst

... and extract just those files from the gcc-4.1.2-sjlj package:

box# cat .filopack/gcc-4.1.2-sjlj.lst | grep libstdc++*.so | xargs tar -xjvf gcc-4.1.2-sjlj.tar.bz2
sys/lib/libstdc++.so
sys/lib/libstdc++.so.6.0.8
sys/lib/libstdc++.so.6

When we try to start deluge daemon again:

box# deluged.sh
/usr/bin/python: Can't resolve symbol '_Unwind_SjLj_Register'

... it seems there are still some missing libraries (or more likely some missing symbols in an older version of the same library). Let's look at the dependencies more closer:

box# LD_TRACE_LOADED_OBJECTS=1 deluged.sh
        /usr/lib/libssl.so.0.9.7 => /usr/lib/libssl.so.0.9.7 (0x40007000)
        /usr/lib/libboost_filesystem-gcc41-mt-1_35.so.1.35.0 => /usr/lib/libboost_filesystem-gcc41-mt-1_35.so.1.35.0 (0x40030000)
        libpython2.5.so.1.0 => /mnt/C/sys/lib//libpython2.5.so.1.0 (0x4003f000)
        libpthread.so.0 => /mnt/C/sys/lib//libpthread.so.0 (0x40166000)
        libdl.so.0 => /lib/libdl.so.0 (0x40178000)
        libutil.so.0 => /mnt/C/sys/lib//libutil.so.0 (0x4017b000)
        libm.so.0 => /lib/libm.so.0 (0x4017d000)
        libc.so.0 => /lib/libc.so.0 (0x4018b000)
        libcrypto.so.0.9.7 => /mnt/C/sys/lib//libcrypto.so.0.9.7 (0x4021f000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x402f4000)
        libboost_system-gcc41-mt-1_35.so.1.35.0 => /mnt/C/sys/lib//libboost_system-gcc41-mt-1_35.so.1.35.0 (0x402fc000)
        librt.so.0 => /mnt/C/sys/lib//librt.so.0 (0x40300000)
        libstdc++.so.6 => /mnt/C/sys/lib//libstdc++.so.6 (0x40302000)
        ld-uClibc.so.0 => /lib/ld-uClibc.so.0 (0x40000000)

Here is a list of libraries present in the gcc package:

dev# cat .filopack/gcc-4.1.2-sjlj.lst | grep "\.so"
sys/lib/libgcc_s.so
sys/lib/libgcc_s.so.1
sys/lib/libstdc++.so
sys/lib/libstdc++.so.6.0.8
sys/lib/libstdc++.so.6
sys/lib/libmudflap.so.0.0.0
sys/lib/libmudflap.so.0
sys/lib/libmudflap.so
sys/lib/libmudflapth.so.0.0.0
sys/lib/libmudflapth.so.0
sys/lib/libmudflapth.so
sys/lib/libssp.so.0.0.0
sys/lib/libssp.so.0
sys/lib/libssp.so

The libgcc_s looks like a good candidate. We can look for the missing symbol:

box# nm /lib/libgcc_s.so.1
nm: /lib/libgcc_s.so.1: no symbols

box# nm --dynamic /lib/libgcc_s.so | grep Unwind
00003ebc T _Unwind_Backtrace
00003e98 T _Unwind_DeleteException
00002800 T _Unwind_FindEnclosingFunction
00004e2c T _Unwind_Find_FDE
00003c50 T _Unwind_ForcedUnwind
000027dc T _Unwind_GetCFA
00002838 T _Unwind_GetDataRelBase
00003f60 T _Unwind_GetGR
00003f84 T _Unwind_GetIP
000027e8 T _Unwind_GetLanguageSpecificData
000027f4 T _Unwind_GetRegionStart
00002844 T _Unwind_GetTextRelBase
00003a08 T _Unwind_RaiseException
00003d00 T _Unwind_Resume
00003dc8 T _Unwind_Resume_or_Rethrow
00003f70 T _Unwind_SetGR
00003f90 T _Unwind_SetIP

The /lib/libgcc_s.so is an "old" version of the library being replaced by the new one contained in the gcc package. Apparently the symbol '_Unwind_SjLj_Register' is missing in the list, but there are similar symbols.
Let's list the libgcc related files:

box# cat .filopack/gcc-4.1.2-sjlj.lst | grep libgcc_s*.so
sys/lib/libgcc_s.so
sys/lib/libgcc_s.so.1

Now we can extract the files from the package tarball:

box# cat .filopack/gcc-4.1.2-sjlj.lst | grep libgcc_s*.so | xargs tar -xjvf gcc-4.1.2-sjlj.tar.bz2
sys/lib/libgcc_s.so
sys/lib/libgcc_s.so.1

Make sure that the symbol is actually there:

box# nm --dynamic /mnt/C/sys/lib/libgcc_s.so | grep Unwind
00005361 T _Unwind_Backtrace
00005345 T _Unwind_DeleteException
0000522e T _Unwind_FindEnclosingFunction
00004e4b T _Unwind_Find_FDE
000051d6 T _Unwind_GetCFA
00005235 T _Unwind_GetDataRelBase
000051c5 T _Unwind_GetGR
000051fb T _Unwind_GetIP
0000521a T _Unwind_GetLanguageSpecificData
00005227 T _Unwind_GetRegionStart
0000523c T _Unwind_GetTextRelBase
000051e7 T _Unwind_SetGR
00005209 T _Unwind_SetIP
000054a5 T _Unwind_SjLj_ForcedUnwind
000054ef T _Unwind_SjLj_RaiseException
0000511c T _Unwind_SjLj_Register
00005452 T _Unwind_SjLj_Resume
0000557d T _Unwind_SjLj_Resume_or_Rethrow
000051ba T _Unwind_SjLj_Unregister

Now we can append the files to the lst file.

box# cat .filopack/gcc-4.1.2-sjlj.lst | grep libgcc_s*.so >> .filopack/deluge-1.0.5-gcclibs.lst

And test the deluge again:

[root@storage C]# deluged.sh

... now the daemon starts without problems.
The last step is to create the deluge-1.0.5-gcclibs package.

box# cat .filopack/deluge-1.0.5-gcclibs.lst
sys/lib/libstdc++.so
sys/lib/libstdc++.so.6.0.8
sys/lib/libstdc++.so.6
sys/lib/libgcc_s.so
sys/lib/libgcc_s.so.1

We can use the --pack action of the filopack script with -R option in order to reuse the (already created) lst file:

box# ./filopack.sh -R --pack deluge-1.0.5-gcclibs
Configuration file .filopack/.config file found and used
tar: unrecognized option `--files-from=.filopack/deluge-1.0.5-gcclibs.lst'
BusyBox v1.90-JKR (2008.04.09-06:53 CET) multi-call binary

Usage: tar -[czjZxtvO] [-f TARFILE] [-C DIR] [FILE(s)] ...

Create, extract, or list files from a tar file.

Options:
        c               create
        x               extract
        t               list

Archive format selection:
        z               Filter the archive through gzip
        j               Filter the archive through bzip2
        Z               Filter the archive through compress

File selection:
        f               name of TARFILE or "-" for stdin
        O               extract to stdout
        C               change to directory DIR before operation
        v               verbosely list files processed

... oops, it seems that I have different versions of tar on my box and on my dev# system (Chroot-ed environment on top of the Colinux with Debian), there the --files-from works ok.
Let's make sure. First find the tar on the box:

box# which tar
/bin/tar
box# ls -l /bin/tar
lrwxrwxrwx 1 root root 7 2008-05-21 13:40 /bin/tar -> busybox

And now on the dev# system:

dev# tar --version
tar (GNU tar) 1.15.1

dev# which tar
/bin/tar

dev# ls -l /bin/tar
-rwxr-xr-x  1 root root 166880 Apr 16  2008 /bin/tar*

As a solution I have updated the filopack script to be compliant with the busybox tar version. I changed the following statement:

tar $PACK_CMD $PACKAGE.$EXT --files-from $CONFIG_DIR/$PACKAGE.lst 2>&1 | tee $CONFIG_DIR/$PACKAGE.err

to the following:

cat $CONFIG_DIR/$PACKAGE.lst | xargs tar $PACK_CMD $PACKAGE.$EXT 2>&1 | tee $CONFIG_DIR/$PACKAGE.err

.. the only thing I am not sure of is whether the number of command line argument is somewhat limited; such limitation could cause some problems for packages with many files.
Let's try it directly:

box# ./filopack.sh -R --pack deluge-1.0.5-gcclibs
Configuration file .filopack/.config file found and used
tar: Creating bzip2 compressed archives is not currently supported.

Equivalent command:
cat .filopack/deluge-1.0.5-gcclibs.lst | xargs tar cjvf deluge-1.0.5-gcclibs.tar.bz2

Bzip compression is not supported, so try the tgz:

box# ./filopack.sh -R -o tgz --pack deluge-1.0.5-gcclibs
Configuration file .filopack/.config file found and used
sys/lib/libgcc_s.so.1
sys/lib/libgcc_s.so
sys/lib/libstdc++.so.6
sys/lib/libstdc++.so.6.0.8
sys/lib/libstdc++.so

Equivalent command:
cat .filopack/deluge-1.0.5-gcclibs.lst | xargs tar czvf deluge-1.0.5-gcclibs.tgz

... this works. We have successfully created the package.
Note: Finally I created the bz2 version on my mirror system.

dev# ./filopack.sh -R --pack deluge-1.0.5-gcclibs
...

So, that's all for now. I have uploaded the package and now it is publicly available:

box# ./filopack.sh --packages
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Available packages:
...
deluge-1.0.5
deluge-1.0.5-deplibs
...

You can download it to your box:

box# ./filopack.sh --download deluge-1.0.5-gcclibs
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Downloading package deluge-1.0.5-gcclibs from http://filodej.ic.cz ...
connected!

Length: 113 [text/plain]
connected!

Length: 1,191,263 [application/x-tar]

And install it as follows:

box# ./filopack.sh --install deluge-1.0.5-gcclibs
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Sure to unpack deluge-1.0.5-gcclibs locally at /mnt/C (y/n)?y
sys/lib/libstdc++.so
sys/lib/libstdc++.so.6.0.8
sys/lib/libstdc++.so.6
sys/lib/libgcc_s.so
sys/lib/libgcc_s.so.1

Unresolved 'PyUnicodeUCS2_DecodeUTF8' symbol

Q:

$ deluged.sh
/usr/bin/python: Can't resolve symbol '_Unwind_SjLj_Register'
$
$
$ LD_PRELOAD="/usr/lib/libssl.so.0.9.7 /usr/lib/libboost_filesystem-gcc41-mt-1_35.so.1.35.0 /usr/lib/libgcc_s.so" deluged -d
[DEBUG ] 08:53:55 configmanager:44 ConfigManager started..
[INFO ] 08:53:55 daemon:44 Deluge daemon 1.0.5
[DEBUG ] 08:53:55 daemon:45 options: {'logfile': None, 'config': None, 'port': None, 'donot': True}
[DEBUG ] 08:53:55 daemon:46 args: []
/usr/bin/python: can't resolve symbol 'PyUnicodeUCS2_DecodeUTF8'
$

A:

Likely this symbol should be defined in python library. Let's look if it is the case:

box# nm /mnt/C/sys/lib/libpython2.5.so.1.0 | grep DecodeUTF8
00073f08 T PyUnicodeUCS4_DecodeUTF8
00073f30 T PyUnicodeUCS4_DecodeUTF8Stateful

So it seems there is a UCS4 vs. UCS2 mismatch on your system. When I googled PyUnicodeUCS4 PyUnicodeUCS2 I got the following explanation:

You are using a version of Python that uses a 4-byte representation for Unicode characters, but some C extension module you are importing was compiled using a Python that uses a 2-byte representation for Unicode characters (the default).

So it seems to be exactly the case on your system. Look at the python console just to make sure:

box# python
Python 2.5.2 (r252:60911, May  3 2008, 16:19:27)
[GCC 3.3.6] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print sys.maxunicode > 65535 and 'UCS4 build' or 'UCS2 build'
UCS4 build
>>>

Now we have to find out what is the python extension to be blamed, but unfortunately I cannot do it on my own since it seems there is no such problem on my box. At least I can try to help you a bit how to investigate that.

You can use strace (system calls and signals tracer) for that. Here is the a short explanation how I built it and here you can directly download the package, or possibly if you are using the filopack system you can simply write:

$ cd /mnt/C
$ ./filopack.sh --download strace-4.5.15
$ ./filopack.sh --install strace-4.5.15

... and (hopefully) you are prepared.

To replicate a scenario similar to yours, I can run the deluge daemon without the LD_PRELOAD prefix, in such case I am also missing some symbols:

box# /usr/bin/deluged
box# /usr/bin/python: can't resolve symbol '__cxa_pure_virtual'

So now I can try to trace all the system calls:

box# strace /usr/bin/deluged
...
stat64("/mnt/C/sys/root/.config/deluge", {st_mode=S_IFDIR|0700, st_size=4096, ...}) = 0
fork()                                  = 17025
_exit(0)                                = ?
Process 17020 detached
box# /usr/bin/python: can't resolve symbol '__cxa_pure_virtual'

It seems that the loader problem was in another instance of deluged process. Let's look at strace help to see what we can do about it:

box# strace -h
usage: strace [-dffhiqrtttTvVxx] [-a column] [-e expr] ... [-o file]
              [-p pid] ... [-s strsize] [-u username] [-E var=val] ...
              [command [arg ...]]
   or: strace -c [-e expr] ... [-O overhead] [-S sortby] [-E var=val] ...
              [command [arg ...]]
-c -- count time, calls, and errors for each syscall and report summary
-f -- follow forks, -ff -- with output into separate files
-F -- attempt to follow vforks, -h -- print help message
-i -- print instruction pointer at time of syscall
-q -- suppress messages about attaching, detaching, etc.
-r -- print relative timestamp, -t -- absolute timestamp, -tt -- with usecs
-T -- print time spent in each syscall, -V -- print version
-v -- verbose mode: print unabbreviated argv, stat, termio[s], etc. args
-x -- print non-ascii strings in hex, -xx -- print all strings in hex
-a column -- alignment COLUMN for printing syscall results (default 40)
-e expr -- a qualifying expression: option=[!]all or option=[!]val1[,val2]...
   options: trace, abbrev, verbose, raw, signal, read, or write
-o file -- send trace output to FILE instead of stderr
-O overhead -- set overhead for tracing syscalls to OVERHEAD usecs
-p pid -- trace process with process id PID, may be repeated
-s strsize -- limit length of print strings to STRSIZE chars (default 32)
-S sortby -- sort syscall counts by: time, calls, name, nothing (default time)
-u username -- run command as username handling setuid and/or setgid
-E var=val -- put var=val in the environment for command
-E var -- remove var from the environment for command

Now we can give it another try, this time with the -f option:

box# strace -f /usr/bin/deluged
...
...
close(7)                                = 0
munmap(0x40006000, 4096)                = 0
open("./libbz2.so.1.0", O_RDONLY)       = -1 ENOENT (No such file or directory)
open("/mnt/C/sys/lib//libbz2.so.1.0", O_RDONLY) = 7
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40006000
read(7, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0$\21\0\000"..., 4096) = 4096
old_mmap(NULL, 57344, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40e57000
old_mmap(0x40e57000, 52556, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 7, 0) = 0x40e57000
old_mmap(0x40e64000, 3712, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 7, 0xd000) = 0x40e64000
close(7)                                = 0
munmap(0x40006000, 4096)                = 0
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40006000
write(2, "", 0)                         = 0
write(2, "/usr/bin/python", 15/usr/bin/python)         = 15
write(2, ": can\'t resolve symbol \'", 24: can't resolve symbol ') = 24
write(2, "__cxa_pure_virtual", 18__cxa_pure_virtual)      = 18
write(2, "\'\n", 2'
)                     = 2
munmap(0x40006000, 4096)                = 0
_exit(1)                                = ?
Process 17039 detached

Most likely the libbz2.so module is requiring the unresolved __cxa_pure_virtual symbol.

Back to your problem, once we know the particular python extension requiring the 2 byte unicode version of the DecodeUTF8 method we can try to re-build or re-install it appropriately...

Part II.

Dear Filodej,

you are right! I have followed your instructions and indeed this is the problem. 
I hope that the console output after executing strace commandes would be found 
usefull for you.
Here is the complete output from executing strace, and here for the strace -f.

Thank you in advance, for helping me out :-) hope others will benefit as well.

Regards,
Christos

Good, so the relevent part of your trace is here:

$ strace -f /usr/bin/deluged
execve("/usr/bin/deluged", ["/usr/bin/deluged"], [/* 31 vars */]) = 0
...
ioctl(9, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbffeddc8) = -1 ENOTTY (Inappropriate ioctl for device)
fstat64(9, {st_mode=S_IFREG|0755, st_size=411371, ...}) = 0
open("/usr/lib/python2.5/site-packages/_xmlplus/parsers/pyexpat.so", O_RDONLY) = 10
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40006000
read(10, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0$7\0\000"..., 4096) = 4096
old_mmap(NULL, 180224, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4059c000
old_mmap(0x4059c000, 167720, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 10, 0) = 0x4059c000
old_mmap(0x405c5000, 9872, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 10, 0x29000) = 0x405c5000
close(10)                               = 0
munmap(0x40006000, 4096)                = 0
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40006000
write(2, "", 0)                         = 0
write(2, "/usr/bin/python", 15/usr/bin/python)         = 15
write(2, ": can\'t resolve symbol \'", 24: can't resolve symbol ') = 24
write(2, "PyUnicodeUCS2_DecodeUTF8", 24PyUnicodeUCS2_DecodeUTF8) = 24
write(2, "\'\n", 2'
)                     = 2
munmap(0x40006000, 4096)                = 0
_exit(1)                                = ?
Process 2748 detached

I wonder where did you get the pyexpat library since I did not build the PyXML as part of my python package. Let's look at the JoKeR's package (it seems as a good candidate):

box# wget http://mgb111.pradnik.net/addons/soft-lang/Python-25-PyXML-084.tar.gz
connected!

Length: 17,194,098 [application/x-gzip]

box# tar tzf Python-25-PyXML-084.tar.gz | grep pyexpat
./sys/lib/python2.5/site-packages/_xmlplus/parsers/pyexpat.so
./sys/lib/python2.5/site-packages/_xmlplus/sax/drivers/drv_pyexpat.py
./sys/lib/python2.5/site-packages/_xmlplus/sax/drivers/drv_pyexpat.pyc
./sys/lib/python2.5/site-packages/_xmlplus/sax/drivers2/drv_pyexpat.py
./sys/lib/python2.5/site-packages/_xmlplus/sax/drivers2/drv_pyexpat.pyc
./sys/lib/python2.5/test/output/test_pyexpat
./sys/lib/python2.5/test/test_pyexpat.py
./sys/lib/python2.5/test/test_pyexpat.pyc
./sys/lib/python2.5/test/test_pyexpat.pyo
./sys/lib/python2.5/lib-dynload/pyexpat.so

Bingo! You probably did not remove the JoKeR's installation and just extract my python package over it.
Since I have built python with different unicode settings that JoKeR (it is nothing like Microsoft's intended incompatibilities, I swear I had no idea about it at the time ;-) now you are experiencing the missing symbol(s).

Just to make sure I am right in my assumption, I try to replicate your problem on my box. Let's extract the _xmlplus directory from JoKeR's package:

box# tar xzvf Python-25-PyXML-084.tar.gz ./sys/lib/python2.5/site-packages/_xmlplus/*
...

Now I can try to do some python xml parsing and see what happens:

box# strace -o st.txt /usr/bin/python
Python 2.5.2 (r252:60911, May  3 2008, 16:19:27)
[GCC 3.3.6] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import xml.dom.minidom
>>> xml.dom.minidom.parseString( 'test')
/usr/bin/python: can't resolve symbol 'PyUnicodeUCS2_DecodeUTF8'

Great, now I am in the same trouble as you ;-)
Let's look at the trace log:

box# tail -n 20 st.txt
stat64("/usr/lib/python2.5/site-packages/_xmlplus/parsers/pyexpat", 0xbfff4a4c) = -1 ENOENT (No such file or directory)
open("/usr/lib/python2.5/site-packages/_xmlplus/parsers/pyexpat.so", O_RDONLY|O_LARGEFILE) = 5
ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfff49e8) = -1 ENOTTY (Inappropriate ioctl for device)
fstat64(5, {st_mode=S_IFREG|0755, st_size=411371, ...}) = 0
open("/usr/lib/python2.5/site-packages/_xmlplus/parsers/pyexpat.so", O_RDONLY) = 6
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40006000
read(6, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0$7\0\000"..., 4096) = 4096
old_mmap(NULL, 180224, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40310000
old_mmap(0x40310000, 167720, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED, 6, 0) = 0x40310000
old_mmap(0x40339000, 9872, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 6, 0x29000) = 0x40339000
close(6)                                = 0
munmap(0x40006000, 4096)                = 0
old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40006000
write(2, "", 0)                         = 0
write(2, "/usr/bin/python", 15)         = 15
write(2, ": can\'t resolve symbol \'", 24) = 24
write(2, "PyUnicodeUCS2_DecodeUTF8", 24) = 24
write(2, "\'\n", 2)                     = 2
munmap(0x40006000, 4096)                = 0
_exit(1)                                = ?

and try to launch the deluge daemon:

box# deluged.sh
box# /usr/bin/python: can't resolve symbol 'PyUnicodeUCS2_DecodeUTF8'

Ok, now I am pretty sure this is exactly your situation. Let's remove the _xmlplus package again, we cannot use it anyway, since it is built incompatibly with the rest of the python framework:

box# rm -d -r ./sys/lib/python2.5/site-packages/_xmlplus/

Now let's try the xml parsing and launch the deluge daemon again:

box# python
Python 2.5.2 (r252:60911, May  3 2008, 16:19:27)
[GCC 3.3.6] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import xml.dom.minidom
>>> xml.dom.minidom.parseString( '<a>test</a>')
<xml.dom.minidom.Document instance at 0x4027940c>
>>>

box# deluged.sh
box# 

Everything works again.

The removal of the _xmlplus pyhon package should help in your case as well, but note that there could be other incompatible files. The ultimate solution would be to remove both python packages (mine and JoKeR's) and install just mine again. In case you needed the PyXML package (which seems unlikely to me, since it does not work for you anyway) I could sometimes build one compatible with 4-byte unicode.

Unresolved 'g_assertion_message_expr' symbol

Q:

Hello again,

I have erased everything from my box in order to perform a clean installation based on your instructions. 
Unfortunately, this didn't go as it was expected.
Initially I installed Deluge and all of the dependencies, apart from setuptools. 
This was due to the fact that I couldn't run python cause there was a library .so-something missing 
- unfortunately I did not write down the name.
In order to fix this (and hopefully it was fixed) I downloaded & installed Joker's "minimal-config" & "libs-essential".

Now when I run:
LD_PRELOAD="/usr/lib/libssl.so.0.9.7 /usr/lib/libboost_filesystem-gcc41-mt-1_35.so.1.35.0 /usr/lib/libgcc_s.so" deluged -d

I am getting the following error:
/usr/bin/python: can't resolve symbol 'g_assertion_message_expr'
...
regards,
Christos

A:

First I googled a bit and found that the g_assertion_message_expr symbol should be present in the libglib-2.0.so library. It seemed little bit weird that you are experiencing such error, since this library is listed in my list of dependencies. It is clear that it needs a deeper analysis...

First let's make sure the libglib library is really present on the system:

box# ls -l /mnt/C/sys/lib/libglib*
-rwxr-xr-x 1 root root     878 2008-06-30 00:06 /mnt/C/sys/lib/libglib-2.0.la
lrwxrwxrwx 1 root root      23 2008-12-03 23:04 /mnt/C/sys/lib/libglib-2.0.so -> libglib-2.0.so.0.1702.0
lrwxrwxrwx 1 root root      23 2008-12-03 23:04 /mnt/C/sys/lib/libglib-2.0.so.0 -> libglib-2.0.so.0.1702.0
-rwxr-xr-x 1 root root 2631967 2008-04-17 12:34 /mnt/C/sys/lib/libglib-2.0.so.0.1400.0
rwxr-xr-x 1 root root 1131687 2008-06-30 00:06 /mnt/C/sys/lib/libglib-2.0.so.0.1702.0

... as can be seen there are two of them but just one of them is linked. This fact could be related to your problem.

We can learn what is present in my glib package:

box# ./filopack.sh --download glib-2.17.2                                 
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Downloading package glib-2.17.2 from http://filodej.ic.cz ...
connected!

Length: 21,692 [text/plain]
connected!

Length: 1,947,202 [application/x-tar]

box# tar tjvf glib-2.17.2.tar.bz2 | grep libglib
-rwxr-xr-x 0/0   1131687 2008-06-30 00:06:55 sys/lib/libglib-2.0.so.0.1702.0
lrwxrwxrwx 0/0         0 2008-06-30 00:06:55 sys/lib/libglib-2.0.so.0 -> libglib-2.0.so.0.1702.0
lrwxrwxrwx 0/0         0 2008-06-30 00:06:55 sys/lib/libglib-2.0.so -> libglib-2.0.so.0.1702.0
-rwxr-xr-x 0/0       878 2008-06-30 00:06:55 sys/lib/libglib-2.0.la

So we see that the 2.17.2 version comes from my package. What about the other one?

You write that you have downloaded & installed Joker's "minimal-config" & "libs-essential", so let's look at that:

box# wget http://mgb111.pradnik.net/addons/libs-essential-20080502.tar.bz2
connected!

Length: 13,454,268 [application/x-bzip2]

box# tar tjvf libs-essential-20080502.tar.bz2 | grep libglib
-rwxr-xr-x 0/0       834 2008-04-17 12:34:55 ./sys/lib/libglib-2.0.la
lrwxrwxrwx 0/0         0 2008-05-10 20:41:07 ./sys/lib/libglib-2.0.so -> libglib-2.0.so.0.1400.0
lrwxrwxrwx 0/0         0 2008-05-10 20:41:07 ./sys/lib/libglib-2.0.so.0 -> libglib-2.0.so.0.1400.0
-rwxr-xr-x 0/0   2631967 2008-04-17 12:34:55 ./sys/lib/libglib-2.0.so.0.1400.0

... ok, so another one (2.14.0 version) is from JoKeR's package.

Let's dig a little bit deeper, what about symbols?

box# nm --dynamic /mnt/C/sys/lib/libglib-2.0.so.0.1702.0 | grep g_assertion_message_expr
0006409b T g_assertion_message_expr
box# nm --dynamic /mnt/C/sys/lib/libglib-2.0.so.0.1400.0 | grep g_assertion_message_expr

so it seems that the g_assertion_message_expr symbol was introduced right between 14.0 and 17.2 versions (or possibly the lack of it in the older version is caused by different build configuration). In any case the problem was caused by the fact that you extracted JoKeR's essential-libs package AFTER my glib package and thus effectively downgraded the glib from 2.17.2 version down to 2.14.0.

Sorry for that complication, I built my system on top of JoKeR's minimal-config and libs-essential and I mention it in the post about the box configuration; but it is sure my fault that I did not mention it explicitly among the deluge dependencies. I am doing it now to prevent further confusion.

That's all for now. You can now redirect the symlinks (libglib-2.0.so and libglib-2.0.so.0) by hand or better re-install my glib package, supposedly it should fix your problem. As can be seen such subtle details like installation order can sometimes make a big difference :-)

Missing bash shell

Q:

...
Anyway, I then ran the "./filopack.sh --download deluge-1.0.5" line, and I got this: "-ash: ./filopack.sh: not found." Now, that's not good, right?

Any idea what it is that I'm doing wrong please?
...

A:

In a bash session with bash shell installed on the system the filopack script is running fine:

box# ./filopack.sh
    
Stupid packaging system (version 0.3.7)

Usage:
 
  filopack.sh [OPTIONS] [] []

Basic usage:
  ...

When we look at the first line of the script there is bash shell specified as an interpreter:

box# head -n1 ./filopack.sh
#!/bin/bash

Let's look where is the bash shell installed:

box# which bash
/usr/bin/bash

The /bin directory is on a read-only file system, but /bin/bash is just a symlink to /usr/bin/bash:

box# ls -l /bin/bash 
lrwxrwxrwx 1 root root 19 2008-05-21 13:40 /bin/bash -> /mnt/C/sys/bin/bash

Let's try to "delete" the bash shell for a moment:

box# mv /usr/bin/bash{,.del}

Now when we try to run the filopack script:

 
box# ./filopack.sh           
-bash: ./filopack.sh: /bin/bash: bad interpreter: No such file or directory

The bash shell (we are still running the bash session even if the bash executable is removed) is sufficiently verbose in its error messages so the cause of the problem is obvious. Let's try the same in ash shell:

box# ash
box$ cd /mnt/C/
box$ ./filopack.sh
ash: ./filopack.sh: not found

Ok now we have replicated your problem. You have to install the bash shell in order to run the filopack script.