Showing posts with label WMU-6500FS box with JoKeR's fw. Show all posts
Showing posts with label WMU-6500FS box with JoKeR's fw. Show all posts

2009/11/19

WMU-6500FS - Print server installation

Introduction

Since I have one LPT printer and several computers at home I considered buing a print server for a long time. What reliably discouraged me however was the price - for example HP print server price starts at around 140$.
Unfortunately I haven't bought a router with USB ports like Asus WL-500W, but rather Linksys WRT54GL and so the printer sharing as described at DD-WRT wiki was not applicable in my case.
Then I noticed that there is a p910nd print daemon package available at the JoKer's site.
There was not much information about the installation on the Web and so I was pretty unsure but I decided to give it a try.

Note: Before you follow the steps I describe here, please, make sure your printer is compliant with the JetDirect technology.

Hardware installation

My HP LaserJet 1100 printer is not USB but LPT, so there is some USB to LPT reduction necessary.
I also realized that there is not a standard printer cable but rather a Mini Centronics cable used for the printer connection.

So it seemed that besides something like the USB to Parallel port adapter (14$) also another reduction like Centronics to Mini Centronics adapter (11$) was necessary (there is also a combo for 24$ so you can spare a buck ;).

Since I already had a standard printer cable the best choice for me was the ST Lab U-370 Dongle. For only 11$ does the job very well.
So that's it regarding to hardware installation.

Software installation

We download and install the p910nd print daemon (I use 0.92 version, I had no luck with the latest 0.93 version, I got /var/lock/subsys/p9100d file not found error).
Edit: now the problem with 0.93 is solved, see the discussion below
box# cd /mnt/C
box# wget http://mgb111.pradnik.net/addons/servers-print/p910nd-0.92-081017.tar
box# tar xvf p910nd-0.92-081017.tar
./sys/
./sys/etc/
./sys/man/
./sys/man/man8/
./sys/man/man8/p910nd.8
./sys/sbin/
./sys/sbin/p910nd
Now we are ready to plug the printer to USB port (suppose you use the USB1 port specifically) and run the daemon:
box# /mnt/C/sys/sbin/p910nd -b -f /dev/usblp0
We can see that the process is now up and running. The fact that it is named p9100d (instead of original p910nd) means that it is listening at port 9100.
box# ps | grep p910
  271 root        284 S   /mnt/C/sys/sbin/p9100d -b -f /dev/usblp0
 1863 root        216 S   grep p910
We can make sure it is actually listening at the port; when we run the network statistics to list all the ports all the processes are are listening for, newly we can see the jetdirect port.
box# netstat -l
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
...
tcp        0      0 *:jetdirect             *:*                     LISTEN      
...
We can also look at the system log (with help of the dmesg command). There we can find something similar to the following:
box# dmesg
...
hub.c: new USB device 00:0a.0-2, assigned address 2
printer.c: usblp0: USB Bidirectional printer dev 2 if 0 alt 1 proto 2 vid 0x067B pid 0x2305
...
When we retrieve the description string of the USB device, we get that there is a IEEE-1284 (parallel communication) controller made by the Prolific Technology Inc..
box# cat /proc/printer/usblp0
Prolific Technology Inc. IEEE-1284 Controller
0x18
Note that in case you have a USB printer you should probably expect somewhat more meaningful string like
Hewlett-Packard HP LaserJet 1005 series
Now we are ready to try some printing:
box# echo "printer test" > /dev/usblp0 
...
Initially I got the following error message:
bash: /dev/usblp0: Device or resource busy
... then I realized that there is a mini-lpd process running holding the device. mini-lpd is a small, non-queueing LPD implementation. I do not know why there is mini-lpd active on the box, I have not tried to make it work, I just killed it and used p910nd daemon instead:
box# killall mini-lpd
After that everything started to work.
To make the changes permanent, I have added add the following lines:
### printserv
killall mini-lpd
/mnt/C/sys/sbin/p910nd -b -f /dev/usblp0 &
to the /mnt/C/sys/etc/rc-local file.

Ok, that's it on the server side, let's look at the clients...

Client installation

Now it is time to install the print clients:
Windows XP
I presume you already have the printer installed locally, and so the only thing you need is to create a new port and set it up as a Standard TCP/IP Port: Now you select the box IP or network name: As a device type you choose the Hewlett Pacjkard Jet Direct: Here is a result page(SNMP not supported, RAW protocol, port 9100): Here we see the newly created port: And finally we are ready to print test page:
Ubuntu 9.10
In menu System/Administration/Printing we choose to create a new printer and in device selection choose Network Printer and AppSocket/HP JetDirect.In Host and Port fields we fill the box IP or network name and as port we use 9100: The next step is the Printer model and driver selection: Now we can specify the printer name and description: And finally we are ready to print test page:

Links



Read more...

2009/04/13

WMU-6500FS - Deluge 1.1.5

I just finished a build of the deluge 1.1.5. It is bit outdated (the 1.1.6 version is out now) but I had to solve some problems along the way which taken more time than expected. Once it was finished I did not find a morale to step one bugfix version further.

Build result

[binary] [file list]

Prerequisites

The same as for the previous version plus:
[patched python socket module] [tar 1.22] [bzip2 lib]

Uninstall

If you have the deluge-1.1.0 installed, you have to clean it up first (while preserve all the dependencies).
Stop the deluge daemon if it is running. You can do it via console:
box# deluge --ui=null
>>> halt
>>> quit
Thanks!

or forcibly:
box# killall deluged
Now uninstall the previous version:
dev# cd /mnt/C/
dev# ./filopack.sh --remove deluge-1.1.0
Configuration file .filopack/.config file found and used
Sure to remove deluge-1.1.0 locally at /mnt/C (y/n)?y
...
If you are not using the filopack packaging system, you can remove the previous version as follows:
box# cd /mnt/C/
box# wget http://filodej.ic.cz/filopack/.filopack/deluge-1.1.0.lst
box# xargs rm -f < deluge-1.1.0.lst

Update the system

Before we start installing the new version, we have to update the tar archiver. The one which is part of the busybox has an ugly bug corrupting file names in long paths.
box# ./filopack.sh --download bzip2-1.0.5
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Downloading package bzip2-1.0.5 from http://filodej.ic.cz ...
connected!

Length: 191 [text/plain]
connected!

Length: 40,244 [application/x-tar]

box# ./filopack.sh --install bzip2-1.0.5
...
box# ./filopack.sh --download tar-1.22
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Downloading package tar-1.22 from http://filodej.ic.cz ...
connected!

Length: 1,523 [text/plain]
connected!

Length: 625,455 [application/x-tar]

box# ./filopack.sh --install tar-1.22
...

For details about the bug see this section.
Also it may be necessary to download a patched version of python socket library, you can test your system as follows:
box# python -c 'import socket; print socket.gethostbyaddr("80.68.88.204")[2];'
Segmentation fault
... if you encounter the segfault, it is better to download and install the patched python socket library:
box# wget http://filodej.ic.cz/filopack/_socket.so
connected!

Length: 116,767 [text/plain]

box# mv sys/lib/python2.5/lib-dynload/_socket.so{,.backup}
box# mv _socket.so sys/lib/python2.5/lib-dynload/
Now the problem should be fixed:
box# python -c 'import socket; print socket.gethostbyaddr("80.68.88.204")[2];'
['80.68.88.204']
For details about this issue see this section.

Install

After we updated the system we are ready to install the new version:
box# ./filopack.sh --download deluge-1.1.5
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Downloading package deluge-1.1.5 from http://filodej.ic.cz ...
connected!

Length: 89,430 [text/plain]
connected!

Length: 16,084,717 [application/x-tar]

box# ./filopack.sh --install deluge-1.1.5
Sure to unpack deluge-1.1.5 locally at /mnt/C (y/n)? y
...

Run daemon

Now we are ready to try the daemon, still it is necessary to use the LD_PRELOAD prefix or deluged.sh script:
box# deluged.sh
That's all. Following text just describes details related to the issues I solved. Nothing for ordinary users ;-)

Busybox tar bug

When I run the deluge client (console version) some commands was not properly interpreted:
box# deluge
>>> info
 * unknown command: info
>>> help
 * unknown command: help
I found out that any command is implemented in a separate python file:
box# cd sys/lib/python2.5/site-packages/deluge-1.1.5-py2.5-linux-i686.egg
box# ls deluge/ui/console/commands/
add.py          config.pyc   debug.pyc       help.pyc        __init__.pyc    quit.pyc    rm.pyc
add.pyc         connect.py   halt.py0000755  info.py0000755  pause.py        resume.py
add.pyc0000644  connect.pyc  halt.pyc        info.pyc        pause.pyc       resume.pyc
config.py       debug.py     help.py0000755  __init__.py     quit.py0000755  rm.py
... it seems there are some ill-named files in the command directory, and so the console does not know the commands at all.
Let's find all such corrupted files:
box# find . -name *0000*
./deluge/core/preferencesmanager.pyc0000644
./deluge/ui/console/commands/quit.py0000755
./deluge/ui/console/commands/help.py0000755
./deluge/ui/console/commands/halt.py0000755
./deluge/ui/console/commands/info.py0000755
./deluge/ui/console/commands/add.pyc0000644
./deluge/ui/gtkui/torrentdetails.pyc0000644
./deluge/ui/gtkui/queuedtorrents.pyc0000644
./deluge/ui/gtkui/filtertreeview.pyc0000644
./deluge/ui/webui/page_decorators.py0000755
./deluge/ui/webui/torrent_options.py0000755
./deluge/ui/webui/lib/egg_handler.py0000755
./deluge/ui/webui/lib/egg_render.pyc0000644
./deluge/ui/webui/lib/webpy022/db.py0000755
./deluge/plugins/Label-0.1-py2.5.egg0000644
./deluge/plugins/webuipluginbase.pyc0000644
./deluge/data/pixmaps/checking16.png0000644
./deluge/data/pixmaps/inactive16.png0000644
Let's look also in the deluge tar archive:
box# tar tjvf deluge-1.1.5.tar.bz2 | grep 0000
-rw-r--r-- 0/0     21100 2009-04-01 12:22:09 sys/lib/python2.5/site-packages/deluge-1.1.5-py2.5-linux-i686.egg/deluge/core/preferencesmanager.pyc0000644
-rwxr-xr-x 0/0      1079 2009-04-01 12:22:09 sys/lib/python2.5/site-packages/deluge-1.1.5-py2.5-linux-i686.egg/deluge/ui/console/commands/quit.py0000755
-rwxr-xr-x 0/0      2299 2009-04-01 12:22:09 sys/lib/python2.5/site-packages/deluge-1.1.5-py2.5-linux-i686.egg/deluge/ui/console/commands/help.py0000755
-rwxr-xr-x 0/0      1125 2009-04-01 12:22:09 sys/lib/python2.5/site-packages/deluge-1.1.5-py2.5-linux-i686.egg/deluge/ui/console/commands/halt.py0000755
-rwxr-xr-x 0/0      5296 2009-04-01 12:22:09 sys/lib/python2.5/site-packages/deluge-1.1.5-py2.5-linux-i686.egg/deluge/ui/console/commands/info.py0000755
-rw-r--r-- 0/0      2036 2009-04-01 12:22:09 sys/lib/python2.5/site-packages/deluge-1.1.5-py2.5-linux-i686.egg/deluge/ui/console/commands/add.pyc0000644
-rw-r--r-- 0/0     13688 2009-04-01 12:22:10 sys/lib/python2.5/site-packages/deluge-1.1.5-py2.5-linux-i686.egg/deluge/ui/gtkui/torrentdetails.pyc0000644
-rw-r--r-- 0/0      7346 2009-04-01 12:22:10 sys/lib/python2.5/site-packages/deluge-1.1.5-py2.5-linux-i686.egg/deluge/ui/gtkui/queuedtorrents.pyc0000644
-rw-r--r-- 0/0     13073 2009-04-01 12:22:10 sys/lib/python2.5/site-packages/deluge-1.1.5-py2.5-linux-i686.egg/deluge/ui/gtkui/filtertreeview.pyc0000644
-rwxr-xr-x 0/0      5062 2009-04-01 12:22:10 sys/lib/python2.5/site-packages/deluge-1.1.5-py2.5-linux-i686.egg/deluge/ui/webui/page_decorators.py0000755
-rwxr-xr-x 0/0      3233 2009-04-01 12:22:10 sys/lib/python2.5/site-packages/deluge-1.1.5-py2.5-linux-i686.egg/deluge/ui/webui/torrent_options.py0000755
-rwxr-xr-x 0/0      1553 2009-04-01 12:22:10 sys/lib/python2.5/site-packages/deluge-1.1.5-py2.5-linux-i686.egg/deluge/ui/webui/lib/egg_handler.py0000755
-rw-r--r-- 0/0      1522 2009-04-01 12:22:10 sys/lib/python2.5/site-packages/deluge-1.1.5-py2.5-linux-i686.egg/deluge/ui/webui/lib/egg_render.pyc0000644
-rwxr-xr-x 0/0     20480 2009-04-01 12:22:10 sys/lib/python2.5/site-packages/deluge-1.1.5-py2.5-linux-i686.egg/deluge/ui/webui/lib/webpy022/db.py0000755
-rw-r--r-- 0/0     38041 2009-04-01 12:22:11 sys/lib/python2.5/site-packages/deluge-1.1.5-py2.5-linux-i686.egg/deluge/plugins/Label-0.1-py2.5.egg0000644
-rw-r--r-- 0/0      2982 2009-04-01 12:22:11 sys/lib/python2.5/site-packages/deluge-1.1.5-py2.5-linux-i686.egg/deluge/plugins/webuipluginbase.pyc0000644
-rw-r--r-- 0/0       699 2009-04-01 12:22:11 sys/lib/python2.5/site-packages/deluge-1.1.5-py2.5-linux-i686.egg/deluge/data/pixmaps/checking16.png0000644
-rw-r--r-- 0/0       595 2009-04-01 12:22:11 sys/lib/python2.5/site-packages/deluge-1.1.5-py2.5-linux-i686.egg/deluge/data/pixmaps/inactive16.png0000644
At a first glance it seem that the archive is corrupted but when I tried the same operation on my mirror system (on the PC) no corrupted file appeared in the archive.
The difference was that while on the mirror system I have the GNU tar 1.20 installed, on the box there is a busybox version containing tar utility:
box# which tar
/bin/tar
box# ls -l /bin/tar
lrwxrwxrwx 1 root root 7 2008-05-21 13:40 /bin/tar -> busybox
I decided to build the newest GNU tar version (1.22) and install it to the box. A new post containing the build procedure will follow. After the installation the problem disappeared.

Socket related crash

I am not sure whether it was new to this version, but after the installation from time to time I have experienced a weird crash of the deluge daemon. Also the Windows client did not respond for long time when was connected to the daemon running on the box. After some experimenting with the deluge log I decided to debug the daemon to find out what is going on.
I was running the gdbserver on the box:
box# LD_PRELOAD="/usr/lib/libssl.so.0.9.7 /usr/lib/libboost_filesystem-gcc41-mt-1_35.so.1.35.0" gdbserver colinux:2345 `which python` `which deluged` -d
Process /usr/bin/python created; pid = 31831
Listening on port 2345
Remote debugging from host 192.168.1.102
...
Then I was ready to connect to the gdbserver from my mirror system:
deb# gdb `which python`
GNU gdb 6.3
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-linux-uclibc"...Using host libthread_db library "/lib/libthread_db.so.1".

(gdb) target remote storage:2345
Remote debugging using storage:2345
0x40000c90 in ?? ()
(gdb) cont
Continuing.
...
After some time when I was connecting and disconnectiong the windows client to the daemon the problem appeared:
[New Thread 32801]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 32801]
0x400a0fb0 in PyString_FromString (str=0xc7c3815b <Address 0xc7c3815b out of bounds>) at Objects/stringobject.c:108
108             size = strlen(str);
(gdb)
It was at the following stack:
(gdb) where
#0  0x400a0fb0 in PyString_FromString (str=0xc7c3815b <Address 0xc7c3815b out of bounds>) at Objects/stringobject.c:108
#1  0x407e8154 in gethost_common (h=0xbb9fdae8, addr=0xbb9fdb08, alen=128, af=2) at /usr/local/src/Python-2.5.2/Modules/socketmodule.c:3048
#2  0x407e5825 in socket_gethostbyaddr (self=0x0, args=0x40f2f48c) at /usr/local/src/Python-2.5.2/Modules/socketmodule.c:3273
#3  0x40094bd2 in PyCFunction_Call (func=0x405ac70c, arg=0x40f2f48c, kw=0x0) at Objects/methodobject.c:108
#4  0x400dcf1a in PyEval_EvalFrameEx (f=0x81f8624, throwflag=0) at Python/ceval.c:3573
#5  0x400de1e6 in PyEval_EvalCodeEx (co=0x405b94e8, globals=0x405b624c, locals=0x0, args=0x820022c, argcount=1, kws=0x8200230, kwcount=0, defs=0x405be518, defcount=1,
    closure=0x0) at Python/ceval.c:2836
#6  0x400dd6d0 in PyEval_EvalFrameEx (f=0x82000e4, throwflag=0) at Python/ceval.c:3669
#7  0x400dda59 in PyEval_EvalFrameEx (f=0x81e77d4, throwflag=0) at Python/ceval.c:3659
#8  0x400de1e6 in PyEval_EvalCodeEx (co=0x405c13c8, globals=0x405b602c, locals=0x0, args=0x406200b0, argcount=4, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0)
    at Python/ceval.c:2836
...
a crash in PyString_FromString seems to be just a consequence, let's look up a bit:
(gdb) up
#1  0x407e8154 in gethost_common (h=0xbb9fdae8, addr=0xbb9fdb08, alen=128, af=2) at /usr/local/src/Python-2.5.2/Modules/socketmodule.c:3048
3048                            tmp = PyString_FromString(*pch);
(gdb) list
3043
3044            /* SF #1511317: h_aliases can be NULL */
3045            if (h->h_aliases) {
3046                    for (pch = h->h_aliases; *pch != NULL; pch++) {
3047                            int status;
3048                            tmp = PyString_FromString(*pch);
3049                            if (tmp == NULL)
3050                                    goto err;
3051
3052                            status = PyList_Append(name_list, tmp);
(gdb) print h
$1 = (struct hostent *) 0xbb9fdae8
(gdb) print *h
$2 = {h_name = 0xbb9f9b00 "filodej.doma", h_aliases = 0x40172891, h_addrtype = 2, h_length = 4, h_addr_list = 0xbb9f9aec}
(gdb) print h->h_aliases
$3 = (char **) 0x40172891
(gdb) print *h->h_aliases
$4 = 0xc7c3815b <Address 0xc7c3815b out of bounds>
... It seems that a host entry should contain an array of strings - aliases - but this array is apparently corrupted - it causes a crash of the C string -> Python string conversion routine. Let's look who is responsible for filling the array:
(gdb) up
#2  0x407e5825 in socket_gethostbyaddr (self=0x0, args=0x40f2f48c) at /usr/local/src/Python-2.5.2/Modules/socketmodule.c:3273
3273            ret = gethost_common(h, (struct sockaddr *)&addr, sizeof(addr), af);
(gdb) list
3268            PyThread_acquire_lock(netdb_lock, 1);
3269    #endif
3270            h = gethostbyaddr(ap, al, af);
3271    #endif /* HAVE_GETHOSTBYNAME_R */
3272            Py_END_ALLOW_THREADS
3273            ret = gethost_common(h, (struct sockaddr *)&addr, sizeof(addr), af);
3274    #ifdef USE_GETHOSTBYNAME_LOCK
3275            PyThread_release_lock(netdb_lock);
3276    #endif
3277            return ret;
The gethostbyaddr seems as a good candidate. At first I wondered whether the function is thread safe (see the following text about uClibc thread safety), but after further debugging I have found out that the apparently thread safe variant gethostbyaddr_r is called in my case, so there should be not threading issue there.
(gdb) cont
Continuing.
[New Thread 1024]
Breakpoint 2 at 0x407e574c: file /usr/local/src/Python-2.5.2/Modules/socketmodule.c, li
Pending breakpoint "socket_gethostbyaddr" resolved
[New Thread 19476]
[Switching to Thread 19476]

Breakpoint 2, socket_gethostbyaddr (self=0x0, args=0x40f27ccc) at /usr/local/src/Python
3229            if (!PyArg_ParseTuple(args, "s:gethostbyaddr", &ip_num))
...
(gdb) next
3252            Py_BEGIN_ALLOW_THREADS
(gdb) next
3255            result = gethostbyaddr_r(ap, al, af,
(gdb) next
3272            Py_END_ALLOW_THREADS
(gdb) print buf
$12 = "À¨\001eè\232?½", '\0' <repeats 16 times>, "filodej.doma\000.in-addr.arpa", '\0' <repeats 16333 times>, "@"
(gdb) next
3273            ret = gethost_common(h, (struct sockaddr *)&addr, sizeof(addr), af);
(gdb) print *h
$14 = {h_name = 0xbd3f9b00 "filodej.doma", h_aliases = 0x40172891, h_addrtype = 2, h_length = 4, h_addr_list = 0xbd3f9aec}
(gdb) print h->h_aliases
$15 = (char **) 0x40172891
(gdb) print *h->h_aliases
$16 = 0xc7c3815b <Address 0xc7c3815b out of bounds>
... after some googling I found some hints in the following forum.
Actually it seem there were two related bugs: the first in gethostbyname_r seems to be fixed now, steps to simply reproduce the problem:
box# python -c 'import socket; print socket.gethostbyname_ex("wh0rd.org")[2];'
['80.68.88.204']
... it was ok in my case, it seems to be fixed in 0.9.27 version.
The second bug is in gethostbyaddr_r function. The step to reproduce is following:
box# python -c 'import socket; print socket.gethostbyaddr("80.68.88.204")[2];'
Segmentation fault
... it crashes on my box. Given you have the Python installed, you can test your configuration the same way.
For the solution I decided not to patch or upgrade the uClibc but rather make a workaround in the python socket library. Zeroing the structure prior to the gethostbyaddr_r call should be enough. Let's modify the python Modules/socketmodule.c file. In the PySocket_gethostbyaddr functon there is a hp_allocated structure we are going to reset to zeroes. Just before the gethostbyaddr_r call we add the following code:
#if   defined(HAVE_GETHOSTBYNAME_R_6_ARG)
        memset((void *) &hp_allocated, '\0', sizeof(hp_allocated));
        result = gethostbyaddr_r(ap, al, af,
                &hp_allocated, buf, buf_len,
                &h, &errnop);
... and rebuild and re-pack the python package:
deb# cd /usr/local/src/Python-2.5.2
deb# make
case $MAKEFLAGS in \
*-s*) LD_LIBRARY_PATH=/usr/local/src/Python-2.5.2::/mnt/C/sys/lib:/mnt/C/sys/X11/lib CC='gcc -pthread' LDSHARED='gcc -pthread -shared' OPT='-DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes' ./python -E ./setup.py -q build;; \
*) LD_LIBRARY_PATH=/usr/local/src/Python-2.5.2::/mnt/C/sys/lib:/mnt/C/sys/X11/lib CC='gcc -pthread' LDSHARED='gcc -pthread -shared' OPT='-DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes' ./python -E ./setup.py build;; \
esac
running build
running build_ext
db.h: found (4, 3) in /mnt/C/sys/include
db lib: using (4, 3) db-4.3
/mnt/C/sys/include/sqlite3.h: version 3.5.8
INFO: Can't locate Tcl/Tk libs and/or headers
building '_socket' extension
gcc -pthread -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -I/usr/local/src/Python-2.5.2/./Include -I/mnt/C/sys/include -I. -IInclude -I./Include -I/usr/local/include -I/usr/local/src/Python-2.5.2/Include -I/usr/local/src/Python-2.5.2 -c /usr/local/src/Python-2.5.2/Modules/socketmodule.c -o build/temp.linux-i686-2.5/usr/local/src/Python-2.5.2/Modules/socketmodule.o
gcc -pthread -shared -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes build/temp.linux-i686-2.5/usr/local/src/Python-2.5.2/Modules/socketmodule.o -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib -L/usr/local/lib -L. -lpython2.5 -o build/lib.linux-i686-2.5/_socket.so
building 'nis' extension
gcc -pthread -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -I/usr/local/src/Python-2.5.2/./Include -I/mnt/C/sys/include -I. -IInclude -I./Include -I/usr/local/include -I/usr/local/src/Python-2.5.2/Include -I/usr/local/src/Python-2.5.2 -c /usr/local/src/Python-2.5.2/Modules/nismodule.c -o build/temp.linux-i686-2.5/usr/local/src/Python-2.5.2/Modules/nismodule.o
gcc -pthread -shared -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes build/temp.linux-i686-2.5/usr/local/src/Python-2.5.2/Modules/nismodule.o -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib -L/usr/local/lib -L. -lnsl -lpython2.5 -o build/lib.linux-i686-2.5/nis.so
*** WARNING: renaming "nis" since importing it failed: dynamic module does not define init function (initnis)
running build_scripts
box# make install
...
deb# cd /mnt/C/
deb# ./filopack.sh -R --pack python-2.5.2
...

Build process

The build process was identical to 1.1.0 version, I just had to create .pth link file in order to be able to remove the duplicate directories:
 
dev# cd /mnt/C/sys/lib/python2.5/site-packages
dev# echo "./deluge-1.1.5-py2.5-linux-i686.egg" > deluge.pth

dev# cd /mnt/C/
dev# ./filopack.sh --pack deluge-1.1.5
Remove all the "site-packages/deluge/" sub-paths (they seem to be redundant) and re-pack the package once more.

Read more...

2009/01/18

WMU-6500FS - Deluge 1.1.0

The new deluge version is here!

Build result

[binary] [file list]
If you don't have the previous version (deluge-1.0.5) installed, follow the previous deluge post, install all the prerequisites and dependencies and come back here for the rest. Note that the deluge-1.0.5-gcclibs package is valid even for deluge-1.1.0, so don't get confused with its misleading name.
On the other hand if you have the deluge-1.0.5 installed, you have to clean it up first (while preserve all the already installed dependencies).

Stop the deluge daemon if it is running. You can do it via console:
box# deluge --ui=null
>>> halt
>>> quit
Thanks!

or forcibly:
box# killall deluged
Now uninstall the previous version:
dev# cd /mnt/C/
dev# ./filopack.sh --remove deluge-1.0.5
Configuration file .filopack/.config file found and used
Sure to remove deluge-1.0.5 locally at /mnt/C (y/n)?y
...
If you are not using the filopack packaging system, you can remove the previous version as follows:
box# cd /mnt/C/
box# wget http://filodej.ic.cz/filopack/.filopack/deluge-1.0.5.lst
box# xargs rm -f < deluge-1.0.5.lst
It is likely that deluge developers preserved the compatibility of configuration and the following step is not necessary, I just to be sure delete the configuration as well:
box# mv ~/.config/deluge{,.del}
Now we are ready to install the new version:
box# ./filopack.sh --install deluge-1.1.0
Sure to unpack deluge-1.1.0 locally at /mnt/C (y/n)? y
...
Now we are ready to try the daemon, still it is necessary to use the LD_PRELOAD prefix or deluged.sh script - now updated in order to support parameters (like -d for example) - see this post on mascat for details - if we are lucky, everything runs smoothly:
box# deluged.sh

Default GUI

It is possible to set a default GUI (if you prefer other than predefined GTK GUI):
box# deluge --set-default-ui=console
The default UI has been changed to console
It is nice that now it is not necessary to modify the configuration by hand (like it is described here).

Remote access

The CLI ui has been renamed from null to console. Notice that it is necessary to use LD_PRELOAD prefix just for daemon and GTK GUI, there is no need to use it for the console UI).
We can use it for enabling the remote access (we have to restart the daemon in order to make the change active):
box# deluge --ui=console
>>> config --set allow_remote True
>>> halt
>>> quit
Thanks!

box# deluged.sh

Authentication

Another new feature is the user authentication. For details see [Authentication] and [ThinClient settings]. Without adding a username and password to ~/.config/deluge/auth configuration file you won't be able to remotely connect to the daemon nor event see whether the daemon is running (which is a nice).

Edit: since the ~/.config/deluge/auth file initially does not contain trailing newline the instructions did not work, now it is updated appropriately
box# echo -n -e "\n<username>:<password>" >> ~/.config/deluge/auth
Be sure to append to the file, in case you rewrote the file you won't have been able to connect to it locally, since it initially contains a localclient record. If you accidentally rewrite the file, you can just delete it, restart the daemon and a ui (e.g. the console) and the default file containing localclient authentication info is created for you.

Web GUI

If you want to use the Web GUI (and new Ajax UI seems pretty good to me), now it is not necessary to run the web client on the same machine as daemon. Via the Web GUI you can connect to any daemon you want (not just localhost). Not sure whether I personally utilize it, but it seems to me like a nice feature.

That's all folks.
(The following text is just a (boring) build protocol, totally unnecessary for ordinary users ;-)

Build sequence

It was about the same like in previous version so just in short:

Remove the previous version:
dev# cd /mnt/C/
dev# ./filopack.sh --remove deluge-1.0.5
Configuration file .filopack/.config file found and used
Sure to remove deluge-1.0.5 locally at /mnt/C (y/n)?y
...
Init the timestamp for the new one:
dev# ./filopack.sh --init  deluge-1.1.0
Configuration file .filopack/.config file found and used
Timestamp written to file .filopack/deluge-1.1.0.ts
Download and extract the source:
dev# cd /usr/local/src
dev# wget http://download.deluge-torrent.org/source/1.1.0/deluge-1.1.0.tar.bz2
...
04:37:59 (55.28 KB/s) - `deluge-1.1.0.tar.bz2' saved [2196924/2196924]

dev# tar xjvf deluge-1.1.0.tar.bz2
dev# cd deluge-1.1.0
Setup the include and library paths:
dev# export CFLAGS=-I/mnt/C/sys/include/boost-1_35
dev# export LDFLAGS=-L/mnt/C/sys/lib
Add the missing define:
nano libtorrent/include/libtorrent/socket.hpp

      #ifndef IPV6_V6ONLY
      #define IPV6_V6ONLY 26
      #endif

I did not find the libtorrent/src/memdebug.cpp, so undefining the content of this file is not necessary anymore.

Build the code and install the binaries:
dev# python setup.py build
...
dev# python setup.py install --prefix=/mnt/C/sys/
...
Try to run the daemon:
dev# deluged
dev# /usr/bin/python: can't resolve symbol '__cxa_pure_virtual'
... issue wit h unresolved symbol is still the same. As a workaround the LD_PRELOAD prefix can be used. Let;s create scripts for it:
dev# echo LD_PRELOAD=\"/usr/lib/libssl.so.0.9.7 /usr/lib/libboost_filesystem-gcc41-mt-1_35.so.1.35.0\" deluged \"\$@\" > /mnt/C/sys/bin/deluged.sh
dev# chmod +x /mnt/C/sys/bin/deluged.sh 
dev# echo LD_PRELOAD=\"/usr/lib/libssl.so.0.9.7 /usr/lib/libboost_filesystem-gcc41-mt-1_35.so.1.35.0\" deluge \"\$@\" > /mnt/C/sys/bin/deluge.sh
dev# chmod +x /mnt/C/sys/bin/deluge.sh 
Now the daemon runs ok:
dev# deluged.sh
Let's try the GTK client:
dev# deluge --version
1.1.0
dev# deluge
...
  File "/mnt/C/sys/lib/python2.5/site-packages/deluge-1.1.0-py2.5-linux-i686.egg/deluge/ui/gtkui/common.py", line 

45, in get_logo
    size, size)
gobject.GError: Unrecognized image file format
ChangeAs a fix we can change the extension:
dev# sed -i 's/deluge.svg/deluge.png/g' /mnt/C/sys/lib/python2.5/site-packages/deluge/ui/gtkui/common.py
dev# deluge
Touch related files with older timestamps (in order to include them to the package):
dev# find /mnt/C/sys -path "*deluge*" -type f -exec touch {} \;
Then we are ready to create the package:
dev# cd /mnt/C/sys
dev# ./filopack.sh --pack deluge-1.1.0
...
I noticed that there was duplicity after the installation which made the package twice as big as was necessary: It seems to me that whole directory tree /mnt/C/sys/lib/python2.5/site-packages/deluge-1.1.0-py2.5-linux-i686.egg/deluge was redundant - directory /mnt/C/sys/lib/python2.5/site-packages/deluge had a similar content (including the huge libtorrent.so). I just dropped the former one and everything seems ok and package has 16MB (compared to 31MB).

Read more...

2008/12/22

WMU-6500FS - dropbear-0.52

Introduction

It seems to me that dropbear ssh daemon from JoKeR's package is configured so it does not support public key authentication. It is possible that I am doing something wrong but I was trying hard to make it work and did not succeed.
Finally I decided to build dropbear on my own, since the publickey authentication feature is essential for me (I am using the box for hosting of git repositories and I really do not want to type my password every time I commit a code).

Build result

[binary] [file list]

Installation

box# cd /mnt/C
box# ./filopack.sh --download dropbear-0.52
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Downloading package dropbear-0.52 from http://filodej.ic.cz ...
connected!

Length: 502 [text/plain]
connected!

Length: 177,869 [application/x-tar]
box# ./filopack.sh --install dropbear-0.52
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Sure to unpack dropbear-0.52 locally at /mnt/C (y/n)?y
sys/bin/dbclient
sys/bin/dropbearmulti
sys/bin/dropbearkey
sys/bin/dropbearconvert
sys/bin/scp
sys/sbin/dropbear
box# killall dropbear
box# dropbear
In this post I am going to describe how I built the dropbear package and diagnosed and solved (or worked around) problems which arose during the whole process.

JoKeR's dropbear

Let's try co connect to the box:
If ssh daemon (dropbear in our case) is not running on the box, the error message will likely look as follows:
deb# ssh root@storage
ssh: connect to host storage port 22: Connection refused
If everything is ok, it should look similar to following:
deb# ssh root@storage
root@storage's password: < type a password > 

  [    FW 4.00b4.C009-M[NG]    ]
  [ http://mgb111.pradnik.net/ ]

box#
Now we are trying to get rid of typing the password. We use the RSA public authentication for it.

Publickey authentication

If we do not have a RSA key generated yet, we can generate it as follows:
deb# cd ~
deb# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/filodej/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/filodej/.ssh/id_rsa.
Your public key has been saved in /home/filodej/.ssh/id_rsa.pub.
The key fingerprint is:
89:...:b4... filodej@debian
Deploy the public key to the box:
deb# scp .ssh/id_rsa.pub root@storage:/mnt/C/sys/root/.ssh/
root@storage's password:
id_rsa.pub                                                              100%  393     0.4KB/s   00:00
Now, on the box, append the public key to the list of authorized keys:
box# cd /mnt/C/sys/root/.ssh/
box# cat id_rsa.pub >> authorized_keys
From now on, the ssh client should not ask for anything (more precisely, it should not ask for password, it should ask for publickey passphrase if we did not leave it empty - for someone it can look similar as password, but it is completely different thing).

Let's try:
deb# ssh root@storage
root@storage's password:

Simple diagnostics

Ooops, it seems we did not succeed. Let's do some diagnostics (we can use either of "-v", "-vv", "-vvv" verbose modes, one "v" is sufficiend here):
dev# ssh -v root@storage
OpenSSH_4.3p2 Debian-9, OpenSSL 0.9.8c 05 Sep 2006
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to storage [192.168.1.104] port 22.
debug1: Connection established.
debug1: identity file /home/filodej/.ssh/identity type -1
debug1: identity file /home/filodej/.ssh/id_rsa type 1
debug1: identity file /home/filodej/.ssh/id_dsa type -1
debug1: Remote protocol version 2.0, remote software version dropbear_0.50
debug1: no match: dropbear_0.50
...
debug1: Host 'storage' is known and matches the RSA host key.
debug1: Found key in /home/filodej/.ssh/known_hosts:4
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: password
debug1: Next authentication method: password
root@storage's password:
Let's try to connect to another machine and compare the trace:
deb# ssh -v root@notasw
OpenSSH_4.3p2 Debian-9, OpenSSL 0.9.8c 05 Sep 2006
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to notasw [192.168.1.103] port 22.
debug1: Connection established.
debug1: identity file /home/filodej/.ssh/identity type -1
debug1: identity file /home/filodej/.ssh/id_rsa type 1
debug1: identity file /home/filodej/.ssh/id_dsa type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_4.7p1 Debian-8ubuntu1.2
debug1: match: OpenSSH_4.7p1 Debian-8ubuntu1.2 pat OpenSSH*
...
debug1: Host 'notasw' is known and matches the RSA host key.
debug1: Found key in /home/filodej/.ssh/known_hosts:8
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Trying private key: /home/filodej/.ssh/identity
debug1: Offering public key: /home/filodej/.ssh/id_rsa
debug1: Authentications that can continue: publickey,password
debug1: Trying private key: /home/filodej/.ssh/id_dsa
debug1: Next authentication method: password
root@notasw's password:
We can diff the logs to easily see the differences:
deb# ssh -v root@storage 2> storage.txt
root@storage's password:
<Ctrl-D>
deb# ssh -v root@notasw 2> notas.txt
root@notasw's password:
<Ctrl-D>
deb# diff storage.txt notas.txt
4c4
< debug1: Connecting to storage [192.168.1.104] port 22.
---
> debug1: Connecting to notasw [192.168.1.103] port 22.
9,10c9,10
< debug1: Remote protocol version 2.0, remote software version dropbear_0.50
< debug1: no match: dropbear_0.50
---
> debug1: Remote protocol version 2.0, remote software version OpenSSH_4.7p1 Debian-8ubuntu1.2
> debug1: match: OpenSSH_4.7p1 Debian-8ubuntu1.2 pat OpenSSH*
23,26c23,28
...
< debug1: Host 'storage' is known and matches the RSA host key.
< debug1: Found key in /home/filodej/.ssh/known_hosts:4
---
...
> debug1: Host 'notasw' is known and matches the RSA host key.
> debug1: Found key in /home/filodej/.ssh/known_hosts:8
33c35,40
< debug1: Authentications that can continue: password
---
> debug1: Authentications that can continue: publickey,password
> debug1: Next authentication method: publickey
> debug1: Trying private key: /home/filodej/.ssh/identity
> debug1: Offering public key: /home/filodej/.ssh/id_rsa
> debug1: Authentications that can continue: publickey,password
> debug1: Trying private key: /home/filodej/.ssh/id_dsa
It seems that the JoKeR's daemon is not configured for publickey authentication at all. Ok, let's build our own.

Build process

First we have to download, extract and configure the build:
dev# cd /usr/local/src/
dev# wget http://matt.ucc.asn.au/dropbear/dropbear-0.52.tar.

bz2
dev# tar xjvf dropbear-0.52.tar.bz2 
dev# cd dropbear-0.52
dev# ./configure --prefix /mnt/C/sys
...
configure: Now edit options.h to choose features.
As the message suggests, now we can set up some additional build options:
dev# nano options.h

change:
  #define RSA_PRIV_FILENAME "/etc/dropbear/dropbear_rsa_host_key"
to:
  #define RSA_PRIV_FILENAME "/mnt/C/sys/etc/dropbear_rsa_host_key"

and possibly comment out:
  #define DROPBEAR_DSS
or change:
  #define DSS_PRIV_FILENAME "/etc/dropbear/dropbear_dss_host_key"
to:
  #define DSS_PRIV_FILENAME "/mnt/C/sys/etc/dropbear_dss_host_key"

leave following unchanged:
  #define ENABLE_SVR_PASSWORD_AUTH
  /* PAM requires ./configure --enable-pam */
  /*#define ENABLE_SVR_PAM_AUTH*/
  #define ENABLE_SVR_PUBKEY_AUTH

  /* Wether to ake public key options in authorized_keys file into account */
  #ifdef ENABLE_SVR_PUBKEY_AUTH
  #define ENABLE_SVR_PUBKEY_OPTIONS
  #endif

  #define ENABLE_CLI_PASSWORD_AUTH
  #define ENABLE_CLI_PUBKEY_AUTH
  #define ENABLE_CLI_INTERACT_AUTH
... and make:
dev# make PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp"
...
It compiled successfully, but I have realized that I prefer to link the dependencies statically:
dev# make PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp" clean
dev# make STATIC=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp"
...
/mnt/C/sys/lib/gcc/i386-linux-uclibc/4.1.2/../../../../i386-linux-uclibc/bin/ld: cannot find -lz
collect2: ld returned 1 exit status
... the error means that linker is unable to find a static version of zlib library.
When we investigate that we see taht there is just a dynamic version of the library on the system:
dev# ls /lib/libz*
/lib/libz.so.1  /lib/libz.so.1.2.2
dev# ls /mnt/C/sys/lib/libz*
/mnt/C/sys/lib/libz*: No such file or directory
We could pick the sources and build a static version of zlib library, but for now I decided to re-configure the dropbear build and disable the zlib support:
dev# make STATIC=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp" clean
dev# ./configure --prefix /mnt/C/sys --disable-zlib STATIC=1 
dev# make STATIC=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp"
... now it compiled without problems, we can try to install it and upload to the box:
dev# make STATIC=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp" install
...
dev# scp /mnt/C/sys/sbin/dropbear root@storage:/tmp/
Now we can start the brand new version of dropbear daemon on the box:
box# killall dropbear
box# /tmp/dropbear
And try to log in over ssh client (from my debian system):
deb# ssh -vvv root@storage -i ~/.ssh/id_rsa.pub
...
debug2: key: /home/filodej/.ssh/id_rsa.pub (0xb7f7b480)
debug2: key:  (0xb7f7c8e0)
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Trying private key: /home/filodej/.ssh/identity
debug1: Offering public key: /home/filodej/.ssh/id_rsa
debug1: Authentications that can continue: publickey,password
debug1: Trying private key: /home/filodej/.ssh/id_dsa
debug1: Next authentication method: password
root@storage's password:
debug1: Authentications that can continue: publickey,password
Permission denied, please try again.
root@storage's password:
debug1: Authentications that can continue: publickey,password
Permission denied, please try again.
root@storage's password:
It seems we have done some progress here, but still are not able to authenticate via publickey. And even if we are using the correct password, permission is denied.

List of valid shells

Let's restart the daemon, but now not forking to the background and logging to the stderr rather that syslog:
box# dropbear -E -F
[4456] Nov 30 23:59:30 Not backgrounding
[4462] Nov 30 23:59:56 Child connection from 192.168.1.103:49013
[4462] Dec 01 00:00:08 user 'root' has invalid shell, rejected
Ok, there is something wrong with the shell settings, let's investigate that:
box# cat /var/config/passwd
root: ... :0:0:root:/mnt/C/sys/root:/mnt/C/sys/bin/bash
...
box# ls /mnt/C/sys/bin/bash
/mnt/C/sys/bin/bash
User root has /mnt/C/sys/bin/bash in his settings, the file exists and is at correct place. Let's look at the list of valid shells:
box# cat /etc/shells
/bin/sh
/sbin/nologin
/bin/nologin
/bin/ash
/sbin/ash
/bin/false
/sbin/false
/mnt/C/sys/bash
/bin/bash
/usr/bin/bash
The path /mnt/C/sys/bash is wrong since the bash executable is in /mnt/C/sys/bin directory, but the /usr/bin/bash path should be ok, since the /usr/bin is just a symlink to /mnt/C/sys/bin:
box# ls -l /usr/bin
lrwxrwxrwx    1 root     root           14 May 21  2008 /usr/bin -> /mnt/C/sys/bin
We have to look at the problem in more detail. Since there is no verbose mode available in dropbear by default, we can enable it in code (I got the inspiration from this post):
dev# nano debug.h
    uncomment:
        #define DEBUG_TRACE
Now when we rebuilt and re-deployed the dropbear, we can run in in verbose mode:
box# dropbear -v -E -F 
...
TRACE (4848): shell is /mnt/C/sys/bin/bash
TRACE (4848): test shell is '/bin/sh'
TRACE (4848): test shell is '/sbin/nologin'
TRACE (4848): test shell is '/bin/nologin'
TRACE (4848): test shell is '/bin/ash'
TRACE (4848): test shell is '/sbin/ash'
TRACE (4848): test shell is '/bin/false'
TRACE (4848): test shell is '/sbin/false'
TRACE (4848): test shell is '/mnt/C/sys/bash'
TRACE (4848): test shell is '/bin/bash'
TRACE (4848): test shell is '/usr/bin/bash'
As can be seen in the svr-auth.c source code:
00273       while ((listshell = getusershell()) != NULL) {
00274             TRACE(("test shell is '%s'", listshell))
00275             if (strcmp(listshell, usershell) == 0) {
00276                   /* have a match */
00277                   goto goodshell;
00278             }
00279       }
... the shell path is compared as plain strings and so /usr/bin/bash and /mnt/C/sys/bin/bash are two different things.

Since the /etc/shells file is placed on read-only file system we have to make a change in /var/config/passwd file:
box# sed -i 's|/mnt/C/sys/bin/|/usr/bin/|g' /var/config/passwd
Since it is overwritten on every boot, we have to make the correction in our rc.cfg file, to make it permanent:
nano /mnt/C/sys/etc/rc.d/rc.cfg
   add the following line:
    sed -i 's|/mnt/C/sys/bin/|/usr/bin/|g' /var/config/passwd 
Now when we try to connect again, we can see that the problem is solved:
[394] Dec 01 01:32:25 pubkey auth succeeded for 'root' with key md5 

3a:...:05 from 192.168.1.103:44521

Read only file system

There is another problem:
[394] Dec 01 01:32:25 chown(/dev/ttyp2, 0, 0) failed: Read-only file system
[394] Dec 01 01:32:26 exit after auth (root): chmod(/dev/ttyp2, 0622) failed: Read-only file system
The problem seems to be in the file sshpty.c in function pty_setowner but I am not sure what exactly is wrong. We can try to debug it remotely in order to find out what is going on. One option in debug.h can be handy - we can disable the forking the sub-processes - then there is just one dropbear process to attach.
/* To debug with GDB it is easier to run with no forking of child processes.
   You will need to pass "-F" as well. */
/* #define DEBUG_NOFORK */
Let's rebuild it with debug info:
dev# nano Makefile
...
CFLAGS=-I. -I$(srcdir) -I$(srcdir)/libtomcrypt/src/headers/ $(CPPFLAGS) -I/mnt/C/sys/include 

-I/mnt/C/sys/X11/include -g
...
Rebuild and redeploy the dropbear binary.

Now we can run the dropbear process on the box:
box# dropbear -E -F
box# [4498] Dec 21 23:50:01 Running in background
...
On another console connected to the box we start the gdbserver:
box# gdbserver colinux:2345 --attach 4498
Attached; pid = 4498
Listening on port 2345
...
Now on the mirror system we can start the gdb:
dev# gdb dropbear
GNU gdb 6.3
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-linux-uclibc"...Using host libthread_db library "/lib/libthread_db.so.1".

(gdb) target remote 192.168.1.104:2345
Remote debugging using 192.168.1.104:2345
0x0806d7ad in fast_s_mp_sqr (a=0xbfffe348, b=0xbfffe348) at bn_fast_s_mp_sqr.c:73
73               _W += ((mp_word)*tmpx++)*((mp_word)*tmpy--);
(gdb) break pty_setowner
Breakpoint 1 at 0x805bfcc: file sshpty.c, line 364.
(gdb) cont
Continuing.
Now we are ready to connect over ssh:
dev# ssh root@storage
root@storage's password:
... and debugger stops on the breakpoint:
Breakpoint 1, pty_setowner (pw=0x80dd480, tty_name=0x80e1578 "/dev/ttyp3") at sshpty.c:364
364             grp = getgrnam("tty");
(gdb) step
365             if (grp) {
(gdb) step
369                     gid = pw->pw_gid;
(gdb) step
370                     mode = S_IRUSR | S_IWUSR | S_IWGRP | S_IWOTH;
(gdb) step
378             if (stat(tty_name, &st)) {
(gdb) step
382             dropbear_log(LOG_ERR, "stat: %u", (unsigned int)st.st_uid );
(gdb) print st
$1 = {st_dev = 7936, __pad1 = 45352, __st_ino = 363, st_mode = 8630, st_nlink = 1, st_uid = 0, st_gid = 5,
  st_rdev = 771, __pad2 = 0, st_size = 0, st_blksize = 4096, st_blocks = 0, st_atime = 1206970720,
  st_atimensec = 134760895, st_mtime = 1206970720, st_mtimensec = 134691188, st_ctime = 1206970720,
  st_ctimensec = 135124096, st_ino = 363}
(gdb) print tty_name
$2 = 0x80e1578 "/dev/ttyp3"
(gdb) next
383             if (st.st_uid != pw->pw_uid || st.st_gid != gid) {
(gdb) print *pw
$3 = {pw_name = 0x80dd380 "root", pw_passwd = 0x80dd385 "...", pw_uid = 0, pw_gid = 0, pw_gecos = 

0x80dd3a4 "root", pw_dir = 0x80dd3a9 "/mnt/C/sys/root", pw_shell = 0x80dd3b9 "/usr/bin/bash"}
...
(gdb) step
401 if ((st.st_mode & (S_IRWXU|S_IRWXG|S_IRWXO)) != mode) {
(gdb) print /o st.st_mode
$4 = 020666
(gdb) step
402                     if (chmod(tty_name, mode) < 0) {
(gdb) step
403                             if (errno == EROFS &&
(gdb) step
409                                     dropbear_exit("chmod(%.100s, 0%o) failed: %.100s",
Ok, we have analyzed the situation. The pty_setowner function is trying to initialize the device file representing the terminal we are trying to connect to. Device file is on read only file system and so the attribute changes do not succeed.

If changing the tty owner does not succeed, the code at (1) activates more relaxed mode in case the file system is read only and tty is owned by root or the uids match. In our case the relaxed mode is activated and the server just writes the warning chown(/dev/ttyp2, 0, 0) failed: Read-only file system and continues.

Then there is code changing tty read mode. This code tries to prevent unauthorized users from reading tty content. In order to get just a warning, the reading by group and reading by others must be prohibited. In our case the st.st_mode is 020666 (read/write by everyone) and so we get the error chmod(/dev/ttyp2, 0622) failed: Read-only file system.

 /*
  * Change owner and mode of the tty as required.
  * Warn but continue if filesystem is read-only and the uids match/
  * tty is owned by root.
  */
 if (stat(tty_name, &st)) {
  dropbear_exit("pty_setowner: stat(%.101s) failed: %.100s",
    tty_name, strerror(errno));
 }

 if (st.st_uid != pw->pw_uid || st.st_gid != gid) {
  if (chown(tty_name, pw->pw_uid, gid) < 0) {
   if (errno == EROFS &&
       (st.st_uid == pw->pw_uid || st.st_uid == 0)) {
(1)    dropbear_log(LOG_ERR,
     "chown(%.100s, %u, %u) failed: %.100s",
      tty_name, (unsigned int)pw->pw_uid, (unsigned int)gid,
      strerror(errno));
   } else {
    dropbear_exit("chown(%.100s, %u, %u) failed: %.100s",
        tty_name, (unsigned int)pw->pw_uid, (unsigned int)gid,
        strerror(errno));
   }
  }
 }

 if ((st.st_mode & (S_IRWXU|S_IRWXG|S_IRWXO)) != mode) {

  if (chmod(tty_name, mode) < 0) {
   if (errno == EROFS &&
(2)       (st.st_mode & (S_IRGRP | S_IROTH)) == 0) {
    dropbear_log(LOG_ERR,
     "chmod(%.100s, 0%o) failed: %.100s",
     tty_name, mode, strerror(errno));
   } else {
    dropbear_exit("chmod(%.100s, 0%o) failed: %.100s",
        tty_name, mode, strerror(errno));
   }
  }
 }
Some discussions (for example here and here) recommended to configure build in order to disable openpty:
dev# make STATIC=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp" clean
dev# ./configure --prefix /mnt/C/sys --disable-zlib --disable-openpty STATIC=1 
dev# make STATIC=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp"
dev# make STATIC=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp" install
But it did not help in this case.

Nowadays I do not know a better workaround in order to make it work than change the dropbear_exit to dropbear_log. I have found a discussion about this on Web and the following rationale seems reasonable:
Ah. If it's a single user system, the easiest solution might just be to disable the chown() and chmod() call - it's not like there're any other users who might be sniffing :)
(sshpty.c is where you'll find those chown/chmods).
... but be careful, you were warned ;-)

Minor issues

There were also two another problems (just warnings). The first was wtmp problem:
[2398] Dec 01 12:13:04 wtmp_write: problem writing /var/log/wtmp: No such file or directory
It was solved by adding the --disable-wtmp to the configure command line.

The second was logging problem:
dev# ssh root@storage
root@storage's password:
[2719] Dec 01 13:25:53 lastlog_perform_login: Couldn't stat /var/log/lastlog: No such file or directory
[2719] Dec 01 13:25:53 lastlog_openseek: /var/log/lastlog is not a file or directory!

  [    FW 4.00b4.C009-M[NG]    ]
  [ http://mgb111.pradnik.net/ ]
We are not alone on this (see this post or OpenWrt ticket, also this archive served as a source of information).

It was solved by adding the --disable-lastlog to the configure command line.

Final build

As a result we have the following configure command line:
dev# ./configure --prefix /mnt/C/sys --disable-zlib --disable-openpty --disable-wtmp --disable-lastlog
Now we can modify the Makefile in order to generate code optimized for size:
dev# nano Makefile
...
CFLAGS=-I. -I$(srcdir) -I$(srcdir)/libtomcrypt/src/headers/ $(CPPFLAGS) -I/mnt/C/sys/include -I/mnt/C/sys/X11/include -Os
If we want to create multi-application instead of individual applications, we can use MULTI variable on the make command line:
dev# make STATIC=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp" clean
dev# make STATIC=1 MULTI=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp"
We can strip the sources to spare some space:
dev# make STATIC=1 MULTI=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp" strip
And are ready to install and create the package:
dev# make STATIC=1 MULTI=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp" install
dev# cd /mnt/C
dev# ./filopack.sh --pack dropbear-0.52

Test

Start the dropbear server (logging to strerr, not forking to background):
box# dropbear -E -F
[902] Dec 22 02:14:49 Not backgrounding
This is output when I was connecting from a machine without a registered public key:
[906] Dec 22 02:14:52 Child connection from 192.168.1.102:44241
[906] Dec 22 02:15:02 password auth succeeded for 'root' from 192.168.1.102:44241
[906] Dec 22 02:15:02 chown(/dev/ttyp3, 0, 0) failed: Read-only file system
[906] Dec 22 02:15:02 !!! Filodej is warning you. Be sure you are the only user !!!
[906] Dec 22 02:15:02 chmod(/dev/ttyp3, 0622) failed: Read-only file system
[906] Dec 22 02:15:47 chown /dev/ttyp3 0 0 failed: Read-only file system
[906] Dec 22 02:15:47 chmod /dev/ttyp3 0666 failed: Read-only file system
[906] Dec 22 02:15:47 exit after auth (root): Exited normally
This is output when I was connecting from a machine with registered public key:
[913] Dec 22 02:15:59 Child connection from 192.168.1.102:44242
[913] Dec 22 02:16:06 pubkey auth succeeded for 'root' with key md5 f1:...:3b from 192.168.1.102:44242
[913] Dec 22 02:16:06 chown(/dev/ttyp3, 0, 0) failed: Read-only file system
[913] Dec 22 02:16:06 !!! Filodej is warning you. Be sure you are the only user !!!
[913] Dec 22 02:16:06 chmod(/dev/ttyp3, 0622) failed: Read-only file system
[913] Dec 22 02:16:21 chown /dev/ttyp3 0 0 failed: Read-only file system
[913] Dec 22 02:16:21 chmod /dev/ttyp3 0666 failed: Read-only file system
[913] Dec 22 02:16:21 exit after auth (root): Exited normally

Read more...

2008/12/20

WMU-6500FS - xterm 237

GUI dependencies: [X.Org]

Build result

[binary] [file list]

Installation

box# cd /mnt/C
box# ./filopack.sh --download xterm-237
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Downloading package xterm-237 from http://filodej.ic.cz ...
connected!

Length: 502 [text/plain]
connected!

Length: 177,869 [application/x-tar]
box# ./filopack.sh --install xterm-237
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Sure to unpack xterm-237 locally at /mnt/C (y/n)?y
sys/X11/share/pixmaps/xterm-color_32x32.xpm
sys/X11/share/pixmaps/xterm-color_48x48.xpm
sys/X11/share/pixmaps/xterm_32x32.xpm
sys/X11/share/pixmaps/xterm_48x48.xpm
sys/X11/man/man1/xterm.1
sys/X11/man/man1/resize.1
sys/X11/man/man1/uxterm.1
sys/X11/man/man1/koi8rxterm.1
sys/X11/lib/X11/app-defaults/XTerm
sys/X11/lib/X11/app-defaults/XTerm-color
sys/X11/lib/X11/app-defaults/UXTerm
sys/X11/lib/X11/app-defaults/KOI8RXTerm
sys/X11/bin/xterm
sys/X11/bin/resize
sys/X11/bin/uxterm
sys/X11/bin/koi8rxterm
box# xterm

Make sure you have the environment variables set properly (you can make it permanent e.g. by editing the /mnt/C/sys/etc/profile; for details see the X Configuration post):

PATH variable can possibly contain the /mnt/C/sys/X11/bin/ path:
box# export PATH=$PATH:/mnt/C/sys/X11/bin/
LD_LIBRARY_PATH must (among others) contain the /mnt/C/sys/X11/lib path:
box# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/mnt/C/sys/X11/lib/
DISPLAY environment variable properly set to a display of a machine running xserver:
box# export DISPLAY=<machine>:<display-number>
... or in case you are using putty, X11 forwarding enabled:

Build procedure

First we have to initialize the package timestamp.
dev# cd /mnt/C
dev# ./filopack.sh --init xterm-237 
Configuration file .filopack/.config file found and used
Timestamp written to file .filopack/xterm-237.ts
Now we can download and extract the source.
dev# cd /usr/local/src
dev# wget ftp://invisible-island.net/xterm/xterm-237.tgz
...
Length: 860,424 (unauthoritative)

100%[==========================================================>] 860,424       14.59K/s    ETA 00:00

14:18:14 (17.14 KB/s) - `xterm-237.tgz' saved [860424]

dev# tar xzvf xterm-237.tgz 
...
dev# cd xterm-237
We configure the build in order to have access to XOrg header files and libraries:
dev#  ./configure --x-includes=/mnt/C/sys/X11/include/ --x-libraries=/mnt/C/sys/X11/lib/ --prefix=/mnt/C/sys/X11
...
... and make and install the source:
dev# make
...
dev# make install
...
We can test the installation:
dev# /mnt/C/sys/X11/bin/xterm
Warning: This program is an suid-root program or is being run by the root user.
The full text of the error or warning message cannot be safely formatted
in this environment. You may get a more descriptive message by running the
program as a non-root user or by removing the suid bit on the executable.
/mnt/C/sys/X11/bin/xterm Xt error: Can't open display: %s
/mnt/C/sys/X11/bin/xterm:  DISPLAY is not set
Oops, I forgot to set the display variable on the mirror machine. Let's do it now (note that the IP address 192.168.2.1 is specific to my network configuration):
dev# export DISPLAY=192.168.2.1:0
dev# /mnt/C/sys/X11/bin/xterm
xterm: Error 18, errno 2: No such file or directory
Reason: spawn: open() failed on ptsname
It seems there is still some error. Let's investigate it with strace:
dev# strace -f /mnt/C/sys/X11/bin/xterm 2> strace.txt

<Ctrl-C>

dev# tail -n 120 strace.txt
[pid  8119] close(8 <unfinished ...>
[pid  8120] <... setsid resumed> )      = 8120
[pid  8119] <... close resumed> )       = 0
[pid  8120] setpgid(0, 0 <unfinished ...>
[pid  8119] close(5 <unfinished ...>
[pid  8120] <... setpgid resumed> )     = -1 EPERM (Operation not permitted)
[pid  8119] <... close resumed> )       = 0
[pid  8120] ioctl(4, TIOCSPTLCK <unfinished ...>
[pid  8119] read(7,  <unfinished ...>
[pid  8120] <... ioctl resumed> , [0])  = 0
[pid  8120] ioctl(4, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
[pid  8120] ioctl(4, TIOCGPTN, [2])     = 0
[pid  8120] open("/dev/pts/2", O_RDWR)  = -1 ENOENT (No such file or directory)
[pid  8120] write(2, "xterm", 5xterm)        = 5
[pid  8120] write(2, ": Error ", 8: Error )     = 8
[pid  8120] write(2, "18", 218)           = 2
[pid  8120] write(2, ", errno ", 8, errno )     = 8
[pid  8120] write(2, "2", 12)            = 1
[pid  8120] write(2, ": ", 2: )           = 2
[pid  8120] write(2, "No such file or directory", 25No such file or directory) = 25
[pid  8120] write(2, "\n", 1
)           = 1
[pid  8120] write(2, "Reason: ", 8Reason: )     = 8
[pid  8120] write(2, "spawn: open() failed on ptsname", 31spawn: open() failed on ptsname) = 31
[pid  8120] write(2, "\n", 1
)           = 1
[pid  8120] ioctl(4, TCFLSH, 0x1)       = 0
[pid  8120] chown("/dev/pts/2", 0, 0)   = -1 ENOENT (No such file or directory)
[pid  8120] chmod("/dev/pts/2", 0666)   = -1 ENOENT (No such file or directory)
[pid  8120] close(4)                    = 0
[pid  8120] munmap(0xb7c10000, 33608)   = 0
[pid  8120] munmap(0xb7c0b000, 17560)   = 0
[pid  8120] _exit(18)                   = ?
Process 8120 detached
<... read resumed> "", 1048)            = 0
--- SIGCHLD (Child exited) @ 0 (0) ---
...
xterm is unable to open the pseudo terminal device, it should be OK when deployed to the box (there is no full device mapping on my mirror system; it is just chroot-ed to some sub-directory of hosting system).

Let's create the package:
dev# ./filopack.sh --pack xterm-237
Configuration file .filopack/.config file found and used
-rw-r--r-- root/root      1381 2008-12-20 14:27:33 sys/X11/share/pixmaps/xterm-color_32x32.xpm
-rw-r--r-- root/root      2710 2008-12-20 14:27:33 sys/X11/share/pixmaps/xterm-color_48x48.xpm
-rw-r--r-- root/root      2110 2008-12-20 14:27:33 sys/X11/share/pixmaps/xterm_32x32.xpm
-rw-r--r-- root/root      2586 2008-12-20 14:27:33 sys/X11/share/pixmaps/xterm_48x48.xpm
-rw-r--r-- root/root    174641 2008-12-20 14:27:32 sys/X11/man/man1/xterm.1
-rw-r--r-- root/root      2678 2008-12-20 14:27:33 sys/X11/man/man1/resize.1
-rw-r--r-- root/root      2967 2008-12-20 14:27:33 sys/X11/man/man1/uxterm.1
-rw-r--r-- root/root      3188 2008-12-20 14:27:33 sys/X11/man/man1/koi8rxterm.1
-rw-r--r-- root/root      6853 2008-12-20 14:27:33 sys/X11/lib/X11/app-defaults/XTerm
-rw-r--r-- root/root      4324 2008-12-20 14:27:33 sys/X11/lib/X11/app-defaults/XTerm-color
-rw-r--r-- root/root      2109 2008-12-20 14:27:33 sys/X11/lib/X11/app-defaults/UXTerm
-rw-r--r-- root/root       747 2008-12-20 14:27:33 sys/X11/lib/X11/app-defaults/KOI8RXTerm
-rwxr-xr-x root/root    334759 2008-12-20 14:27:32 sys/X11/bin/xterm
-rwxr-xr-x root/root     11157 2008-12-20 14:27:32 sys/X11/bin/resize
-rwxr-xr-x root/root      2088 2008-12-20 14:27:32 sys/X11/bin/uxterm
-rwxr-xr-x root/root      2163 2008-12-20 14:27:32 sys/X11/bin/koi8rxterm
... and deploy the package to the box:
dev# ./filopack.sh --upload xterm-237
Configuration file .filopack/.config file found and used
Sure to upload xterm-237 to root@storage:/mnt/C (y/n)?y
root@storage's password:
root@storage's password:
Now on the box, we can install it:
box# [root@storage C]# ./filopack.sh --install xterm-237
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Module xterm-237 not found in package list - use 'tar.bz2' extension
Sure to unpack xterm-237 locally at /mnt/C (y/n)?y
sys/X11/share/pixmaps/xterm-color_32x32.xpm
sys/X11/share/pixmaps/xterm-color_48x48.xpm
sys/X11/share/pixmaps/xterm_32x32.xpm
sys/X11/share/pixmaps/xterm_48x48.xpm
sys/X11/man/man1/xterm.1
sys/X11/man/man1/resize.1
sys/X11/man/man1/uxterm.1
sys/X11/man/man1/koi8rxterm.1
sys/X11/lib/X11/app-defaults/XTerm
sys/X11/lib/X11/app-defaults/XTerm-color
sys/X11/lib/X11/app-defaults/UXTerm
sys/X11/lib/X11/app-defaults/KOI8RXTerm
sys/X11/bin/xterm
sys/X11/bin/resize
sys/X11/bin/uxterm
sys/X11/bin/koi8rxterm
And test the installation:
box# xterm
Cannot chown /dev/ttyp4 to 0,0: Read-only file system
Cannot chmod /dev/ttyp4 to 620 currently 666: Read-only file system
Cannot chown /dev/ttyp4 to 0,0: Read-only file system
The problem here is that xterm is trying to change the rights for the terminal device it is using (/dev/ttyp4 in this case) in order to prevent a read access of an unauthorized person (octal value 620 means that only root can read and write the terminal and users from the same group can just write). However, since the file system is read-only the chmod operation does not succeed and so the terminal device stays readable for everyone.
I was dealing with the same issue during the dropbear build.

Anyway, in case you are using the box on a trusted network with no one "sniffing around" and have it properly secured from the outside world then the warning should not mean any harm. Maybe it could be solved by a change in configuration (or by a slight change in source code), but I am not going to solve it now.


Read more...

2008/12/08

WMU-6500FS - built sw -Python 2.5.2

[Homepage]

[files] [binary]

First I briefly write about the initial build:
dev# cd /usr/local/src
dev# wget http://python.org/ftp/python/2.5.2/Python-2.5.2.tgz
dev# tar xvzf Python-2.5.2.tgz
dev# cd Python-2.5.2
dev# ./configure --prefix=/mnt/C/sys --enable-shared
dev# make
There was an error with missing 'yperr_string' symbol (I am not sure whether it was during the build or when I launched python console). The symbol was needed by the nis extension module. I do not know how on earth I would utilize an Interface to Sun's NIS (Yellow pages) so as a workaround I simply disabled it.
dev# make
dev# make install
dev# python
Another failure - error while loading shared libraries: libpython2.5.so.1.0 (it is located in /mnt/C/sys/lib).
As a solution, if there was a dynamic loader configuration on the box (/etc/ld.so.conf.d) we could do following:
dev# echo /usr/local/lib > /etc/ld.so.conf.d/python2.5.conf
dev# ldconfig
As there is not, we can use the LD_LIBRARY_PATH variable instead:
dev# nano /mnt/C/sys/etc/rc-local
Add following:
    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/mnt/C/sys/lib
Now everything works ok.

After some time I realized that there is a readline extension module missing on the system. This module is needed for interactive console applications, and particularly is needed for the deluge console:
box# deluge --ui=null
[INFO    ] 21:58:04 main:93 Deluge ui 1.0.5
[DEBUG   ] 21:58:04 main:94 options: {'config': None, 'logfile': None, 'ui': 'null'}
[DEBUG   ] 21:58:04 main:95 args: []
[DEBUG   ] 21:58:05 configmanager:44 ConfigManager started..
[INFO    ] 21:58:05 main:98 Starting ui..
[DEBUG   ] 21:58:05 ui:44 UI init..
[DEBUG   ] 21:58:05 configmanager:88 Getting config 'ui.conf'
[DEBUG   ] 21:58:05 config:47 Config created with filename: ui.conf
[DEBUG   ] 21:58:05 config:48 Config defaults: {'default_ui': 'gtk'}
[INFO    ] 21:58:05 ui:68 Starting NullUI..
[DEBUG   ] 21:58:06 client:54 CoreProxy init..
Traceback (most recent call last):
  File "/usr/bin/deluge", line 8, in <module>
    load_entry_point('deluge==1.0.5', 'console_scripts', 'deluge')()
  File "/mnt/C/sys/lib/python2.5/site-packages/deluge-1.0.5-py2.5-linux-i686.egg/deluge/main.py", line 99, in start_ui
    UI(options, args)
  File "/mnt/C/sys/lib/python2.5/site-packages/deluge-1.0.5-py2.5-linux-i686.egg/deluge/ui/ui.py", line 69, in __init__
    from deluge.ui.null.deluge_shell import NullUI
  File "/mnt/C/sys/lib/python2.5/site-packages/deluge-1.0.5-py2.5-linux-i686.egg/deluge/ui/null/deluge_shell.py", line 29, in <module>
    import readline
ImportError: No module named readline
We can replicate this problem directly from the python console:
dev# python
Python 2.5.2 (r252:60911, May  3 2008, 16:19:27)
[GCC 3.3.6] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import readline
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named readline
>>>
So let's look at the old log of python's configure script to see why the readline extension package was not built:
dev# cat config.log | grep readline
configure:20980: checking for readline in -lreadline
configure:21015: gcc -pthread -o conftest -g -O2   conftest.c -lreadline  -lpthread -ldl  -lutil >&5
/usr/bin/ld: cannot find -lreadline
| char readline ();
| return readline ();
configure:21126: checking for rl_callback_handler_install in -lreadline
configure:21161: gcc -pthread -o conftest -g -O2   conftest.c -lreadline  -lpthread -ldl  -lutil >&5
/usr/bin/ld: cannot find -lreadline
configure:21254: checking for rl_pre_input_hook in -lreadline
configure:21289: gcc -pthread -o conftest -g -O2   conftest.c -lreadline  -lpthread -ldl  -lutil >&5
/usr/bin/ld: cannot find -lreadline
configure:21325: checking for rl_completion_matches in -lreadline
configure:21360: gcc -pthread -o conftest -g -O2   conftest.c -lreadline  -lpthread -ldl  -lutil >&5
/usr/bin/ld: cannot find -lreadline
ac_cv_lib_readline_readline=no
ac_cv_lib_readline_rl_callback_handler_install=no
ac_cv_lib_readline_rl_completion_matches=no
ac_cv_lib_readline_rl_pre_input_hook=no
We can see that the library simply was not on the system and so the python configure script decided to exclude it's python wrapper from the build.

Before we proceed, we have to build the GNU readline and create a new package for it. How it was done is described in this post.

Now we have the libreadline.so library on the system and so can proceed to re-configure and re-build the python package. First we create a new timestamp file for python package and touch all the old files (in order to be included in the package). We can use the filopack script for that purpose:
dev# cd /mnt/C
dev# ./filopack.sh --init python-2.5.2
dev# ./filopack.sh --touch python-2.5.2
Configuration file .filopack/.config file found and used
Sure to touch all files of the python-2.5.2 locally at /mnt/C (y/n)?y
...
Now we can move to the python source directory and re-configure the build:
dev# cd /usr/local/src/Python-2.5.2
dev# ./configure --prefix=/mnt/C/sys --enable-shared
...
Let's check whether the readline package was found and included to the build:
dev# cat config.log | grep readline
configure:20980: checking for readline in -lreadline
configure:21015: gcc -pthread -o conftest -I/mnt/C/sys/include -I/mnt/C/sys/X11/include  -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib conftest.c -lreadline  -lpthread -ldl  -lutil >&5
/mnt/C/sys/lib/libreadline.so: undefined reference to `tgetnum'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tgoto'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tgetflag'
/mnt/C/sys/lib/libreadline.so: undefined reference to `BC'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tputs'
/mnt/C/sys/lib/libreadline.so: undefined reference to `PC'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tgetent'
/mnt/C/sys/lib/libreadline.so: undefined reference to `UP'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tgetstr'
...
ac_cv_lib_readline_readline=no
ac_cv_lib_readline_rl_callback_handler_install=no
ac_cv_lib_readline_rl_completion_matches=no
ac_cv_lib_readline_rl_pre_input_hook=no
The situation changed, the readline library is (presumably) no longer missing, but still there are some missing symbols and thus the configure test failed.

After a short googling (for query "libreadline.so: undefined reference to BC PC UP") I found this page.
It seems that the missing symbols should be present in ncurses library.

Let's look what symbols are at the library in our system:
dev# nm /lib/libncurses.so | grep "tgetnum\|tgoto\|tgetflag\|BC\|PC\|UP"
0003a544 B BC
0003a770 B PC
0003a540 B UP
000248e1 T tgetflag
00024958 T tgetnum
00024b0c T tgoto
It means that the symbols are there, but the python configuration script did not manage to find the library.

For quick feedback for our experiments we can create a dummy C file and use the same compiler command line as the configure script did:
dev# echo "int main() { return 0; }" > conftest.c
dev# gcc -pthread -o conftest -I/mnt/C/sys/include -I/mnt/C/sys/X11/include  -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib conftest.c -lreadline  -lpthread -ldl  -lutil
/mnt/C/sys/lib/gcc/i386-linux-uclibc/4.1.2/../../../../i386-linux-uclibc/lib/crt1.o: In function `_start':
crt1.S:(.text+0x27): undefined reference to `main'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tgetnum'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tgoto'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tgetflag'
/mnt/C/sys/lib/libreadline.so: undefined reference to `BC'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tputs'
/mnt/C/sys/lib/libreadline.so: undefined reference to `PC'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tgetent'
/mnt/C/sys/lib/libreadline.so: undefined reference to `UP'
/mnt/C/sys/lib/libreadline.so: undefined reference to `tgetstr'
collect2: ld returned 1 exit status
We see that the linker ended up with the same error. Now we can try to fix the command line. There is missing ncurses library on the command line; what about adding it explicitly?
dev# gcc -pthread -o conftest -I/mnt/C/sys/include -I/mnt/C/sys/X11/include  -L/mnt/C/sys/lib
 -L/mnt/C/sys/X11/lib conftest.c -lncurses -lreadline  -lpthread -ldl  -lutil
The test build now works, so now we can remove the temporary C-file and re-configure the python build:
dev# rm conftest.c

dev# LIBS="-lncurses" ./configure --prefix=/mnt/C/sys --enable-shared
...
Let's make sure that readline package passed the configure test:
 
dev# cat config.log | grep readline
configure:20980: checking for readline in -lreadline
configure:21015: gcc -pthread -o conftest -I/mnt/C/sys/include -I/mnt/C/sys/X11/include  -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib conftest.c -lreadline  -lpthread -ldl -lncurses -lutil >&5
configure:21126: checking for rl_callback_handler_install in -lreadline
configure:21161: gcc -pthread -o conftest -I/mnt/C/sys/include -I/mnt/C/sys/X11/include  -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib conftest.c -lreadline  -lreadline -lpthread -ldl -lncurses -lutil >&5
configure:21254: checking for rl_pre_input_hook in -lreadline
configure:21289: gcc -pthread -o conftest -I/mnt/C/sys/include -I/mnt/C/sys/X11/include  -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib conftest.c -lreadline  -lreadline -lpthread -ldl -lncurses -lutil >&5
configure:21325: checking for rl_completion_matches in -lreadline
configure:21360: gcc -pthread -o conftest -I/mnt/C/sys/include -I/mnt/C/sys/X11/include  -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib conftest.c -lreadline  -lreadline -lpthread -ldl -lncurses -lutil >&5
ac_cv_lib_readline_readline=yes
ac_cv_lib_readline_rl_callback_handler_install=yes
ac_cv_lib_readline_rl_completion_matches=yes
ac_cv_lib_readline_rl_pre_input_hook=yes
Yes, everything is ok now, we can build the source and install the binaries:
dev# make
...
/mnt/C/sys/include/sqlite3.h: version 3.5.8
INFO: Can't locate Tcl/Tk libs and/or headers
building 'readline' extension
gcc -pthread -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -I/usr/local/src/Python-2.5.2/./Include -I/mnt/C/sys/include -I. -IInclude -I./Include -I/usr/local/include -I/usr/local/src/Python-2.5.2/Include -I/usr/local/src/Python-2.5.2 -c /usr/local/src/Python-2.5.2/Modules/readline.c -o build/temp.linux-i686-2.5/usr/local/src/Python-2.5.2/Modules/readline.o
gcc -pthread -shared -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes build/temp.linux-i686-2.5/usr/local/src/Python-2.5.2/Modules/readline.o -L/usr/lib/termcap -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib -L/usr/local/lib -L. -lreadline -lncurses -lpython2.5 -o build/lib.linux-i686-2.5/readline.so
building 'nis' extension
gcc -pthread -fPIC -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -I/usr/local/src/Python-2.5.2/./Include -I/mnt/C/sys/include -I. -IInclude -I./Include -I/usr/local/include -I/usr/local/src/Python-2.5.2/Include -I/usr/local/src/Python-2.5.2 -c /usr/local/src/Python-2.5.2/Modules/nismodule.c -o build/temp.linux-i686-2.5/usr/local/src/Python-2.5.2/Modules/nismodule.o
gcc -pthread -shared -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes build/temp.linux-i686-2.5/usr/local/src/Python-2.5.2/Modules/nismodule.o -L/mnt/C/sys/lib -L/mnt/C/sys/X11/lib -L/usr/local/lib -L. -lnsl -lpython2.5 -o build/lib.linux-i686-2.5/nis.so
*** WARNING: renaming "nis" since importing it failed: dynamic module does not define init function (initnis)
running build_scripts

dev# make install
...
Now, let's test the new installation:
dev# python
Python 2.5.2 (r252:60911, Dec  8 2008, 07:26:01)
[GCC 4.1.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import readline
This is fine; and what about the deluge console:
dev# deluge --ui=null
[INFO    ] 07:28:18 main:93 Deluge ui 1.0.5
[DEBUG   ] 07:28:18 main:94 options: {'config': None, 'logfile': None, 'ui': 'null'}
[DEBUG   ] 07:28:18 main:95 args: []
[DEBUG   ] 07:28:18 configmanager:44 ConfigManager started..
[INFO    ] 07:28:18 main:98 Starting ui..
[DEBUG   ] 07:28:18 ui:44 UI init..
[DEBUG   ] 07:28:18 configmanager:88 Getting config 'ui.conf'
[DEBUG   ] 07:28:18 config:47 Config created with filename: ui.conf
[DEBUG   ] 07:28:18 config:48 Config defaults: {'default_ui': 'gtk'}
[INFO    ] 07:28:18 ui:68 Starting NullUI..
[DEBUG   ] 07:28:18 client:54 CoreProxy init..
Welcome to deluge-shell. Type 'help' to see a list of available commands.
> help

Available commands:
        * add: Add a torrent
        * config-set: Change a configuration setting
        * configs: Show configuration values
        * connect: Connect to a new deluge server.
        * del: Remove a torrent
        * exit: Exit from the client.
        * halt: Shutdown the deluge server.
        * help: Show help
        * info: Show information about the torrents
        * pause: Pause a torrent
        * quit: Exit from the client.
        * resume: Resume a torrent
        * rm: Remove a torrent
> quit

Thanks
Everything is now ok, we can create a new python package:
dev# cd /mnt/C
dev# ./filopack.sh --pack python-2.5.2
...
I realized that there are many files which are not a necessary part of the resulting package. Thhose are files with .pyc and .pyo extensions - just in time compiled python files. So I removed them from the list and re-created the package:
dev# ./filopack.sh -R --pack python-2.5.2
...
I have uploaded the resulting package on the web site. Now we can fix the problem on the box.

First let's look at the summary of installed files:
box# cd /mnt/C
box# ./filopack.sh --summary
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)

Filodej package summary:
Package name                          downloaded [#installed/#total]
--------------------------------------------------------------------
...
python-2.5.2                              no           [3525/3648]
...
... so the python package is installed; let's remove it:
box# ./filopack.sh --remove python-2.5.2
Configuration file .filopack/.config file found and used
Sure to remove python-2.5.2 locally at /mnt/C (y/n)?y
...
Now we can download and install both readline-5.2:
box# ./filopack.sh --download readline-5.2
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Downloading package readline-5.2 from http://filodej.ic.cz ...
connected!

Length: 573 [text/plain]
connected!

Length: 265,962 [application/x-tar]

box# ./filopack.sh --install readline-5.2
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Sure to unpack readline-5.2 locally at /mnt/C (y/n)?y
sys/lib/libreadline.a
sys/lib/libhistory.a
sys/lib/libhistory.so.5
sys/lib/libhistory.so.5.2
sys/lib/libhistory.so
sys/lib/libreadline.so.5.2
sys/lib/libreadline.so.5
sys/lib/libreadline.so
sys/man/man3/readline.3
sys/man/man3/history.3
sys/include/readline/rltypedefs.h
sys/include/readline/chardefs.h
sys/include/readline/history.h
sys/include/readline/keymaps.h
sys/include/readline/readline.h
sys/include/readline/rlconf.h
sys/include/readline/rlstdc.h
sys/include/readline/tilde.h
sys/info/readline.info
sys/info/rluserman.info
sys/info/history.info
... and python-2.5.2 package:
box# ./filopack.sh --download python-2.5.2
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Downloading package python-2.5.2 from http://filodej.ic.cz ...
connected!

Length: 134,205 [text/plain]
connected!

Length: 12,984,464 [application/x-tar]

box# ./filopack.sh --install python-2.5.2
Configuration file .filopack/.config file found and used
Retrieving package index... (Connecting to http://filodej.ic.cz)
Sure to unpack python-2.5.2 locally at /mnt/C (y/n)?y
...
After the installation everything seemed ok:
box# python
Python 2.5.2 (r252:60911, Dec  8 2008, 07:26:01)
[GCC 4.1.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import readline
>>>
... but surprisingly one more problem arose when I ran the deluge console:
box# deluged.sh

box# deluge --ui=null
[INFO    ] 16:56:44 main:93 Deluge ui 1.0.5
[DEBUG   ] 16:56:44 main:94 options: {'config': None, 'logfile': None, 'ui': 'null'}
[DEBUG   ] 16:56:44 main:95 args: []
[DEBUG   ] 16:56:45 configmanager:44 ConfigManager started..
[INFO    ] 16:56:45 main:98 Starting ui..
[DEBUG   ] 16:56:45 ui:44 UI init..
[DEBUG   ] 16:56:45 configmanager:88 Getting config 'ui.conf'
[DEBUG   ] 16:56:45 config:47 Config created with filename: ui.conf
[DEBUG   ] 16:56:45 config:48 Config defaults: {'default_ui': 'gtk'}
[INFO    ] 16:56:45 ui:68 Starting NullUI..
[DEBUG   ] 16:56:46 client:54 CoreProxy init..
Welcome to deluge-shell. Type 'help' to see a list of available commands.
Traceback (most recent call last):
  File "/usr/bin/deluge", line 8, in 
    load_entry_point('deluge==1.0.5', 'console_scripts', 'deluge')()
  File "/mnt/C/sys/lib/python2.5/site-packages/deluge-1.0.5-py2.5-linux-i686.egg/deluge/main.py", line 99, in start_ui
    UI(options, args)
  File "/mnt/C/sys/lib/python2.5/site-packages/deluge-1.0.5-py2.5-linux-i686.egg/deluge/ui/ui.py", line 70, in __init__
    ui = NullUI(args)
  File "/mnt/C/sys/lib/python2.5/site-packages/deluge-1.0.5-py2.5-linux-i686.egg/deluge/ui/null/deluge_shell.py", line 470, in __init__
    readline.read_init_file()
IOError: [Errno 2] No such file or directory
When we look at the python readline documentation we see the following:
readline.read_init_file([filename])
    Parse a readline initialization file. The default filename is the last filename used.
It seems that there is a readline initialization file missing on the box (probably it was outside the /mnt/C/sys directory tree and so was not transferred to the box).

Let's try to replicate the problem directly from python console:
box# python
Python 2.5.2 (r252:60911, Dec  8 2008, 07:26:01)
[GCC 4.1.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import readline
>>> readline.read_init_file()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IOError: [Errno 2] No such file or directory
When I tried it on the colinux system, everything was ok:
dev# python
Python 2.5.2 (r252:60911, Dec  8 2008, 07:26:01)
[GCC 4.1.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import readline
>>> readline.read_init_file()
>>>
Hopefully the system call trace will help:
box# strace -o readline-init.txt python
Python 2.5.2 (r252:60911, Dec  8 2008, 07:26:01)
[GCC 4.1.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import readline
>>> readline.read_init_file()
Traceback (most recent call last):
  File "<stdin>", line 1, in 
IOError: [Errno 2] No such file or directory
>>>
Let's look at the end of the trace log:
box# tail -n 100 readline-init.txt
...
rt_sigprocmask(SIG_BLOCK, [INT], [RTMIN], 8) = 0
ioctl(0, SNDCTL_TMR_STOP or TCSETSW, {B38400 opost isig icanon echo ...}) = 0
rt_sigprocmask(SIG_SETMASK, [RTMIN], NULL, 8) = 0
rt_sigaction(SIGWINCH, {SIG_DFL}, {0x40139066, [], SA_RESTORER|SA_RESTART, 0x401c45a8}, 8) = 0
time(NULL)                              = 1228752774
stat("/mnt/C/sys/root/.inputrc", 0xbffff4ec) = -1 ENOENT (No such file or directory)
stat("/etc/inputrc", 0xbffff4ec)        = -1 ENOENT (No such file or directory)
write(2, "Traceback (most recent call last"..., 35) = 35
...
write(2, "  File \"<stdin>\", line 1, in <mo"..., 38) = 38
write(2, "IOError", 7)                  = 7
write(2, ": ", 2)                       = 2
write(2, "[Errno 2] No such file or direct"..., 35) = 35
write(2, "\n", 1)                       = 1
ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
rt_sigprocmask(SIG_BLOCK, [INT], [RTMIN], 8) = 0
ioctl(0, TIOCGWINSZ, {ws_row=31, ws_col=84, ws_xpixel=0, ws_ypixel=0}) = 0
ioctl(0, TIOCSWINSZ, {ws_row=31, ws_col=84, ws_xpixel=0, ws_ypixel=0}) = 0
ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(0, SNDCTL_TMR_STOP or TCSETSW, {B38400 opost isig -icanon -echo ...}) = 0
rt_sigprocmask(SIG_SETMASK, [RTMIN], NULL, 8) = 0
rt_sigaction(SIGWINCH, {0x40139066, [], SA_RESTORER|SA_RESTART, 0x401c45a8}, {SIG_DFL}, 8) = 0
write(1, ">>> ", 4)                     = 4
select(1, [0], NULL, NULL, NULL)        = 1 (in [0])
rt_sigprocmask(SIG_BLOCK, NULL, [RTMIN], 8) = 0
rt_sigprocmask(SIG_BLOCK, NULL, [RTMIN], 8) = 0
read(0, "\4", 1)                        = 1
rt_sigprocmask(SIG_BLOCK, [INT], [RTMIN], 8) = 0
ioctl(0, SNDCTL_TMR_STOP or TCSETSW, {B38400 opost isig icanon echo ...}) = 0
rt_sigprocmask(SIG_SETMASK, [RTMIN], NULL, 8) = 0
rt_sigaction(SIGWINCH, {SIG_DFL}, {0x40139066, [], SA_RESTORER|SA_RESTART, 0x401c45a8}, 8) = 0
write(2, "\n", 1)                       = 1
rt_sigaction(SIGINT, {SIG_DFL}, {0x40139066, [], SA_RESTORER, 0x401c45a8}, 8) = 0
munmap(0x402ad000, 16188)               = 0
munmap(0x402b1000, 216100)              = 0
_exit(0)                                = ?
So we found that neither /etc/inputrc nor /mnt/C/sys/root/.inputrc is present on the box and thus the readline initialization complains.

As a solution I choosen to copy the initialization file to root's user directory and re-build the readline package:
dev# cp /etc/inputrc /mnt/C/sys/root/.inputrc
dev# echo "sys/root/.inputrc" >> .filopack/readline-5.2.lst
dev# ./filopack.sh -R --pack readline-5.2
After the re-deployment to the box everything works ok:
dev# deluge --ui=null
[INFO    ] 17:22:53 main:93 Deluge ui 1.0.5
[DEBUG   ] 17:22:53 main:94 options: {'config': None, 'logfile': None, 'ui': 'null'}
[DEBUG   ] 17:22:53 main:95 args: []
[DEBUG   ] 17:22:53 configmanager:44 ConfigManager started..
[INFO    ] 17:22:53 main:98 Starting ui..
[DEBUG   ] 17:22:53 ui:44 UI init..
[DEBUG   ] 17:22:53 configmanager:88 Getting config 'ui.conf'
[DEBUG   ] 17:22:53 config:47 Config created with filename: ui.conf
[DEBUG   ] 17:22:53 config:48 Config defaults: {'default_ui': 'gtk'}
[INFO    ] 17:22:53 ui:68 Starting NullUI..
[DEBUG   ] 17:22:55 client:54 CoreProxy init..
Welcome to deluge-shell. Type 'help' to see a list of available commands.
> help

Available commands:
        * add: Add a torrent
        * config-set: Change a configuration setting
        * configs: Show configuration values
        * connect: Connect to a new deluge server.
        * del: Remove a torrent
        * exit: Exit from the client.
        * halt: Shutdown the deluge server.
        * help: Show help
        * info: Show information about the torrents
        * pause: Pause a torrent
        * quit: Exit from the client.
        * resume: Resume a torrent
        * rm: Remove a torrent

> connect


> info

*** ID: 33820db6dd5e5928d23bc811bbac2f4ae94cb882
*** Name: ubuntu-8.10-desktop-i386.iso
*** Path: torrentfiles
*** Completed: 0.0 KiB/698.8 MiB
*** Status: Paused


> resume

Usage: resume  [ ...]


> resume 33820db6dd5e5928d23bc811bbac2f4ae94cb882


> info

*** ID: 33820db6dd5e5928d23bc811bbac2f4ae94cb882
*** Name: ubuntu-8.10-desktop-i386.iso
*** Path: torrentfiles
*** Completed: 2.1 MiB/698.8 MiB
*** Status: Downloading
*** Download Speed: 60.9 KiB/s
*** Upload Speed: 0.0 KiB/s
*** ETA: 3h 15m


> exit

Thanks
box#
That's all for now.

Read more...