2008/07/26

WMU-6500FS - Deluge torrent (part II)



Overview

In previous part we have started building the deluge torrent ...

We ended up with a curious error, process was aborted after an exception thrown in libtorrent::bdecode. It seems that for some reason there is exception thrown and not caught which causes immediate process termination.

Screenshot

Here is a screenshot of the remote debugging in action:

Uncaught exception

When we look at the strace output:
...
close(11)                               = 0
write(1, "Applying preferences\n", 21Applying preferences
)  = 21
write(1, "Starting DHT...\n", 16Starting DHT...
)       = 16
open("/root/.config/deluge/dht.state", O_RDONLY) = -1 ENOENT (No such file or directory)
rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, 8) = 0
kill(31503, SIGABRT)                    = 0
--- SIGABRT (Aborted) @ 0 (0) ---
+++ killed by SIGABRT +++
Process 31503 detached
Call stack at the moment of termination:
...
Program received signal SIG32, Real-time event 32.
0x4019cb64 in __rt_sigsuspend () from /lib/libc.so.0
(gdb) Quit
(gdb) where
#0  0x4019cb64 in __rt_sigsuspend () from /lib/libc.so.0
#1  0x4019cb9b in sigsuspend () from /lib/libc.so.0
#2  0x40139cf9 in __pthread_wait_for_restart_signal () from /lib/libpthread.so.0
#3  0x401396bd in pthread_create () from /lib/libpthread.so.0
#4  0x416c9f26 in boost::thread::start_thread () from /mnt/C/sys/lib//libboost_thread-gcc33-mt-1_35.so.1.35.0
#5  0x415d68c8 in disk_io_thread (this=0x821acc8, block_size=-4) at thread.hpp:151
...
An the corresponding source file src/deluge_core.cpp
    boost::filesystem::path tempPath(DHT_path, empty_name_check);
    boost::filesystem::ifstream DHT_state_file(tempPath, std::ios_base::binary);
    DHT_state_file.unsetf(std::ios_base::skipws);

    entry DHT_state;
    try
    {
(1)     DHT_state = bdecode(std::istream_iterator<char>(DHT_state_file),
            std::istream_iterator<char>());
        M_ses->start_dht(DHT_state);
        //        printf("DHT state recovered.\r\n");

        //        // Print out the state data from the FILE (not the session!)
        //        printf("Number of DHT peers in recovered state: %ld\r\n", count_DHT_peers(DHT_state));

    }
(2) catch (std::exception&)
    {
        printf("No DHT file to resume\r\n");
        M_ses->start_dht();
    }
... it seems obvious what happened. Application tried to open the dht.state file and since there was no such file an exception was thrown in the std::istream_iterator constructor (source code line marked as (1)). This exception is nevertheless not caught in the catch block (source code line marked as (2)). I do not know why, there is no aparent reason (the exception has definitely the std::exception base), the fact is that due to this the application is aborted.

Uncaught exception - workaround

For now I decided to make the following workaround, explicitly handle the missing file and so bypass the exception:
    boost::filesystem::path tempPath(DHT_path, empty_name_check);
    boost::filesystem::ifstream DHT_state_file(tempPath, std::ios_base::binary);
    DHT_state_file.unsetf(std::ios_base::skipws);

    entry DHT_state;
    if ( !exists( tempPath ) )
    {
        printf("No DHT file to resume\r\n");
        M_ses->start_dht();
    }
    else try
    {
        DHT_state = bdecode(std::istream_iterator<char>(DHT_state_file),
            std::istream_iterator<char>());
        M_ses->start_dht(DHT_state);
        //        printf("DHT state recovered.\r\n");

        //        // Print out the state data from the FILE (not the session!)
        //        printf("Number of DHT peers in recovered state: %ld\r\n", count_DHT_peers(DHT_state));

    }
    catch (std::exception&)
    {
        printf("No DHT file to resume\r\n");
        M_ses->start_dht();
    }
The unified diff looks as follows:
dev# diff -u src/deluge_core.cpp.old src/deluge_core.cpp
--- src/deluge_core.cpp.old     2008-06-04 20:03:49.000000000 -0600
+++ src/deluge_core.cpp 2008-07-02 16:32:01.000000000 -0600
@@ -1889,7 +1889,12 @@
     DHT_state_file.unsetf(std::ios_base::skipws);

     entry DHT_state;
-    try
+    if ( !exists( tempPath ) )
+    {
+        printf("No DHT file to resume\r\n");
+        M_ses->start_dht();
+    }
+    else try
     {
         DHT_state = bdecode(std::istream_iterator<char>(DHT_state_file),
             std::istream_iterator<char>());
With this workaround we are able to sucessfully open the application. But when we add a torrent file, there is another (but similar) problem. Another exception is thrown which leads to process termination. Since the local debugging is really slow on the box, before we proceed to solve this problem, let's look how to debug remotely.

Remote debugging

For remote debugging we will use the gdbserver. Following relevant inf is in the GDB remote debugging manual:
gdbserver is a control program for Unix-like systems, which allows you to connect your program 
with a remote GDB via target remote - but without linking in the usual debugging stub.
...
The `host:2345' argument means that gdbserver is to expect a TCP connection from machine `host' 
to local TCP port 2345. (Currently, the `host' part is ignored.)
...
On some targets, gdbserver can also attach to running programs. This is accomplished via the 
--attach argument.
...
On the GDB host machine, you need an unstripped copy of your program, since GDB needs symbols 
and debugging information. Start up GDB as usual, using the name of the local copy of your 
program as the first argument. ... After that, use target remote to establish communications 
with gdbserver.
The use of gdbserver is ideal for us, since due to the deployment process we have identical copied of the debugged program both on the box# and on the build dev# system.

So on the box we can run the deluge
box# deluge &
[1] 29193
... and attach to it with the gdbserver:
box# gdbserver colinux:2345 --attach 29193
Attached; pid = 29193
Listening on port 2345
or alternatively we can run the gdbserver running the deluge roght from the beggining:
box# gdbserver colinux:2345 `which python` `which deluge`
Process /mnt/C/sys/bin/python created; pid = 29235
Listening on port 2345
Now we can connect to the gdbserver with gdb running on the build machine (PC). Since the deluge just a python script, the executable we will debug is in fact python interpreter:
dev# gdb `which python`
...
(gdb) target remote 192.168.1.104:2345
(gdb) cont
Continuing.
[New Thread 1024]
...
Ok, that's it.

Note: sometimes, when we interrupt the debugging process, the deluge is not properly killed, there remains an instance. When we start deluge again the output looks like this:
...
create proxy object
create iface
send to iface

Child exited with retcode = 0

Child exited with status 0
GDBserver exiting
In such situation we have to find the hanging processes and kill them:
box# ps ax | grep deluge
29429 root      14136 S     1.7  /mnt/C/sys/bin/python /mnt/C/sys/bin/deluge
29440 root      14136 S     0.0  /mnt/C/sys/bin/python /mnt/C/sys/bin/deluge
29482 root        324 S     0.0  grep deluge
box# kill -9 29429 29440

Uncaught exceptions - looking for a cause

So as I already said, the workaround pushed us forward, but another exception is thrown later on, when we open a torrent file:
(gdb) where
...
#81 0xbfffca60 in ?? ()
#82 0x4027140f in __cxa_call_unexpected () from /mnt/C/sys/lib//libstdc++.so.5
#83 0x4027140f in __cxa_call_unexpected () from /mnt/C/sys/lib//libstdc++.so.5
#84 0x40271439 in std::terminate () from /mnt/C/sys/lib//libstdc++.so.5
#85 0x40271560 in __cxa_throw () from /mnt/C/sys/lib//libstdc++.so.5
#86 0x414d0d39 in libtorrent::entry::operator[] (this=0x821a4e0, key=0x41644ef4 "url-list") at entry.hpp:77
#87 0x415a5bc8 in libtorrent::torrent_info::read_torrent_info (this=0xbfffd090, torrent_file=@0xbfffcee0)
    at libtorrent/src/torrent_info.cpp:519
#88 0x4159e59e in torrent_info (this=0xbfffd090, torrent_file=@0xbfffcee0) at libtorrent/src/torrent_info.cpp:236
#89 0x41629068 in internal_get_torrent_info (torrent_name=@0x6) at src/deluge_core.cpp:264
#90 0x4162d0e7 in torrent_dump_file_info (self=0x0, args=0x41a7ab2c) at stl_alloc.h:652
#91 0x4005d122 in PyCFunction_Call (func=0x4127122c, arg=0x41a7ab2c, kw=0x6) at Objects/methodobject.c:108
...
When we look at the stack frame 87 and torrent_file structure:
(gdb) frame 87
#87 0x415a5bc8 in libtorrent::torrent_info::read_torrent_info (this=0xbfffd090, torrent_file=@0xbfffcee0)
    at libtorrent/src/torrent_info.cpp:519
519                             entry const& url_seeds = torrent_file["url-list"];

(gdb) list
514                     catch (type_error) {}
515
516                     // if there are any url-seeds, extract them
517                     try
518                     {
519                             entry const& url_seeds = torrent_file["url-list"];
520                             if (url_seeds.type() == entry::string_t)
521                             {
522                                     m_url_seeds.push_back(url_seeds.string());
523                             }

(gdb) print torrent_file
$1 = (const libtorrent::entry &) @0xbfffcee0: {m_type = dictionary_t, {data = "øv!\b\004\000\000\000ògB@",
    dummy_aligner = 17316280056}}
... and to the src/deluge-torrent-0.5.9.3/libtorrent/src/entry.cpp file :
#ifndef BOOST_NO_EXCEPTIONS
        const entry& entry::operator[](char const* key) const
        {
                dictionary_type::const_iterator i = dict().find(key);
                if (i == dict().end()) throw type_error(
                        (std::string("key not found: ") + key).c_str());
                return i->second;
        }

        const entry& entry::operator[](std::string const& key) const
        {
                return (*this)[key.c_str()];
        }
#endif
... we can see that the torrent_file is a python-like dictionary, when one tries to access a value stored with a key and there is no such entry, an exception is thrown. As in previous case neither this exception is not caught. It is time to stop with workarounds and look at the exception issue in a finer detail.
This discussion contains an ellegant test for exception catching. It seems that I am not alone having this sort of problems:
> So, the throw+catch are not hooking up properly under uClibc.
> This is uClibc's problem.

What version of uClibc?  gcc?  binutils?  How did you build your
toolchain?  Was gcc built with --enable-sjlj-exceptions?  What
kernel version are you using?
...
> part3:
> comment on gdb #5: __cxa_throw ()
> >From the backtrace above it looks that somethings goes wrong during stack 
> unwinding - does exception catching work for other C++ programs ?

It does if gcc was built properly...

    http://codepoet.org/throw1.cpp
    http://codepoet.org/throw2.cpp
So let's make the same test:
dev# cd /usr/local/src/
dev# mkdir test_exceptions
dev# cd test_exceptions/
dev# wget http://codepoet.org/throw1.cpp
dev# cat throw1.cpp
#include <features.h>
#include "iostream"
int main(void)
{
    try {
        throw("This is an exception");
    }
    catch (...) {
        std::cout << "caught an exception\n";
    }
    return(0);
}
dev# g++ -Wall -O2 throw1.cpp -o throw1
dev# ldd throw1
        libstdc++.so.5 => /lib/libstdc++.so.5 (0xb7edf000)
        libm.so.0 => /lib/libm.so.0 (0xb7ed1000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7ec9000)
        libc.so.0 => /lib/libc.so.0 (0xb7e35000)
        ld-uClibc.so.0 => /lib/ld-uClibc.so.0 (0xb7f91000)
dev# ./throw1
Aborted
So even in such a trivial case the exception is not caught and process is aborted.
Now let's look at our gcc configuration:
dev# gcc -v
Using built-in specs.
Configured with: /opt/buildroot/toolchain_build_i386/gcc-3.3.6/configure --prefix=/usr --build=i386-pc-linux-gnu --host=i386-linux-uclibc 
--target=i386-linux-uclibc --enable-languages=c,c++,objc --enable-shared --with-gxx-include-dir=/usr/include/c++ --disable-__cxa_atexit 
--enable-target-optspace --with-gnu-ld --disable-nls --enable-multilib : 
(reconfigured) /opt/buildroot/toolchain_build_i386/gcc-3.3.6/configure --prefix=/usr --build=i386-pc-linux-gnu --host=i386-linux-uclibc 
--target=i386-linux-uclibc --enable-languages=c,c++,objc --enable-shared --with-gxx-include-dir=/usr/include/c++ --disable-__cxa_atexit 
--enable-target-optspace --with-gnu-ld --disable-nls --enable-multilib : 
(reconfigured) /home/joker/CR/opt/buildroot/toolchain_build_i386/gcc-3.3.6/configure --prefix=/usr --build=i386-pc-linux-gnu 
--host=i386-linux-uclibc --target=i386-linux-uclibc --enable-languages=c,c++,objc --enable-shared --with-gxx-include-dir=/usr/include/c++ 
--disable-__cxa_atexit --enable-target-optspace --with-gnu-ld --disable-nls --enable-multilib
Thread model: posix
gcc version 3.3.6
... it seems that it is not configured with --enable-sjlj-exceptions.
This message named "libgcc_s.so compatibility between 3.3 and 3.4 (sjlj/dwarf2 exceptions)" describes a problem with the change of default exception model. It seems that the gcc change management (so versioning) did not go very well at that point:
On m68k-linux and parisc-linux between 3.3 and 3.4 the default exception model changed from sjlj based exceptions to dw2 based exceptions. Unfortunately at this time the soversion number of the shared libgcc was not bumped. ... I didn't look at other architectures, if the distinction is needed as well.
Regarding to this message it seems that re-build of GCC with --enable-sjlj-exceptions could be a solution. The problem is that I have not prepared the uClibc build environment (I just used the one prebuilt by JoKeR) and so have no experience with building the GCC.
I did not want to start such a big task, so I decided to give another shot to a workaround:

Disable exceptions

The code in the src/deluge-torrent-0.5.9.3/libtorrent/src/entry.cpp file inspired me to this workaround:
#ifndef BOOST_NO_EXCEPTIONS
        const entry& entry::operator[](char const* key) const
        {
                dictionary_type::const_iterator i = dict().find(key);
                if (i == dict().end()) throw type_error(
                        (std::string("key not found: ") + key).c_str());
                return i->second;
        }

        const entry& entry::operator[](std::string const& key) const
        {
                return (*this)[key.c_str()];
        }
#endif
It seems that the libtorrent code is prepared for cases when there is no exception support in the platform (as is the case for some embedded systems for example). So why not try to disable the boost exceptions?
dev# cd /usr/local/src/boost_1_35_0
dev# nano boost/config/user.hpp
append the following at the end of the file:
#define BOOST_NO_EXCEPTIONS
... and now re-build the boost libraries:
dev# make
...
libs/serialization/src/xml_woarchive.cpp:58:   instantiated from here
boost/archive/iterators/wchar_from_mb.hpp:119: error: `dataflow_exception'
   undeclared in namespace `boost::archive::iterators'
...
...failed updating 9 targets...
...skipped 2 targets...
Not all Boost libraries built properly.
It seems not all the libraries are compatible with this define, anyway, let's proceed (finish the boost installation and restart the deluge build process):
dev# make install
...
dev# cd ../deluge-torrent-0.5.9.3
dev#  python setup.py clean
dev# python setup.py build
...
libtorrent/src/metadata_transfer.cpp: In member function `virtual bool
   libtorrent::<unnamed>::metadata_peer_plugin::on_extension_handshake(const
   libtorrent::entry&)':
libtorrent/src/metadata_transfer.cpp:275: error: passing `const
   libtorrent::entry' as `this' argument of `libtorrent::entry&
   libtorrent::entry::operator[](const char*)' discards qualifiers
error: command 'gcc' failed with exit status 1
In libtorrent/src/metadata_transfer.cpp there is a following code:
    virtual bool on_extension_handshake(entry const& h)
    {
            entry const& messages = h["m"];
            if (entry const* index = messages.find_key("LT_metadata"))
            {
                    m_message_index = int(index->integer());
                    return true;
            }
            else
            {
                    m_message_index = 0;
                    return false;
            }
    }
... in libtorrent/include/libtorrent/entry.hpp there are following map accessors:
                entry& operator[](char const* key);
                entry& operator[](std::string const& key);
#ifndef BOOST_NO_EXCEPTIONS
                const entry& operator[](char const* key) const;
                const entry& operator[](std::string const& key) const;
#endif
                entry* find_key(char const* key);
                entry const* find_key(char const* key) const;
                entry* find_key(std::string const& key);
the solution was to change the operator[] access to find_key() method call:
    virtual bool on_extension_handshake(entry const& h)
    {
//          entry const& messages = h["m"];
            entry const* messages( h.find_key("m") );
            if ( messages )
            {
                if (entry const* index = messages->find_key("LT_metadata"))
                {
                    m_message_index = int(index->integer());
                    return true;
                }
            }
//          else
            {
                    m_message_index = 0;
                    return false;
            }
    }
There is a lot of such places in the code, it seems that the library support for BOOST_NO_EXCEPTIONS directive is not fully implemented yet:
dev# python setup.py build
...
libtorrent/src/ut_pex.cpp: In member function `virtual bool
   libtorrent::<unnamed>::ut_pex_peer_plugin::on_extension_handshake(const
   libtorrent::entry&)':
libtorrent/src/ut_pex.cpp:201: error: passing `const libtorrent::entry' as
   `this' argument of `libtorrent::entry& libtorrent::entry::operator[](const
   char*)' discards qualifiers
error: command 'gcc' failed with exit status 1
I still did not feel ready to rebuild the GCC. In a desperation I looked to the deluge site and noticed that there is a new version out there - 1.0.0_RC3. It promised a change in my miserable situation.

Deluge new version - Disable exceptions

So let's upgrade:
dev# cd /usr/local/src/
dev# wget http://download.deluge-torrent.org/source/0.9.03/deluge-0.9.03.tar.gz
dev# tar xzvf deluge-0.9.03.tar.gz
dev# cd deluge-0.9.03
dev# python setup.py build
...
In file included from /mnt/C/sys/include/boost-1_35/boost/thread.hpp:17,
                 from libtorrent/include/libtorrent/storage.hpp:44,
                 from libtorrent/include/libtorrent/peer_connection.hpp:63,
                 from libtorrent/src/peer_connection.cpp:41:
/mnt/C/sys/include/boost-1_35/boost/thread/recursive_mutex.hpp:18:2: #error "Boost threads unavailable on this platform"
...
We already have a solution for this:
dev# export CFLAGS="-pthread $CFLAGS "
Then there is another build error:
dev# python setup.py build
...
libtorrent/src/torrent_handle.cpp: In member function `const
   libtorrent::torrent_info& libtorrent::torrent_handle::get_torrent_info()
   const':
libtorrent/src/torrent_handle.cpp:503: error: no matching function for call to
   `libtorrent::torrent_info::torrent_info()'
libtorrent/include/libtorrent/torrent_info.hpp:86: error: candidates are:
   libtorrent::torrent_info::torrent_info(const libtorrent::torrent_info&)
libtorrent/include/libtorrent/torrent_info.hpp:130: error:
   libtorrent::torrent_info::torrent_info(const libtorrent::entry&)
libtorrent/include/libtorrent/torrent_info.hpp:92: error:
   libtorrent::torrent_info::torrent_info(const boost::filesystem::path&)
libtorrent/include/libtorrent/torrent_info.hpp:91: error:
   libtorrent::torrent_info::torrent_info(const char*, int)
libtorrent/include/libtorrent/torrent_info.hpp:90: error:
   libtorrent::torrent_info::torrent_info(const libtorrent::lazy_entry&)
libtorrent/include/libtorrent/torrent_info.hpp:89: error:
   libtorrent::torrent_info::torrent_info(const libtorrent::sha1_hash&)
libtorrent/src/torrent_handle.cpp: In member function `libtorrent::entry
   libtorrent::torrent_handle::write_resume_data() const':
libtorrent/src/torrent_handle.cpp:533: error: return-statement with no value,
   in function declared with a non-void return type
error: command 'gcc' failed with exit status 1
When we look at libtorrent/src/torrent_handle.cpp:
        torrent_info const& torrent_handle::get_torrent_info() const
        {
                INVARIANT_CHECK;
#ifdef BOOST_NO_EXCEPTIONS
                const static torrent_info empty;
#endif
                boost::shared_ptr<torrent> t = m_torrent.lock();
        ...
... it is obvious that there the torrent_info has no default constructor, but in case of defined BOOST_NO_EXCEPTIONS such one is used. So I tried to add one:
libtorrent/include/libtorrent/torrent_info.hpp:
#ifdef BOOST_NO_EXCEPTIONS
 torrent_info();
#endif
    torrent_info(sha1_hash const& info_hash);
    torrent_info(lazy_entry const& torrent_file);
    torrent_info(char const* buffer, int size);
    torrent_info(fs::path const& filename);
    ~torrent_info();
libtorrent/src/torrent_info.cpp:
#ifdef BOOST_NO_EXCEPTIONS
        torrent_info::torrent_info()
                : m_creation_date(pt::second_clock::universal_time())
                , m_multifile(false)
                , m_private(false)
                , m_info_section_size(0)
                , m_piece_hashes(0)
        {}
#endif
Another build error and solution was following:
dev# python setup.py build
...
libtorrent/src/torrent_handle.cpp: In member function `libtorrent::entry
   libtorrent::torrent_handle::write_resume_data() const':
libtorrent/src/torrent_handle.cpp:533: error: return-statement with no value,
   in function declared with a non-void return type
error: command 'gcc' failed with exit status 1
Let's look to the libtorrent/src/torrent_handle.cpp file:
        entry torrent_handle::write_resume_data() const
        {
                INVARIANT_CHECK;

                entry ret(entry::dictionary_t);
                TORRENT_FORWARD(write_resume_data(ret));
                t->filesystem().write_resume_data(ret);

                return ret;
        }
When we look at the definition of the macro in the same file:
#ifdef BOOST_NO_EXCEPTIONS

#define TORRENT_FORWARD(call) \
        boost::shared_ptr<torrent> t = m_torrent.lock(); \
        if (!t) return; \
        session_impl::mutex_t::scoped_lock l(t->session().m_mutex); \
        t->call
...
#else

#define TORRENT_FORWARD(call) \
        boost::shared_ptr<torrent> t = m_torrent.lock(); \
        if (!t) throw_invalid_handle(); \
        session_impl::mutex_t::scoped_lock l(t->session().m_mutex); \
        t->call
...
... we see that this version of macro can be used only in function returning void. The solution is use another variant of the macro:
        TORRENT_FORWARD_RETURN2(write_resume_data(ret), entry() );
Let's proceed:
dev# python setup.py build
...
libtorrent/src/memdebug.cpp:34:22: execinfo.h: No such file or directory
libtorrent/src/memdebug.cpp: In constructor `memdebug::memdebug()':
libtorrent/src/memdebug.cpp:61: error: `__malloc_hook' undeclared (first use
   this function)
libtorrent/src/memdebug.cpp:61: error: (Each undeclared identifier is reported
   only once for each function it appears in.)
libtorrent/src/memdebug.cpp:62: error: `__free_hook' undeclared (first use this
   function)
libtorrent/src/memdebug.cpp: In static member function `static void*
   memdebug::my_malloc_hook(unsigned int, const void*)':
libtorrent/src/memdebug.cpp:133: error: `backtrace' undeclared (first use this
   function)
libtorrent/src/memdebug.cpp:144: error: `backtrace_symbols' undeclared (first
   use this function)
error: command 'gcc' failed with exit status 1
This kind of error is not connected with BOOST_NO_EXCEPTIONS. It is due to the missing backtrace support in uClibc.
Fortunately it seems that the memdebug.cpp is used just for memory debugging (prevention of memory leaks and so), this file is self contained and there are no outside dependencies on this file. Let's try simply disable it (simply undefine or clear the file content):
#if 0
#if defined __linux__ && defined __GNUC__
...
#endif
#endif
Here is another BOOST_NO_EXCEPTIONS related build error:
dev# python setup.py build
...
libtorrent/src/http_connection.cpp:208:   instantiated from here
libtorrent/include/libtorrent/variant_stream.hpp:226: error: no matching
   function for call to `libtorrent::ssl_stream<libtorrent::socket_type>::close
   ()'
libtorrent/include/libtorrent/ssl_stream.hpp:158: error: candidates are: void
   libtorrent::ssl_stream<Stream>::close(boost::system::error_code&) [with
   Stream = libtorrent::socket_type]
error: command 'gcc' failed with exit status 1
As we see in the libtorrent/include/libtorrent/ssl_stream.hpp the ssl_stream class contains two overloads of close() method, the non-parametric one disabled in case there are no exceptions:
#ifndef BOOST_NO_EXCEPTIONS
        void close()
        {
                m_sock.next_layer().close();
        }
#endif

        void close(error_code& ec)
        {
                m_sock.next_layer().close(ec);
        }
In libtorrent/include/libtorrent/variant_stream.hpp there are two kinds of close visitor class: close_visitor (one used with exceptions) and close_visitor (one used with error codes), but the "exception" variant is not disabled, let's try to fix it:
 struct close_visitor_ec
    : boost::static_visitor<>
  {
      close_visitor_ec(error_code& ec_)
        : ec(ec_)
      {}

      template <class T>
      void operator()(T* p) const
      { p->close(ec); }

      void operator()(boost::blank) const {}

      error_code& ec;
  };

#ifndef BOOST_NO_EXCEPTIONS
  struct close_visitor
    : boost::static_visitor<>
  {
      template <class T>
      void operator()(T* p) const
      { p->close(); }

      void operator()(boost::blank) const {}
  };
#endif
When we try to rebuild, we can see the place where the inappropriate visitor is called:
dev# python setup.py build
...
libtorrent/include/libtorrent/variant_stream.hpp:665: error: `close_visitor'
   undeclared in namespace `libtorrent::aux'
error: command 'gcc' failed with exit status 1
We have to somewhat fix the libtorrent/include/libtorrent/variant_stream.hpp. There are basically two possibilities: (a) undefine the non-parametric version or (b)(maybe less correctly) use the error code variant and drop the error_code.
... as a quick fix I decided to use the latter method:
    void close(error_code& ec)
    {
        if (!instantiated()) return;
        boost::apply_visitor(
            aux::close_visitor_ec(ec), m_variant
        );
    }

// #ifndef BOOST_NO_EXCEPTIONS   // (1st - more correct - method)
    void close()
    {
//        if (!instantiated()) return;
//        boost::apply_visitor(aux::close_visitor(), m_variant);
        error_code ec;              // 2nd - simpler - method
        close( ec );
    }
// #endif                        // (1st - more correct - method) 
Another build problem (with already known solution) followed - undeclared IPV6_V6ONLY; so I have modified the libtorrent/include/libtorrent/socket.hpp and added the dummy IPV6_V6ONLY definition.

Then there was a bunch of errors with dictionary and lack of non-mutable operator[] access in case of BOOST_NO_EXCEPTIONS (similar to the previous version). I decided to solve it with a helper function called get_subentry_const implementing the missing functionality:
The libtorrent/src/kademlia/dht_tracker.cpp file:
        libtorrent::entry const& get_subentry_const( libtorrent::entry const& e, char const* key )
        {
#ifndef BOOST_NO_EXCEPTIONS
            return e[key];
#else
            libtorrent::entry const* const s( e.find_key( key ) );
            if (!s) throw std::runtime_error("subentry not found");
            return *s;
#endif
        }
This function - in case of missing key - throws the exception - which in turn cause the process termination, the question is whether the missing key in client code is really exceptional or it is used in normal program flow... Alternative would be to return reference to a global empty instance of entry class.

Now the client code has to be modified as follows:
//     std::string const& id = r["id"].string();
       std::string const& id = get_subentry_const( r, "id" ).string();
With all these changes the compilation completed succesfully but I got the following link errors:
dev# python setup.py build
...
`.L7753' referenced in section `.rodata' of build/temp.linux-i686-2.5/./libtorrent/src/create_torrent.o: defined in d
iscarded section `.gnu.linkonce.t._ZN5boost9date_time23gregorian_calendar_baseINS0_19year_month_day_baseINS_9gregoria
n9greg_yearENS3_10greg_monthENS3_8greg_dayEEEmE16end_of_month_dayES4_S5_' of build/temp.linux-i686-2.5/./libtorrent/s
rc/create_torrent.o
`.L7740' referenced in section `.rodata' of build/temp.linux-i686-2.5/./libtorrent/src/create_torrent.o: defined in d
iscarded section `.gnu.linkonce.t._ZN5boost9date_time23gregorian_calendar_baseINS0_19year_month_day_baseINS_9gregoria
n9greg_yearENS3_10greg_monthENS3_8greg_dayEEEmE16end_of_month_dayES4_S5_' of build/temp.linux-i686-2.5/./libtorrent/s
rc/create_torrent.o
...
collect2: ld returned 1 exit status
error: command 'gcc' failed with exit status 1
As you can see in this bug report it seems as a known compiler bug.
There was one promising hint in this discussion:
While trying to build some C++ code with g++-3.3.6 we encountered error
messages  similar to those mentioned in earlier comments in this discussion:

  `xxx' referenced in section `.rodata' of somefile.o: defined 
  in discarded section `.gnu.linkonce.t._zzz' of something.o

  ...

We were able to eliminate these error messages and successfuly compile our code
with g++-3.3.6 by using the '-frepo' option in our g++ compiles.

I understand that this may not be the ultimate solution to the bug(s) being
discussed here, but since we ended up here when searching for this particular
error message, I'm adding this comment in case it helps other people.
Nevertheless when I tried the suggested workaround:
dev# export CFLAGS="$CFLAGS -frepo"
... the result was exactly the same.

As you could see, I tried really hard to avoid the re-configuration and re-compilation of the compiler but at the end it seems inevitable.
But it is enough for now, I feel really tired ;-)
We shall look at that in the next part.

Note: I have created a ticket on libtorrent Trac; it contains the diff with all the changes I have done.

No comments: