Instrumenting GridFTP 2.4.0 with NetLogger 2.0.13

____________________________________________________________________________________________________________________________

_____________________________________________________________________________________________________________________________

 

In order to instrument GridFTP 2.4.0 with NetLogger 2.0.13, I first tried to use the same method as the one described to instrument GridFTP 2.0 with NetLogger 2.0.13.

Updating the Makefiles and scripting xinetd has to be done in the same way. But there were quite a few modifications to the files globus_io_read.c, globus_io_write.c and globus_hack.c between Globus Toolkit 2.0 and Globus Toolkit 2.4.0. Hence a straightforward replacement of these files was not sufficient.

I tried to add code to define BT_NETLOGGER_HACK in the corresponding files in the Globus Software but soon realised that in Globus Toolkit 2.4.0, NetLogger can be enabled by defining GLOBUS_BUILD_WITH_NETLOGGER in the following files:

By doing this when I ran Globus GridFTP and the NetLogger daemon netlogd together and started doing a transfer, at the start of the transfer netlogd would recognise a connection is being made and it would also give details from where the connection is being made. But then the transfer fails and the GridFTP server stops completely:

> netlogd.py -p 5677 -o /tmp/netlogd.out


NetLogger daemon: port 5677, output in ['/tmp/netlogd.out']
+ accepted new connection from 192.91.239.5:32812


Traceback (most recent call last):
  File "/usr/local/bin/netlogd.py", line 253, in ?
    if __name__ == '__main__': main(sys.argv)
  File "/usr/local/bin/netlogd.py", line 248, in main
    g_main_loop.runLoop()
  File "/usr/local/bin/netlogd.py", line 170, in runLoop
    reader = NetLoggerReader(sock.fileno())
  File "/usr/lib/python2.2/site-packages/netlogger.py", line 379, in
__init__
    raise NetLoggerException("read-open")
netlogger.NetLoggerException: Error: read-open. Backtrace:
Error: nl_read_stream_format: timed out
Error: NetLoggerReadOpen: read stream format
 

This probably means that just defining GLOBUS_BUILD_WITH_NETLOGGER is not sufficient to port NetLogger 2.0.13 to Globus Toolkit 2.4.0.

After looking into the GridFTP code more carefully, I found out that  NetLogger version 2.0.13 does not contain the file NetLogger.h which is required by GridFTP 2.4.0. I contacted the developers of NetLogger again and found out that NetLogger.h has to be obtained by downloading NetLogger-lite. Once I downloaded NetLogger-lite and unpacked it, I modified the Makefiles of the following directories of GridFTP to include NetLogger-lite:

So the NetLogger include part of Makefile.am in these directories now looks like:

NETLOGGER_DIR=/opt/netlogger/build/include/
NETLOGGER_LITE_DIR=/opt/NetLogger-lite/

NETLOGGER_INCLUDES=\
$(NETLOGGER_DIR)/netlogger.h\
$(NETLOGGER_DIR)/nl.h\
$(NETLOGGER_DIR)/all_the_other_netloggerinclude_files_in_this_directory\

$(NETLOGGER_LITE_DIR/NetLogger.h

NETLOGGER_LIBS=/opt/netlogger/build/lib/libnetlogger.a

INCLUDES = $(GPT_INCLUDES) $(NETLOGGER_INCLUDES)


libglobus_io___GLOBUS_FLAVOR_NAME__la_LIBADD = @GPT_LIB_LINKS@ $(NETLOGGER_LIBS)

 

After modifying the Makefiles I tried rebuilding GridFTP, but I am getting shared library error while trying to do so. Currently I am looking into this problem in detail.