some problems when commands are hashed

In Unix systems, commands are hashed for performance reasons [man hash]. Today, I got a problem related to this and wasted so much time for resolving. In my solaris machine, patch command is in /usr/bin/patch and its version is very old. So, I have installed a new patch version using pkgadd and it has installed patch command in /usr/local/bin/. In PATH variable, /usr/local/bin/ is before /usr/bin directory.  So, `which patch` command is showing /usr/local/bin/patch. But, when I execute patch on command prompt, it is executing old version of patch [/usr/bin/patch]. After some time, I tried type command `type patch` and it is showing /usr/bin/patch is hashed. Then, I came to know about hash command in solaris. After rehashing, every thing started working fine.

Gmake going in loops??

From many days, I have observed that our build sometimes goes into recursive loops and does not get out of the loop for many hours/days. I found the reason for this problem. It is about the modification time of makefile & C/C++ files. I use tar.gz which preserve file modification times and transfer that tar.gz to other systems. If those systems have different timezone, it might happen that the makefile/c/c++ files are modified in the future. Since gmake considers file modification times when running makefile, it goes in recursive loop. Simple solution is to touch the makefile/c/c++ files with future modification time using touch command.

Do we need __PRETTY_FUNCTION__ like macros in other compilers?

You might know that GCC in addition to __func__ macro, it supports __FUNCTION__ and __PRETTY_FUNCTION__. __func__ is the predefined-macro which contains the function name in a string. It is defined in C specs, But, __FUNCTION__ and __PRETY_FUNCTION__ are extensions by GCC. __FUNCTION__ macro is same as __func__. __PRETTY_FUNCTION__ macro gives the full function declaration including parameters and return values. I tried to look for other alternatives on SUNWspro compilers and HP-C/C++ but could not find any. Then I questioned myself ‘do we need this macro while we can always get sufficient debug information with __LINE__ , __FILE__, __func__??’.  Answer is ‘No’. We can always get sufficient debug information with the macros defined in specs. More over, if we use GCC extensions in code, portability to other platforms is a big pain. BTW, VS.net compiler also has a similar pre-defined macro for getting decorated function name.

My experience with using GNU screen

In my B.Tech, we were taught about using screen command. But at that time, I did not realise the importance of that. Because, I used to open 1 connection to server, write code/compile/execute in that terminal only. But, now I open more than 5 connections. Last week, I read a small tutorial about screen and felt the importance of it. Screen is used for terminal multiplexing. We open only one terminal connection to server. In that terminal, we can use screen to create multiple virtual windows. Since we can detach/attach screens, we can also use it as remote desktop for linux.

In linux, screen is already installed. So, I have started using it directly. In my solaris machine, it is not installed. So, I have downloaded its package from sunfreeware site and installed it. When I started screen, I was not able to use backspace on command line. I could not find any clues on net. But, when I connected to that machine using ssh, that problem was solved. After that when I opened a file using vim in screen, I was not able to use arrow keys. Colors were also not according to default vim colorscheme. I have searched on internet.. but could not find any clues. I knew, it was the problem with TERM & TERMCAP. So, I have exited screen and started it again with TERM=screen screen. Then, I could use arrowkeys and colors were also fine. But at some places, vim was not showing code properly. Again.. I wasted lot of time for searching on the net. After getting frustrated with putty and terminal types, I connected to that solaris machine from my linux box and started screen with the same command [TERM=screen screen]. Luckily, then, everything in screen was working fine. Still, I dont know why I was getting problems if I directly connect to solaris from putty and use screen.

Seeing pre-processor macros in gdb

In gdb, we can see the source code while debugging. But, if there is any macro in the source code which is defined in some other file, then we have to search for that macro outside gdb [using fgrep] and see what it expands to. In complex codes like openssl, it is even difficult to find/understand macros. When I was searching for this kind of feature in gdb, I saw this link. Then, I came to know that this feature is already implemented. After further research on this topic, I read this link. In this link, it says, if we build our source code with -g3 -gdwarf-2 , it will include all the macros in the debug information. So, I have written a test program and built it with those options. After that, in gdb, I have tried $info macro <Macro.Name>, it gave the line no# where the macro is defined and full macro definition. In addition to just seeing macros, we can use ‘macro expand’ command to see what the macro expands to.

One interesting note: I tested a small program and it worked in my SuSE8.1, GCC3.2.2, GDB5.3. I have tried to test the same thing in Fedora-core3, GCC3.4.2, GDB6.1, it did not work. I think it is the problem with GDB [though I am not sure]. GDB needs to support these debugging formats. So, if we build GDB with proper flags, I think it will work.

I read that there are better debug formats than dwarf-2, one among them is stabs. But, once again, your debugger needs to support it.

Stack traces in C/C++ programs in solaris & linux

In Java, there are direct APIs for getting stack traces. But, for C, there are no popular APIs for doing the same. So, I was just searching for this feature in linux. Then I came across this link. It provides very clean and simple interface for getting stack traces. They even gave the sample program in the end of the article, demonstrating how to use it. Note: We have to compile the program with -rdynamic flag. In solaris, I know, with dtrace tool, we can do all these tricks. But, if I want to get/use the stack trace inside the C program, dtrace doesn’t help. After searching for “stack” in /usr/include/*.h contents, I found some functions in ucontext.h, which are relevant to what I want. Then, after seeing man page of one of their functions [man printstack], it is confirmed. I have written a small program to demonstrate stack traces in C program. You can download it here. For C++ programs, we may have to use c++filt [like ./a.out | c++filt] to get the correct functions names from mangled ones.

Magic with memusage library in linux

Today, I have started writing a tool for finding memory leaks in my programs. So, I have started with defining my own memory functions like malloc, realloc, free with exactly same signature as of the original memory functions. In these functions, I explicitly load original memory libraries, call the original functions and I keep note of how many times each of them is called. So, if I wanted to find memory leaks in a program test, instead of executing it like $./test, I would execute it like $LD_PRELOAD=$PWD/libmemleaks.so ./test.

By mistake, I was explicitly loading /lib/libmemusage.so instead of libc.so. Initially I got so many compilation errors in my libmemleaks.so program. So, I removed lot of code and compiled it again succesfully. But, when I executed it finally, I got a table with heap/stack size and no.of times malloc/calloc/free are called. The table is shown below.memory_report

Surprised!! I was not expecting this multi-colored table. I did not write any code for printing this table. In yahoo/google, I searched and did not get any references to this problem [no documentation, problem with open source??]. Then I have used google code search and searched for “Memory usage summary” and got a link to glibc-2.2.5/malloc/memusage.c. After seeing that file, in _fini function, I understood that, they are writing the memory usage report. _fini is the function in shared library which is called, when the system is unloading the library.

After that, I have download glibc 2.2.5 source code and started looking for the clues. In the same directory as glibc-2.2.5/malloc, I found a shell script memusage.sh file. I have copied that into my linux machine and modified a line and executed it [sh memusage.sh ./test]. Then, I got the same multi colored table. Then I came to know, this memusage.so library is written for getting memory stats. memusage.sh, internally executes like LD_PRELOAD= /lib/libmemusage.so <prog.name>. Even I thought of writing my memory leak checking tool with the same concept and at last I have completed my program. But, my program is not as colorful as memusage.sh program. Using memusage.sh, we can find memory stats of any program [including java programs]. Execute sh memusage.sh –help for command help.

Normally, linux distributions dont distribute memusage.sh. You can see the same shell script here. you can download it from here [I have modified at 2 places. Replaced @SLIBDIR@ , @BINDIR@ with /lib, /bin. To get graphs, you need to have memusagestat executable in /bin]. Finally, I am very happy that I found this utility after researching on this for 3 1/2 days.

Multi line greps

    In linux/solaris, we can use fgrep/grep for pattern matching in files. But one limitation with these commands is they limit their pattern matching to one line. They don’t search for the pattern in multiple lines. Then I found an open source tool, pcregrep [available as sdk also]. pcregrep provides multi-line grep. In many latest versions of linux and solaris machines, I have found this command, but they are older versions [maybe stable versions]. Older versions did not have multi-line grep functionality. So, I have downloaded its latest source and built it.

One usecase: I have function names, then I need to find out their declarations from header files. Function could be declared over many lines. This is my usecase. In this scenario, I have used pcregrep.

Version scripts & Function aliasing

    In windows, there is a problem called “DLL HELL” which has been there for many years. I heard that, Vista is going to provide a complete solution to this problem. Unix guys thought about this problem long back and provided solution through “Version scripts” [First implemented in SunOS 2.5 and later also implemented in GCC and linux]. Through version scripts, we can simply maintain versions in shared libraries. When we link an application against a shared library that has versioned symbols, the application itself knows which version of each symbol it requires, and it also knows which version nodes it needs from each shared library it is linked against. Thus at runtime, the dynamic loader can make a quick check to make sure that the libraries you have linked against do in fact supply all of the version nodes that the application will need to resolve all of the dynamic symbols.

Simple version script looks like the following

	VERS_1.1 {
		 global:
			 foo1;
		 local:
			 old*;
			 original*;
			 new*;
		};
	VERS_1.2 {
		 foo2;
	} VERS_1.1;

More info on version scripts is given better in the reference links mentioned in the end.

GCC has added an extension to this and provided an excellent feature for function aliasing. We can do this by adding assember macro like the following in C code.

		__asm__(".symver old_foo,foo@VERS_1.1");
		__asm__(".symver old_foo1,foo@VERS_1.2");
		__asm__(".symver new_foo,foo@@VERS_2.0");

So, for the executables linked with VERS_1.2, a function call to foo will be redirected to  old_foo1 internally.  Similarly for executables linked with VERS_1.1,  function call to foo will be redirected internally to old_foo.

You can find more description about these at

http://people.freebsd.org/~deischen/symver/library_versioning.txt

http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/gnu-linker/version.html

http://snipurl.com/wavf   [This original URL is too long.. so snipped it]

Make a shared library which can be executable in linux

    Great day!! Finally I came to know, how to create a file which can act as both shared library and executable. You might not have observed it in your unix machines. In linux, execute /lib/libc.so as a command, it gives you all the version information. I have been trying to simulate the same feature in my shared libraries also. I have seen glibc source code and got to know that we have to use entry option of linker. With that knowledge, I have written a small program and tried for a complete working day, with the result ending up in Illegal instructions/ segmentation faults. Then I have just sent a mail to gcc mailing list and got the answers 1 ,2 ,3.

I am attaching a simple 5 line c program here. Commands that I have used are

# gcc -g -W -Wall -fPIC -o libtest.so -shared -Wl,-e,test1 test.c

    #./ libtest.so

    Hi dp! you finally made it

    # 

In our C program, we have to declare a char array for .interp section to hold the absolute path of the linker. In our entry function test1, after processing the logic, we should use exit() function instead of returning. I dont know the logic behind these 2 points.

In Solaris also, we can have this feature. But, I have not tried/tested it. If you are interested, you can look at this. Read the whole page and see the comment named “solaris 9 compile”.

« Older entries