Unidev Tech Blog

Posts Tagged ‘debugging a program’

While debugging a program that had dumped core on HP-UX, I noticed some oddities in the stack trace the required a bit of further investigation. As the code I was looking at is client code, I cannot share it online. This post shows analysis done under CentOS Linux of a trivial program I constructed with the same issue.

For those of you who have not used GDB to analyze a core dump, it’s simple to do. Simply invoke GDB with the following syntax:

    gdb /path/to/executable core.file

Once you’re inside of GDB, you can use the commands on the stack frame, just as you would for any other program you were debugging; you can perform a backtrace to get the stack, navigate up and down through the stack frames, and print variables to the screen.

It was while looking through the backtrace and stack frames, I noticed something was wrong – the stack frames seemed corrupted after as it went further down:

     gdb> bt

This had me stumped for a little while. The memory location for the stack frames starting with #7 was simply nonsense – it was nowhere near the location for the other stack frames and was well out of the range of memory that I had on the box. After looking through source code (see Code Listing 1) and variables for what seemed like hours, the underlying problem suddenly came to me. In the checkLinesForErrors function,  the program was appending an error message. The input file causing the program to crash had about 10 errors in it. The error message was about 60 bytes long. The error buffer could only hold 400 bytes.

The problem was the strcat function was overflowing the error buffer.  Since this had been allocated further up the stack and was being passed down, the overflow went downward, overwriting into the next stack frame. The program had crashed after this overflow occurred – it dumped core because the memory references by one of the C++ strings had been overwritten, which caused a segmentation fault when one of those pointers was being accessed.

In fact, after detecting this, I started looking at the stack frame memory addresses and they looked a little peculiar. If you convert them to their ASCII equivalents, they become part of the error message that was being written to the buffer. For example, the stack frame 8’s address is “0×6863206563617073” is “hc ecaps” when converted to ASCII, and  “space ch” when that string is reversed. This is part of the ERROR_MESSAGE string that was appended into errorBuffer.

The fix was quite easy – add a length variable and some bounds checking as shown in code listing 2. One gotcha in the fix was subtracting the 1 byte for the null character, since the buffer is a C-Style String. It turned out that this error had around for quite a while, but went undetected because a typical run of this program would only append one or two records to this error message.

To sum things up, buffer overflows can be hard to detect. This was especially true in this case when looking through the core file. If you’re looking one that it appears that the data for the stack frames makes absolutely no sense, a buffer overflow like this one could be the cause of the problem.

Code Listing 1 – Original Code:

#define ERROR_MESSAGE “The specified line is not valid: It contains whitespace characters. “

bool checkDataFile(vector<string> dataLines)

{

char errorBuffer[400];

errorBuffer[0]=’\0′;

int errorCount;

if (!processDataFileLines(dataLines, errorBuffer))

{

cerr << “Errors occurred in processing” << endl;

cerr << errorBuffer << endl;

return false;

}

return true;

}

bool processDataFileLines(vector<string> dataLines, char * errorBuffer)

{

int errorCount = 0;

bool rc = checkLinesForErrors(dataLines, errorBuffer);

return rc;

}

bool checkLinesForErrors(vector<string> dataLines, char * errorBuffer)

{

int errorCount = 0;

for (unsigned int i=0; i<dataLines.size(); i++)

{

if (dataLines[i].find_first_of(” \r\n\t”)!=string::npos)

{

strcat(errorBuffer, ERROR_MESSAGE);

errorCount++;

}

}

return (errorCount == 0);

}

Code Listing 2 – Corrected Code:

bool checkDataFile(vector<string> dataLines)

{

char errorBuffer[400];

errorBuffer[0]=’\0′;

int errorCount;

processDataFileLines(dataLines, errorBuffer, sizeof(errorBuffer)-1);

if (!processDataFileLines(dataLines, errorBuffer, sizeof(errorBuffer)-1))

{

cerr << “Errors occurred in processing” << endl;

cerr << errorBuffer << endl;

return false;

}

return true;

}

bool processDataFileLines(vector<string> dataLines, char * errorBuffer, unsigned int errorBufferLength)

{

int errorCount = 0;

bool rc = checkLinesForErrors(dataLines, errorBuffer, errorBufferLength);

return rc;

}

bool checkLinesForErrors(vector<string> dataLines, char * errorBuffer, unsigned int errorBufferLength)

{

const char * message_prefix=foo;

int errorCount = 0;

for (unsigned int i=0; i<dataLines.size(); i++)

{

if (dataLines[i].find_first_of(” \r\n\t”)!=string::npos)

{

strncat(errorBuffer, ERROR_MESSAGE, errorBufferLength – strlen(errorBuffer));

errorCount++;

}

}

return (errorCount == 0);

}