Using extern in c and machine endianness

declaration and definition of a variable/function

Declaration of a variable/function simply declares that the variable/function exists somewhere in the program but the memory is not allocated for them. But the declaration of a variable/function serves an important role. And that is the type of the variable/function. Therefore, when a variable is declared, the program knows the data type of that variable. In case of function declaration, the program knows what are the arguments to that functions, their data types, the order of arguments and the return type of the function. So that's all about declaration. Coming to the definition, when we define a variable/function, apart from the role of declaration, it also allocates memory for that variable/function. Therefore, we can think of definition as a super set of declaration. (or declaration as a subset of definition). From this explanation, it should be obvious that a variable/function can be declared any number of times but it can be defined only once. (Remember the basic principle that you can't have two locations of the same variable/function). So that's all about declaration and definition.

Understanding "extern" keyword in C

it's mandatory to understand declaration/defination to understand the "extern" keyword. Let us first take the easy case. Use of extern with C functions. By default, the declaration and definition of a C function have "extern" prepended with them. It means even though we don't use extern with the declaration/definition of C functions, it is present there. For example, when we write.

int foo(int arg1, char arg2);

There's an extern present in the beginning which is hidden and the compiler treats it as below.

extern int foo(int arg1, char arg2);

Same is the case with the definition of a C function (Definition of a C function means writing the body of the function). Therefore whenever we define a C function, an extern is present there in the beginning of the function definition. Since the declaration can be done any number of times and definition can be done only once, we can notice that declaration of a function can be added in several C/H files or in a single C/H file several times. But we notice the actual definition of the function only once (i.e. in one file only). And as the extern extends the visibility to the whole program, the functions can be used (called) anywhere in any of the files of the whole program provided the declaration of the function is known. (By knowing the declaration of the function, C compiler knows that the definition of the function exists and it goes ahead to compile the program). So that's all about extern with C functions.

Now let us the take the second and final case i.e. use of extern with C variables. I feel that it more interesting and information than the previous case where extern is present by default with C functions. So let me ask the question, how would you declare a C variable without defining it? Many of you would see it trivial but it's important question to understand extern with C variables. The answer goes as follows.

extern int var;

Here, an integer type variable called var has been declared (remember no definition i.e. no memory allocation for var so far). And we can do this declaration as many times as needed. (remember that declaration can be done any number of times) So far so good.

Now how would you define a variable. Now I agree that it is the most trivial question in programming and the answer is as follows.

int var;

Here, an integer type variable called var has been declared as well as defined. (remember that definition is the super set of declaration). Here the memory for var is also allocated. Now here comes the surprise, when we declared/defined a C function, we saw that an extern was present by default. While defining a function, we can prepend it with extern without any issues. But it is not the case with C variables. If we put the presence of extern in variable as default then the memory for them will not be allocated ever, they will be declared only. Therefore, we put extern explicitly for C variables when we want to declare them without defining them. Also, as the extern extends the visibility to the whole program, by externing a variable we can use the variables anywhere in the program provided we know the declaration of them and the variable is defined somewhere.

Now let us try to understand extern with examples.

Example 1:
int var;
int main(void)
{
var = 10;
return 0;
}

Analysis: This program is compiled successfully. Here var is defined (and declared implicitly) globally.

Example 2:
extern int var;
int main(void)
{
return 0;
}

Analysis: This program is compiled successfully. Here var is declared only. Notice var is never used so no problems.

Example 3:
extern int var;
int main(void)
{
var = 10;
return 0;
}

Analysis: This program throws error in compilation. Because var is declared but not defined anywhere. Essentially, the var isn't allocated any memory. And the program is trying to change the value to 10 of a variable that doesn't exist at all.

Example 4:
#include "somefile.h"
extern int var;
int main(void)
{
var = 10;
return 0;
}

Analysis: Supposing that somefile.h has the definition of var. This program will be compiled successfully.

Example 5:
extern int var = 0;
int main(void)
{
var = 10;
return 0;
}

Analysis: Guess this program will work? Well, here comes another surprise from C standards. They say that..if a variable is only declared and an initializer is also provided with that declaration, then the memory for that variable will be allocated i.e. that variable will be considered as defined. Therefore, as per the C standard, this program will compile successfully and work.

So that was a preliminary look at "extern" keyword in C.

I'm sure that you want to have some take away from the reading of this post. And I would not disappoint you.
In short, we can say

1. Declaration can be done any number of times but definition only once.
2. "extern" keyword is used to extend the visibility of variables/functions().
3. Since functions are visible through out the program by default. The use of extern is not needed in function declaration/definition. Its use is redundant.
4. When extern is used with a variable, it's only declared not defined.
5. As an exception, when an extern variable is declared with initialization, it is taken as definition of the variable as well.

Mystery of Bigendian and Littleendian

What are these?

Little and big endian are two ways of storing multibyte data-types ( int, float, etc). In little endian machines, last byte of binary representation of the multibyte data-type is stored first. On the other hand, in big endian machines, first byte of binary representation of the multibyte data-type is stored last.

Suppose integer is stored as 4 bytes (For those who are using DOS based compilers such as C++ 3.0 , integer is 2 bytes) then a variable x with value 0×01234567 will be stored as following.

How to see memory representation of multibyte data types on your machine?

Here is a sample C code that shows the byte representation of int, float and pointer.
#include

/* function to show bytes in memory, from location start to start+n*/
void show_mem_rep(char *start, int n)
{
int i;
for (i = 0; i < n; i++)
printf(" %.2x", start[i]);
printf("\n");
}

/*Main function to call above function for 0×01234567*/
int main()
{
int i = 0×01234567;
show_mem_rep((char *)&i, sizeof(i));
getchar();
return 0;
}

When above program is run on little endian machine, gives "67 45 23 01″ as output , while if it is run on endian machine, gives "01 23 45 67″ as output.

Is there a quick way to determine endianness of your machine?

There are n no. of ways for determining endianness of your machine. Here is one quick way of doing the same.
#include
int main()
{
unsigned int i = 1;
char *c = (char*)&i;
if (*c)
printf("Little endian");
else
printf("Big endian");
getchar();
return 0;
}

In the above program, a character pointer c is pointing to an integer i. Since size of character is 1 byte when the character pointer is de-referenced it will contain only first byte of integer. If machine is little endian then *c will be 1 (because last byte is stored first) and if machine is big endian then *c will be 0.

Does endianness matter for programmers?

Most of the times compiler takes care of endianness, however, endianness becomes an issue in following cases.

It matters in network programming: Suppose you write integers to file on a little endian machine and you transfer this file to a big endian machine. Unless there is little andian to big endian transformation, big endian machine will read the file in reverse order. You can find such a practical example here.

Standard byte order for networks is big endian, also known as network byte order. Before transferring data on network, data is first converted to network byte order (big endian).

Sometimes it matters when you are using type casting, below program is an example.
#include
int main()
{
unsigned char arr[2] = {0×01, 0×00};
unsigned short int x = *(unsigned short int *) arr;
printf("%d", x);
getchar();
return 0;
}

In the above program, a char array is typecasted to an unsigned short integer type. When I run above program on little endian machine, I get 1 as output, while if I run it on a big endian machine I get 512. To make programs endianness independent, above programming style should be avoided.

What are bi-endians?

Bi-endian processors can run in both modes little and big endian.

What are the examples of little, big endian and bi-endian machines ?
Intel based processors are little endians. ARM processors were little endians. Current generation ARM processors are bi-endian.

Motorola 68K processors are big endians. PowerPC (by Motorola) and SPARK (by Sun) processors were big endian. Current version of these processors are bi-endians.

Does endianness effects file formats?

File formats which have 1 byte as a basic unit are independent of endianness e..g., ASCII files . Other file formats use some fixed endianness forrmat e.g, JPEG files are stored in big endian format.

Which one is better — little endian or big endian

The term little and big endian came from Gulliver's Travels by Jonathan Swift. Two groups could not agree by which end a egg should be opened -a-the little or the big. Just like the egg issue, there is no technological reason to choose one byte ordering convention over the other, hence the arguments degenerate into bickering about sociopolitical issues. As long as one of the conventions is selected and adhered to consistently, the choice is arbitrary.

Some of the puzzling things about c language

Can you guess why there is no distinct format specifier for 'double' in the printf/scanf format string, although it is one of the four basic data types? (Remember we use %lf for printing the double value in printf/scanf; %d is for integers).

Ans: In older versions of C, there was no 'double'—it was just 'long float' type—and that is the reason why it has the format specifier '%lf' ('%d' was already in use to indicate signed decimal values). Later, double type was added to indicate that the floating point type might be of 'double precision' (IEEE format, 64-bit value). So a format specifier for long float and double was kept the same.

Why is the output file of the C compiler called a.out?

Ans: The a.out stands for 'assembler.output' file [2]. The original UNIX was written using an assembler for the PDP-7 machine. The output of the assembler was a fixed file name, which was a.out to indicate that it was the output file from the assembler. No assembly needs to be done in modern compilers; instead, linking and loading of object files is done. However, this tradition continues and the output of cc is by default a.out!

what is the use of volatile keyword in c?

The meaning of volatile is a popular interview question, particularly for freshers and I've read articles describing the properties of this keyword as if it had super powers.

The volatile keyword does a very simple job. When a variable is marked as volatile, the programmer is instructing the compiler not to cache this variable in a register but instead to read the value of the variable from memory each and every time the variable is used. That's it – simple isn't it?

To illustrate the use of the keyword, consider the following example:

volatile int* vp = SOME_REGISTER_ADDRESS;
for(int i=0; i<100; i++)
foo(*vp);
In this simple example, the pointer vp points to a volatile int. The value of this int is read from memory for each loop iteration. If volatile was not specified then it is likely that the compiler would generate optimized code which would read the value of the int once, temporarily store this in a register and then use the register copy during each iteration.

hope after reading this you reply confidently, when the same question is asked to you in the interview

Integer overflow and why it occurs?

What is integer overflow and why it occurs?

An integer overflow, or integer wrapping, is a potential problem in a program based upon the fact that the value that can be held in a numeric data type is limited by the data type's size in bytes. ANSI C uses the following minimum sizes:

data type size (bytes)
char 1
short 2
int 2
long 4

In practice, many compilers use a 4-byte int. It also should be noted that the actual ranges for the data types depend on whether or not they are signed. for instance, a signed 2-byte short may be between -32767 and 32767, while an unsigned short may be between 0 and 65535. See your [include]/limits.h file for specific numbers for your compiler.

Why should you care?

If you try to put a value into a data type that is too small to hold it, the high-order bits are dropped, and only the low-order bits are stored. Another way of saying that is that modulo-arithmetic is performed on the value before storing it to make sure it fits within the data type. Taking our unsigned short example:

Limit: 65535 or 1111 1111 1111 1111
Too big: 65536 or 1 0000 0000 0000 0000
What's stored: 0 or 0000 0000 0000 0000

As the above makes evident, that result is because the high-order (or left-most) bit of the value that's too big is dropped. Or you could say that what's stored is the result of
Stored = value % (limit + 1) or 65536 % (65535 + 1) = 0
In signed data types, the result is a little different and results in some seemingly weird behavior:

Positive limit: 32767 or 0111 1111 1111 1111
Too big: 32768 or 1000 0000 0000 0000
What's stored: -32768

Why's that?

It's because of "2′s compliment," which is how negative numbers are represented in binary. To make a long story short, the first half of the range (0 thru 0111 1111 1111 1111) is used for positive numbers in order of least to greatest. The second half of the range is then used for negative numbers in order of least to greatest. So, the negative range for a signed 2-byte short is -32768 thru -1, in that order.

When it occurs?

Integer overflow happens because computers use fixed width to represent integers. So which are the operations that result in overflow? Bitwise and logical operations cannot overflow, while cast and arithmetic operations can. For example, ++ and += operators can overflow, whereas && or & operators (or even <> operators) cannot.

Regarding arithmetic operators, it is obvious that operations like addition, subtraction and multiplication can overflow. How about operations like (unary) negation, division and mod (remainder)? For unary negation, -MIN_INT is equal to MIN_INT (and not MAX_INT), so it overflows. Following the same logic, division overflows for the expression (MIN_INT / -1). How about a mod operation? It does not overflow. The only possible overflow case (MIN_INT % -1) is equal to 0 (verify this yourself—the formula for % operator is a % b = a – ((a / b) * b)).

What happens when it occur?

Suppose memory is being allocated based on an unsigned integer data type's value. If that value has wrapped around, it may be that far too little memory will be made available. Or if a comparison is being made between a signed integer value and some other number, assuming that the former should be less than the latter, if that value has over flown into the negative, the comparison would pass. But are things going to behave the way the programmer intended? Probably not.

Sorting and searching techniques are most important for the programmer and we will use it in most of the programs we write. Among them the most widely used are binary search and quick sort for efficiency. In these algorithms we use mean of two elements. If the elements are integers then the mean is prone to be integer overflow condition. This results in undesirable results or bugs. So, let's look at some techniques to detect an overflow before it occurs.

For the statement int k = (i + j);
• If i and j are of different signs, it cannot overflow.
• If i and j are of same signs (- or +), it can overflow.
• If i and j are positive integers, then their sign bit is zero. If k is negative, it means its sign bit is 1—it indicates the value of (i + j) is too large to represent in k, so it overflows.
• If i and j are negative integers, then their sign bit is one. If k is positive, it means its sign bit is 0—it indicates that the value of (i + j) is too small to represent in k, so it overflows.

How to check for the overflow?

To check for overflow, we have to provide checks for conditions 3 and 4. Here is the straightforward conversion of these two statements into code. The function isSafeToAdd(), returns true or false, after checking for overflow.

/* Is it safe to add i and j without overflow? Return value 1 indicates there is no overflow; else it is overflow and not safe to add i and j */
int isSafeToAdd(int i, int j)
{
if( (i < 0 && j =0) || (i > 0 && j > 0) && k INT_MAX) or if ((i + j) INT_MAX) || ((i + j) INT_MAX), we can check the condition (i > INT_MAX – j) by moving j to the RHS of the expression. So, the condition in isSafeToAdd can be rewritten as:

if( (i > INT_MAX – j) || (i < INT_MIN – j) )
return 0;

That works! But can we simplify it further? From condition 2, we know that for an overflow to occur, the signs of i and j should be the same. If you notice the conditions in 3 and 4, the sign bit of the result (k) is different from (i and j). Does this strike you as the check that the ^ operator can be used? How about this check:

int k = (i + j);
if( ((i ^ k) & (j ^ k)) < 0)
return 0;

Let us check it. Assume that i and j are positive values and when it overflows, the result k will be negative. Now the condition (i ^ k) will be a negative value—the sign bit of i is 0 and the sign bit of k is 1; so ^ of the sign bit will be 1 and hence the value of the expression (i ^ k) is negative. So is the case for (j ^ k) and when the & of two values is negative; hence, the condition check with < 0 becomes true when there is overflow. When i and j are negative and k is positive, the condition again is < 0 (following the same logic described above).

So, yes, this also works! Though the if condition is not very easy to understand, it is correct and is also an efficient solution!

memory layout in c..data segment,bss, code segment, stack, heap segment

When we come across memory segments in C program these are the questions that comes to our mind.

• What happens when a c program is loaded into memory?
• Where are the different types of variables allocated?
• Why do we need two data sections, initialized and un-initialized?
• If we initialize a static or global variable with 0 where will it be stored?

Even though the scope of global and static variables are different, why are they stored in same section i.e., data segment?

Let's look at some of these interesting under hood details here. We know that a C program which is compiled to an executable and loaded into memory for execution has 4 main segments in memory. They are data, code, stack, and heap segments.

Global and function static variables are allocated in the data segment. The compiler converts the executable statements in C program such as printf("hello world"); into machine code. They are loaded in the code segment. When the program executes, function calls are made. Executing each function requires allocation of memory, as if in a frame to store different information like the return pointer, local variable…etc. since this allocation is done in the stack, these are known as stack frames. When we do dynamic memory allocation, such as the use of the malloc function, memory is allocated in the heap area.
Static and Dynamic Segments

The data and code segments are of fixed size. When a program is compiled, at that point itself, the sizes required for the segments are fixed and known. Hence they are known as static segments. The sizes of the stack and heap areas are not known when the program gets compiled. Also it is possible to change or configure the sizes of these areas (i.e., increase or decrease). So, these are called dynamic segments.
Let's look at each of these segments in detail.

Data segment:- the data segment is to hold the value of those variables that need to be available throughout the life time of the program. So it is obvious that global variables should be allocated in the data segment. How about local variables declared as static? Yes, they are also allocated in the data area because their values should be available across function calls. If they are allocated in the stack frame itself, they will get destroyed once the function returns. The only option is to allocate them in a global area. Hence, they are allocated in this segment. So, the life time of a local static variable is that of the life time of the program.

There are two parts in this segment. The initialized data segment and u-initialized data segment.
When variables are initialized to some value (other than 0 or which is different value), they are allocated in the initialized segment. When the variables are un initialized they get allocated in the un-initialized data segment. This segment is usually referred to with cryptic acronym called BSS. It stands for block starting with symbol and gets its name from old IBM systems which had that segments initialized to zero.
The data area is separated into two based on explicit initialization, because the variables that are need to be initialized need not be initialized with zeros one by one. However the variables that are not initialized need not to be explicitly initialized with zeros one by one. Instead the job of initialization of variables to zero is left to the operating system to take care of. This bulk initialization can greatly reduce the time required to load.

When we want to run an executable program, the OS starts a program known as loader. When this loads the file into memory, it takes the BSS segment and initializes the whole thing to zeros. That is why the un-initialized global data and static data always get the default value of zero.

The layout of data segment is in the control of the underlying OS. However some loaders give partial control to the users. This information may be useful in applications such as embedded systems.
The data area can be addressed and accessed using pointers from the code. Automatic variables have an overhead in initializing the variables each time they are required, and code is required to do that initialization. However, variables in the data area do not have such runtime overhead, because the initialization is done only once and that too at loading time.

Code segment:- the program code is where the executable code is available for execution. This area is also known as the text segment and is of fixed size. This can be accessed only by function pointers and not by other data pointers. Another important piece of information to take note of here is that the system may consider this area as a read only memory area and any attempt to write in this area can lead to undefined behavior.

Stack and heap segments:- to execute the program two major parts of the memory used are stack and heap. Stack frames area created in the stack for functions and in the heap for dynamic memory allocation. The stack and heap are un-initialized areas. Therefore whatever happens to be in the memory becomes the initial (garbage) value for the objects created in that space.

The local variable and function arguments are allocated in the stack. For the local variables that have an initialization value, code is generated by the compiler to initialize them explicitly to those values when the stack frames are created. For function parameters the compiler generates code to copy the actual arguments to the space allocated for the parameters in the stack frame.

Here, we will take a small program and see where different program elements are stored when that program executes. The comments explain where the variables get stored.

int bss1;
static double bss2;
char *bss3;
// these are stored in initialized to zero segment also known as un-initialized data segment(BSS)
int init1=0;
float init2=10.0f
char *init3="hello world";
// these are stored in initialized data segment
// the code for main function gets stored in the code segment
int main
{
int local1=10; // this variable is stored in the stack and initialization code is generated by compiler.
int local2; //this variable is not initialized hence it has garbage value. It does not get initialized to zero.
static int local3; // this is allocated in the BSS segment and gets initialized to zero
static int local4=100; //this gets allocated in initialed data segment
int (*local-foo) (const char* —)= printf; // printf is in a shared library (libc or c runtime library)
// load-foo is a local variable(function pointer) that points to the printf function local-foo("hello world"); this function call results in the creation of stack frame in stack area
int *loacl5=malloc(sizeof(int));
// allocated in stack however it points to dynamically allocated block in heap.
return 0;
// stack frame for the main function gets destroyed after executing main
}

There several tools to check where a variable gets stored in the memory. But the easy to use tool is nm.

Using nm tool

Gcc program name
nm ./a.out

if no arguments are given to nm, it assumes that it should take the input as a.out and we will get some cryptic output like below.

Where the symbols that we did not type are come from? They have been inserted behind the screen by compiler for various reasons. We can ignore them for now.

Now what are those strange numbers, followed by letters (b, B, t). The numbers are the symbol values followed by the symbol type (displayed as a letter) and the symbol name.

The symbol type requires more explanation. A lowercase means it is local variable and uppercase means global (externally available from the file).
B un-initialized data section (BSS)
D initialized data section
T text/code section
U un identified

Example 1: nm ./a.out | grep bss

Variables bss1, bss3 got allocated in the BSS segment (global) since we put the class as static for variable bss2, it is listed as b (accessible with in the file).

Example 2: nm ./a.out | grep init

These are explicitly initialized and are allocated into initialized data section.

Example 3: nm ./a.out | grep local

Only local3 and local4 are allocated global memory. Since local3 is un-initialized it is allocated in the BSS and since local 4 is explicitly initialized it is allocated in the initialized data segment. As both are local they are indicated by small letters. Since they are local to the function and to avoid accidental mixing them up with other local variables with the same name they have been suffixed by some numbers. (Compilers differ in their approaches in treating local static variables. This approach is for gcc ).

08048354 T main
U malloc@@GLIBC_2.0
U printf@@ GLIBC_2.0

The main function is allocated in the text/code segment. Obviously we can access this function from outside the file ( to start the execution ). So the type of this symbol is T.

The malloc and printf function used in the program are not defined in the program itself ( header files only declare them, they don't define them ). They are defined in the shared library GLIBC, version 2.0. that's what the suffix @@GLIBC_@.0 implies.