Published by AndreyKarpov

Oct 10, 2014 (last update: Oct 10, 2014)

Virtual Method Table and accident prevention

Score: 0.0/5 (0 votes)

As a small warm-up before the article, I would like readers to ask themselves: does a photographer need to know how camera works in order to make qualitative photos? Well, does he need to know the term "diaphragm" at least? "Signal-to-noise ratio"? "Depth of field"? Practice shows that even with a knowledge of such difficult terms photos shot by the most "gifted ones" may be just a little bit better that photos shot by cell phone camera through 0.3 MP "hole". Alternatively, good quality photos may be shot due to the outstanding experience and intuition without any knowledge whatsoever (but usually it is an exception to the rules). Nevertheless, it is unlikely that there is somebody who can argue with me in the fact that professionals who want to get every single possibility from their camera (not only MP in a square millimeter on an image sensor) are required to know these terms, or else they cannot be called professionals at all. That is true not only in digital photography, but in almost every other industry as well.

That is also true for programming, and for programming on C++ it is true twice as much. In this article, I shall explain an important language feature, known as virtual table pointer, which is included in almost every nontrivial class, and how it can accidentally be damaged. Damaged virtual table pointer may lead to very difficult to fix errors. First, I am going to recall what virtual table pointer is, and then I shall share my thoughts what and how can be broken there.

To my regret, in this article will be a lot of reasoning related to low level. However, there is no other way to illustrate the problem. In addition, I should tell that this article is written for Visual C++ compiler in 64-bit mode - results may differ with usage of other compilers and other target systems.

Virtual table pointer

In theory, it is said that vptr pointer, virtual table poiner, or vpointer, is stored in every class that has at least one virtual method. Let us puzzle out what a thing is this. For this, let us write a simple demo program on C++.

#include <iostream>
#include <iomanip>
using namespace std;
int nop() {
  static int nop_x; return ++nop_x; // Don't remove me, compiler!
};

class A
{
public:
  unsigned long long content_A;
  A(void) : content_A(0xAAAAAAAAAAAAAAAAull)
      { cout << "++ A has been constructed" << endl;};
  ~A(void) 
      { cout << "-- A has been destructed" << endl;};

  void function(void) { nop(); };
};

void PrintMemory(const unsigned char memory[],
                 const char label[] = "contents")
{
  cout << "Memory " << label << ": " << endl;
  for (size_t i = 0; i < 4; i++) 
  {
    for (size_t j = 0; j < 8; j++)
      cout << setw(2) << setfill('0') << uppercase << hex
           << static_cast<int> (memory[i * 8 + j]) << " ";
    cout << endl;
  }
}

int main()
{
  unsigned char memory[32];
  memset(memory, 0x11, 32 * sizeof(unsigned char));
  PrintMemory(memory, "before placement new");

  new (memory) A;
  PrintMemory(memory, "after placement new");
  reinterpret_cast<A *>(memory)->~A();

  system("pause");
  return 0;
};

Despite of relatively large size of code, its logic should be clear: first, it allocates 32 bytes on stack, which is filled then with 0x11 values (0x11 value will indicate a "garbage" in memory, i.e. not-initialized memory). Secondly, with usage of placement new operator it creates trivial class A object. Lastly, it prints memory contents, after which destructs A object and terminates normally. Below you can see output of this program (Microsoft Visual Studio 2012, x64).


Memory before placement new:
11 11 11 11 11 11 11 11
11 11 11 11 11 11 11 11
11 11 11 11 11 11 11 11
11 11 11 11 11 11 11 11
++ A has been constructed
Memory after placement new:
AA AA AA AA AA AA AA AA
11 11 11 11 11 11 11 11
11 11 11 11 11 11 11 11
11 11 11 11 11 11 11 11
-- A has been destructed
Press any key to continue . . .

It is easy to notice that size of class in memory is 8 bytes and is equal to size of its only member "unsigned long long content_A".

Let us complicate our program a bit with addition of "virtual" keyword to declaration of void function(void):

virtual void function(void) {nop();};

Program output (hereinafter only part of output will be shown, "Memory before placement new" and "Press any key..." will be omitted):


++ A has been constructed
Memory after placement new:
F8 D1 C4 3F 01 00 00 00
AA AA AA AA AA AA AA AA
11 11 11 11 11 11 11 11
11 11 11 11 11 11 11 11
-- A has been destructed

Again, it is easy to notice that size of class is now 16 bytes. First eight bytes now contain a pointer to virtual method table. On this run it was equal to 0x000000013FC4D1F8 (pointer and content_A are "reversed" in memory due to the Intel64's little-endian byte order; however, in case of content_A it is kind of difficult to notice).

Virtual method table is a special structure in memory that is generated automatically and that contains pointers to all virtual methods listed in this class. When somewhere in code function() method is called in the context of pointer to A class, instead of call to A::function() directly, a call to function located in virtual method table with some offset will be called - this behavior realizes polymorphism. Virtual method table is presented below (it is obtained after compiling with /FAs key; in addition take note to somewhat strange function name in assembly code - it went through "name mangling"):

CONST SEGMENT
??_7A@@6B@ DQ  FLAT:??_R4A@@6B@   ; A::'vftable'
 DQ FLAT:?function@A@@UEAAXXZ
CONST ENDS

__declspec(novtable)

Sometimes such a situation occurs when there is no need in virtual table pointer at all. Let us suppose that we shall never instantiate object of A class, and if we shall, only on weekend and on holidays, meticulously controlling that no one virtual function is called. This situation is frequent in case of abstract classes - it is known that abstract classes cannot be instantiated no matter what. Actually, if function() was declared in A class as abstract method, virtual method table would look like this:

CONST SEGMENT
??_7A@@6B@ DQ FLAT:??_R4A@@6B@ ; A::'vftable'
 DQ FLAT:_purecall
CONST ENDS

It is obvious that an attempt to call this function would result in a shooting one's own leg.

After this, the question arises: if class is never instantiated, is there a reason to initialize virtual table pointer? To prevent compiler from generating redundant code, programmer can give it a __declspec(novtable) attribute (be careful: Microsoft-specific!). Let us rewrite our virtual function example using __declspec(novtable):

class __declspec(novtable) A { .... }

Program output:


++ A has been constructed
Memory after placement new:
11 11 11 11 11 11 11 11
AA AA AA AA AA AA AA AA
11 11 11 11 11 11 11 11
11 11 11 11 11 11 11 11
-- A has been destructed

Notice that size of an object has not changed: it is still 16 bytes. After including __declspec(novtable) attribute there are only two differences: first, on place of virtual table pointer there is an uninitialized memory, secondly - in assembler code there is no virtual method table of class A at all. Nevertheless, virtual table pointer is present and has a size of eight bytes! This is the thing to remember, because...

Inheritance

Let us rewrite our example to realize simplest inheritance technique from the abstract class with virtual table pointer.

class __declspec(novtable) A // I never instantiate
{
public:
  unsigned long long content_A;
  A(void) : content_A(0xAAAAAAAAAAAAAAAAull)
      { cout << "++ A has been constructed" << endl;};
  ~A(void) 
      { cout << "-- A has been destructed" << endl;};

  virtual void function(void) = 0;
};

class B : public A // I always instantiate instead of A
{
public:
  unsigned long long content_B;
  B(void) : content_B(0xBBBBBBBBBBBBBBBBull)
      { cout << "++ B has been constructed" << endl;};
  ~B(void) 
      { cout << "-- B has been destructed" << endl;};

  virtual void function(void) { nop(); };
};

In addition, we need to make that instead of instantiating class A main program would have constructed (and destructed) an object of class B:

....
new (memory) B;
PrintMemory(memory, "after placement new");
reinterpret_cast<B *>(memory)->~B();
....

Program output will be like this:


++ A has been constructed
++ B has been constructed
Memory after placement new:
D8 CA 2C 3F 01 00 00 00
AA AA AA AA AA AA AA AA
BB BB BB BB BB BB BB BB
11 11 11 11 11 11 11 11
-- B has been destructed
-- A has been destructed

Let us try to figure out what has happened. Constructor B::B() had been called. This constructor before executing its body had called base class' constructor A::A(). If __declspec(novtable) attribute was not present, A::A() would have been initialized virtual table pointer; in our case virtual table pointer has not been initialized. Then constructor set content_A value to 0xAAAAAAAAAAAAAAAAull (second field in memory) and returned flow of execution to B::B().

Because there is no __declspec(novtable) attribute, constructor set virtual table pointer (first field in memory) to virtual method table of class B, set content_B value to 0xBBBBBBBBBBBBBBBBull (third field in memory) and then returned flow of execution to main program. Taking in consideration memory contents it is easy to find out that the object of B class was constructed correctly, and program logic makes it clear that one unnecessary operation was skipped. If you are confused: unnecessary operation in this context is an initializing virtual table pointer in a base class' constructor.

It would seem that only one operation was skipped. What is the point in removing it? But what if program have thousands and thousands of classes derived from one abstract class, removing one automatically generated command can significantly affect program performance. Moreover, it will. Do you believe me?

memset function

The main idea of memset() function lies in filling memory field with some constant value (most often with zeroes). In C language it could have been used to quickly initialize all structure field. What is the difference between simple C++ class without virtual table pointer and C structure in terms of memory arrangement? Well, there is none, C raw data is the same as C++ raw data. To initialize really simple C++ classes (in terms of C++11 - standart layout types) it is possible to use memset() function. Well, it is also possible to use memset() function to initialize every class. However, what is the consequences of that? Incorrect memset() call may damage virtual table pointer. This raises the question: maybe it is possible, when class has __declspec(novtable) attribute?

The answer is: possible, but with precautions.

Let us rewrite our classes in another way: add wipe() method, which is used to initialize all contents of A to 0xAA:

class __declspec(novtable) A // I never instantiate
{
public:
  unsigned long long content_A;
  A(void)
    {
      cout << "++ A has been constructed" << endl;
      wipe();
    };
    // { cout << "++ A has been constructed" << endl; };
  ~A(void) 
    { cout << "-- A has been destructed" << endl;};

  virtual void function(void) = 0;
  void wipe(void)
  {
    memset(this, 0xAA, sizeof(*this));
    cout << "++ A has been wiped" << endl;
  };
};

class B : public A // I always instantiate instead of A
{
public:
  unsigned long long content_B;
  B(void) : content_B(0xBBBBBBBBBBBBBBBBull)
      { cout << "++ B has been constructed" << endl;};
      // {
      //   cout << "++ B has been constructed" << endl;
      //   A::wipe();
      // };

  ~B(void) 
      { cout << "-- B has been destructed" << endl;};

  virtual void function(void) {nop();};
};

The output in this case will be as expected:


++ A has been constructed
++ A has been wiped
++ B has been constructed
Memory after placement new:
E8 CA E8 3F 01 00 00 00
AA AA AA AA AA AA AA AA
BB BB BB BB BB BB BB BB
11 11 11 11 11 11 11 11
-- B has been destructed
-- A has been destructed

So far, so good.

Nevertheless, if we change wipe() function call by commenting out constructors lines and uncommenting lines next to them, it will become clear that something went wrong. First call to virtual method function() will cause run-time error due to damaged virtual table pointer:


++ A has been constructed
++ B has been constructed
++ A has been wiped
Memory after placement new:
AA AA AA AA AA AA AA AA
AA AA AA AA AA AA AA AA
BB BB BB BB BB BB BB BB
11 11 11 11 11 11 11 11
-- B has been destructed
-- A has been destructed

Why has it happened? Wipe() function was called after B constructor initialized virtual table pointer. As a result, wipe() damaged this pointer. In other words - it is not advised to zero class with virtual table pointer even it is declared with __declspec(novtable) attribute. Full zeroing will be appropriate only in a constructor of a class that will never be instantiated, but even this should be done only with great caution.

memcpy function

All the words above can be applied to memcpy() function as well. Again, its purpose is to copy standard layout types. However, judging by the practice, some programmers enjoy using it when it is needed and when it is not. In case of non-standard layout types usage of memcpy() is like ropewalking above Niagara falls: one mistake can be fatal, and this fatal mistake can be made surprisingly easy. As an example:

class __declspec(novtable) A
{
  ....
  A(const A &source) { memcpy(this, &source, sizeof(*this)); }
  virtual void foo() { }
  ....
};
class B : public A { .... };

Copy constructor can write anything his digital soul wants into virtual table pointer of an abstract class: constructor of derived class will anyway initialize it with correct value. However, in body of assignment operator usage of memcpy() is forbidden:

class __declspec(novtable) A
{
  ....
  A &operator =(const A &source)
  {
    memcpy(this, &source, sizeof(*this)); 
    return *this;
  }
  virtual void foo() { }
  ....
};
class B : public A { .... };

To finish the picture, remember that nearly every copy constructor and assignment operator have nearly identical bodies. No, it is not as bad as it looks like at first glance: in practice assignment operator may work as expected not due to the correctness of code, but due to the stars' wish. This code copies virtual table pointer from another class and results are highly unpredictable.

PVS-Studio

This article is a result of detailed research about this mysterious __declspec(novtable) attribute, cases when it is possible to use memset() and memcpy() functions in a high-level code, and when it is not. From time to time developers ask us about the fact that PVS-Studio shows too many warnings about virtual table pointer. Developers frequently mail us about virtual table pointer. Programmers think that if __declspec(novtable) is present, class have no virtual method table and no virtual table pointer either. We had started to carefully puzzle out this question, and then we have understood that it is not as simple as it looks.

It should be kept in mind. If __declspec(novtable) attribute is used in class declaration, it does not mean that this class does not contain virtual table pointer! Does class initialize it or not? It is another kind of question.

In future we are going to make our analyzer to suppress warning about usage of memset()/memcpy(), but only in case of base classes with __declspec(novtable).

Conclusion

Unfortunately, this article does not cover a lot of material about inheritance (for example, we have not covered multiple inheritance at all). Nevertheless, I hope that this information would allow understanding that "it is not as simple as it looks" and it is advisable to think three times before using low-level function in conjunction with high-level objects. Moreover, is it worth it?