boost::python iterating over dictionaries

When working with boost::python, it’s quickly noticeable the lack of documentation involved in integrating Python in a C++ application, however using boost::python is a lot easier then using the normal Python/C API so it’s recommended to whomever considers using Python in a C++ application.

Tricky

The downside of it is the lack of examples, and some things not working as expected (perhaps as documented, but I couldn’t find a decent documentation involving Python-in-C++)

So that lack of examples makes the whole ordeal very tedious and error-prone, as with the implementation I’m about to show.

The expected

When programming in C++ trying to access Python variables, one has access to several boost-wrappers to represent most python variable types, one of which is a dictionary (boost::python::dict)

test.py:
d = {
  'key': 'value',
  'another': 'yeah'
}

That’s the easy stuff, now, accessing it from C++:

test.cpp:
#include <boost/python.hpp>
#include <iostream>

int main()
{
  // Initialize Python
  Py_Initialize();

  // Get the main namespace/module up-and-running
  boost::python::object main_module = boost::python::import("__main__");
  boost::python::object main_namespace = main_module.attr("__dict__");

  // Run the python executable, in this case "test.py"
  boost::python::exec_file(boost::python::str("test.py"), main_namespace);

  // Get the 'd' variable
  // This method could be shortened by using boost::python::object d = main_namespace["d"]
  // However, using extract is more general, and could be used in templated functions (hint hint)
  boost::python::object d = boost::python::extract<boost::python::object>(main_namespace["d"]);

  // Get the dictionary, because just the "object" isn't enough to access
  // Note that the previous part can actually be skipped, but it's included for sake of brevity
  boost::python::dict d_dict = boost::python::extract<boost::python::dict>(d);

  // Now, we're going to try and "print" everything involved
  boost::python::list iterkeys = (boost::python::list)d_dict.iterkeys();
  for (int i = 0; i < boost::python::len(iterkeys); i++)
  {
    // Because we know they're strings, we can do this
    std::string key = boost::python::extract<std::string>(iterkeys[i]);
    std::string value = boost::python::extract<std::string>(d_dict[iterkeys[i]]);

    std::cout << "Key: " << key << std::endl;
    std::cout << "Value: " << value << std::endl << std::endl;
  }
}

Compile using g++ -lboost_python -lpython2.6 -I/usr/include/python2.6 test.cpp

Executing this should give:

12216 cpf@core ~/junk/cpp % ./a.out
Key: another
Value: yeah

Key: key
Value: value

The unexpected

Now, change test.py to represent:

test.py:
d = {
 'key': {
 'another': 'yeah'
 }
}

And once again execute the file, notice the error (terminate called after throwing an instance of ‘boost::python::error_already_set’), this error is the one and only error boost::python has!

To figure out what went wrong, change test.cpp like this:

// Because we know they're strings, we can do this
std::string key = "";
std::string value = "";
try
{
  key = boost::python::extract<std::string>(iterkeys[i]);
  value = boost::python::extract<std::string>(d_dict[iterkeys[i]]);
} catch (boost::python::error_already_set const &)
{
  PyErr_Print();
}

It should give you the error: TypeError: No registered converter was able to produce a C++ rvalue of type std::string from this Python object of type dict

Using boost::python::dict for value all the way fixes this.

Another word

During testing of a program I was writing to include Python in it and read a (lot) of variables, I had the weird occasion in which a dict only had 1 key/value pair, and using iterkeys[i] would crash saying TypeError: No to_python (by-value) converter found for C++ type: boost::python::api::proxy<boost::python::api::item_policies>

The solution is to use iterkeys.pop(0) instead of iterkeys[i]

HOWEVER: BE CAREFUL DOING THIS, THIS WILL CHANGE THE ITERKEYS ON-THE-FLY!