6. Taking off the engine cover

How does Python “know” how to add two objects? Or what they should look like when printed to the console? Let’s dig into the underlying mechanisms that Python provides for this.

a = 2
b = 3

a + b

5

What methods does a have?

dir(a)

['__abs__',
 '__add__',
 '__and__',
 '__bool__',
 '__ceil__',
 '__class__',
 '__delattr__',
 '__dir__',
 '__divmod__',
 '__doc__',
 '__eq__',
 '__float__',
 '__floor__',
 '__floordiv__',
 '__format__',
 '__ge__',
 '__getattribute__',
 '__getnewargs__',
 '__gt__',
 '__hash__',
 '__index__',
 '__init__',
 '__init_subclass__',
 '__int__',
 '__invert__',
 '__le__',
 '__lshift__',
 '__lt__',
 '__mod__',
 '__mul__',
 '__ne__',
 '__neg__',
 '__new__',
 '__or__',
 '__pos__',
 '__pow__',
 '__radd__',
 '__rand__',
 '__rdivmod__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__rfloordiv__',
 '__rlshift__',
 '__rmod__',
 '__rmul__',
 '__ror__',
 '__round__',
 '__rpow__',
 '__rrshift__',
 '__rshift__',
 '__rsub__',
 '__rtruediv__',
 '__rxor__',
 '__setattr__',
 '__sizeof__',
 '__str__',
 '__sub__',
 '__subclasshook__',
 '__truediv__',
 '__trunc__',
 '__xor__',
 'bit_length',
 'conjugate',
 'denominator',
 'from_bytes',
 'imag',
 'numerator',
 'real',
 'to_bytes']

All those methods with __ on both sides are methods that are not normally shown in the tab completion list for the object. They are often called “dunder” methods for brevity, so you would say “dunder abs” for __abs__. In the documentation for Python these are called “special methods”.

Special methods are how some fundamental properties of objects are implemented in Python. For instance, you can safely imagine that a + b is translated to

a.__add__(b)

5

This is the mechanism by which different kinds of objects can do very different kinds of things when + is used on them:

a = '2'
b = '3'

a + b

'23'

a.__add__(b)

'23'

We’ve already encountered one special method: __init__. Let’s build a class which stores a value internally.

class TestClass:
    def __init__(self, value):
        self.value = value

We can now create objects of class TestClass:

a = TestClass(2)
b = TestClass(3)

But they are a bit hard to use. For one, they don’t display anything meaningful when we display them

a

<__main__.TestClass at 0x10750fcc0>

We can extend TestClass with a __repr__ method. This is short for representation and is used in the console and the notebook to show an object. By convention, the __repr__ method returns a string that could be copy-pasted to create the object.

class TestClass:
    def __init__(self, value):
        self.value = value
    def __repr__(self):
        return "TestClass({})".format(self.value)

a = TestClass(2)
b = TestClass(3)

a

TestClass(2)

Great, at least we can see what we’re working with now. But let’s say we want to make this class be able to support addition. At the moment, this doesn’t work:

# a + b
# ---------------------------------------------------------------------------
# TypeError                                 Traceback (most recent call last)
# <ipython-input-14-f96fb8f649b6> in <module>()
# ----> 1 a + b

# TypeError: unsupported operand type(s) for +: 'TestClass' and 'TestClass'

class TestClass:
    def __init__(self, value):
        self.value = value

    def __repr__(self):
        return "TestClass({})".format(self.value)

    def __add__(self, other):
        return TestClass(self.value + other.value)

a = TestClass(2)
b = TestClass(3)

a + b

TestClass(5)

This should give you a glimpse into how libraries like SymPy or Numpy obtain their effects. They are using these mechanisms to make objects which “do the right thing” when they are added together, divided and multiplied, as well as giving them “normal” non-underscore methods for additional manipulation. All of the objects you have used to do math so far have implemented these operations. Think of sympy.Symbol or the numpy.array.