Metaprogramming

Metaprogramming is the process of using code to write code.

It is hard and tricky to get right. In almost all cases, do not use metaprogramming, and consider redesigning your code instead.

When not to use

An example of when not to use metaprogramming is as follows: say you have a Basket class, and you want to instantiate an object from this class based on elements in a list.

Instead of using metaprogramming, this might be best achieved using a dictionary:

class Basket:
    # code here
    pass

basket_names = ["apples", "oranges", "bananas"]
baskets = {name: Basket() for name in basket_names}

isinstance(baskets["apples"], Basket)
# returns True

If this doesn’t work, or you think metaprogramming is the best way to achieve your goal, read on.

Using Namespaces

All objects are stored in the namespace, which is a dictionary defined separately for the main module, and each class and function inside it. In this dictionary, - the keys are the names of the objects, as strings - the values are the objects themselves

The main namespace, accessible by everything in a file, is accessed via globals().

For example, the above basket scenario might be implemented using

basket_names = ["apples", "oranges", "bananas"]

for name in basket_names:
    globals()[name] = Basket()

isinstance(apples, Basket)
# returns True

The local namespace, accessible by only the surrounding object and those objects it wraps, is accessed using locals().

Importantly, objects cannot modify globals()! For example, consider the following 

global_var = 1
print(globals())
# {"global_var": 1}

def foo():
    # access globals()
    print(global_var)
    # "1"

    # define a *local* variable which *happens* to have the same name as a global one
    global_var = 2
    print(global_var)
    # 2

    print(globals())
    # {"global_var":1, "foo": <function foo at ...>}
    # showing the original global variable is unchanged

    print(locals())
    # {'global_var': 2, 'local_var': 3}
    # showing we've created a new local variable with the same name, as well as another which isn't in globals()

foo()

The above example shows how objects can access globals(), but can only modify locals(). Be careful what you name your variables!

Attributes

We can get and set the attributes of a class or object using getattr and setattr.

class Example:
    pass

def square(x):
    return x**2

ex1 = Example()
ex2 = Example()

# give ex1 a name
setattr(ex1, "name", "Michael")

# give ex1 a lambda function
setattr(ex1, "foo", lambda self, x: f"foo {self.name}, {x}")

# give ex2 a function
setattr(ex2, "sq", square)

# give the class Example an attribute
setattr(Example, "bar", lambda self: f"bar {self.name}")

print(ex1.name)        # Michael
print(ex1.foo(ex1, 3)) # foo Michael, 3

print(ex2.sq(3))       # 9

print(ex1.bar())       # bar Michael
print(ex2.bar())       # AttributeError: 'Example' object has no attribute 'name'

Here, we see a few key things: