Metaprogramming in Elixir

Usually, we think of a program as something that manipulates data to achieve some result.

But what is data?

Can we use the programs themselves as data? 🤔

In today’s article, we’ll go down the rabbit hole with the assistance of Elixir, a programming language that is permeated by metaprogramming.

I’ll introduce you to metaprogramming in Elixir and show you how to create a macro for defining curried functions.

What is metaprogramming?

Metaprogramming is just writing programs that manipulate programs. It’s a wide term that can include compilers, interpreters, and other kinds of programs.

In this article, we will focus on metaprogramming as it is done in Elixir, which involves macros and compile-time code generation.

Metaprogramming in Elixir

To understand how metaprogramming works in Elixir, it is important to understand a few things about how compilers work. During the compilation process, every computer program is transformed into an abstract syntax tree (AST) – a tree structure that enables the computer to understand the contents of the program.

Quick maths.

In Elixir, each node of the AST (except basic values) is a tuple of three parts: function name, metadata, function arguments.

Elixir enables us to access this internal AST representation via quote.

iex(1)> quote do 2 + 2 - 1 end
{:-, [context: Elixir, import: Kernel],
 [{:+, [context: Elixir, import: Kernel], [2, 2]}, 1]}

We can modify these ASTs with macros, which are functions from AST to AST that are executed at compile-time.

You can use macros to generate boilerplate code, create new language features, or even build domain-specific languages (DSLs).

Actually, a lot of the language constructs that we regularly use in Elixir such as def, defmodule, if, and others are macros. Furthermore, many popular libraries like Phoenix, Ecto, and Absinthe use macros liberally to create convenient developer experiences.

Here’s an example Ecto query from the documentation:

query = from u in "users",
          where: u.age > 18,
          select: u.name

Metaprogramming in Elixir is a powerful tool. It approaches LISP (the OG metaprogramming steam roller) in expressivity but keeps things one level above in abstraction, enabling you to delve into AST only when you need to. In other words, Elixir is basically LISP but readable. 🙃

Getting started

So how do we channel this immense power? 🧙

While metaprogramming can be rather tricky, it is rather simple to start metaprogramming in Elixir. All you need to know are three things.

quote	unquote	defmacro
code -> AST	outside quote -> inside quote	AST -> AST

quote

quote converts Elixir code to its internal AST representation.

You can think of the difference between regular and quoted expressions to be the difference in two different requests.

• Say your name, please. Here, the request is to reply with your name.
• Say “your name”, please. Here, the request is to reply with the internal representation of the request in the language – “your name”.

iex(1)> 2 + 2
4
iex(2)> quote do 2 + 2 end
{:+, [context: Elixir, import: Kernel], [2, 2]}

quote makes it a breeze to write Elixir macros since we don’t have to generate or write the AST by hand.

unquote

But what if we want to have access to variables inside quote? The solution is unquote.

unquote functions like string interpolation, enabling you to pull variables into quoted blocks from the surrounding context.

Here’s how it looks in Elixir:

iex(1)> two = 2
2
iex(2)> quote do 2 + 2 end
{:+, [context: Elixir, import: Kernel], [2, 2]}
iex(3)> quote do two + two end
{:+, [context: Elixir, import: Kernel],
 [{:two, [], Elixir}, {:two, [], Elixir}]}
iex(4)> quote do unquote(two) + unquote(two) end
{:+, [context: Elixir, import: Kernel], [2, 2]}

If we don’t unquote two, we will get Elixir’s internal representation of some unassigned variable called two. If we unquote it, we get access to the variable inside the quote block.

defmacro

Macros are functions from ASTs to ASTs.

For example, suppose we want to make a new type of expression that checks for the oddity of numbers.

We can make a macro for it in just a few lines with defmacro, quote, and unquote.

defmodule My do
  defmacro odd(number, do: do_clause, else: else_clause) do
    quote do
      if rem(unquote(number), 2) == 1, do: unquote(do_clause), else: unquote(else_clause)
    end
  end
end

iex(1)> require My
My
iex(2)> My.odd 5, do: "is odd", else: "is not odd"
"is odd"
iex(3)> My.odd 6, do: "is odd", else: "is not odd"
"is not odd"

When should you use metaprogramming?

"Rule 1: Don’t Write Macros" – Chris McCord, Metaprogramming in Elixir

While metaprogramming can be an awesome tool, it should be used with caution.

Macros can make debugging much harder and increase overall complexity. They should be turned to only when it is necessary – when you run into problems you can’t solve with regular functions or when there is a lot of plumbing behind the scenes that you need to hide.

When used correctly, they can be very rewarding, though. To see how they can improve developer life, let's look at some real-life examples from Phoenix, the main Elixir web framework.

How macros are used in Phoenix

In the following section, we’ll analyze the router submodule of a freshly made Phoenix project as an example of how macros are used in Elixir.

use

If you look at the top of basically any Phoenix file, you will most likely see a use macro. Our router submodule has one.

defmodule HelloWeb.Router do
  use HelloWeb, :router

What this one expands to is:

require HelloWeb
HelloWeb.__using__(:router)

require asks HelloWeb to compile its macros so that they can be used for the module. But what’s using? It, as you might have guessed, is another macro!

  defmacro __using__(which) when is_atom(which) do
    apply(__MODULE__, which, [])
  end 

In our case, this macro invokes the router function from the HelloWeb module.

  def router do
    quote do
      use Phoenix.Router

      import Plug.Conn
      import Phoenix.Controller
    end
  end

router imports two modules and launches another __using__ macro.

As you can see, this hides a lot, which can be both a good and a bad thing. But it also gives us access to a magical use HelloWeb, :router to have everything ready for quick webdev action whenever we need.

pipeline

Now, look below use.

  pipeline :browser do
    plug :accepts, ["html"]
    plug :fetch_session
    plug :fetch_flash
    plug :protect_from_forgery
    plug :put_secure_browser_headers
  end

Yup, more macros.

pipeline and plug define pipelines of plugs, which are functions of functions that transform the connection data structure.

While the previous macro was used for convenient one-line imports, this one helps to write pipelines in a very clear and natural language.

scope

And, of course, the routing table is a macro as well.

  scope "/", HelloWeb do
    pipe_through :browser

    get "/", PageController, :index

  end

scope, pipe_through, get – all macros.

In fact, the whole module is macros and an if statement (which is a macro) that adds an import statement and executes a macro.

I hope that this helps you see how metaprogramming is at the heart of Elixir.

Now, let’s try to build our own macro.

Build your own Elixir macro

Elixir and currying don’t really vibe together. But with some effort, you can create a curried function in Elixir.

Here’s a regular Elixir sum function:

def sum(a, b), do: a + b

Here’s a curried sum function:

def sum() do
  fn x ->
    fn y ->
      x + y
    end
  end
end

Here’s how they both behave:

iex(1)> Example.sum(1,2)
3
iex(2)> plustwo = Example.sum.(2)# Function<10.76762873/1 in Example.sum/0>
iex(3)> plustwo.(2)
4

Let’s say that we want to use curried functions in Elixir for some reason (for example, we want to create a monad library). Writing out every function in our code like that would be, to say the least, inconvenient.

But with the power of metaprogramming, we can introduce curried functions without a lot of boilerplate. Let’s define our own defc macro that will define curried functions for us.

First, we need to take a look at how a regular def looks as an AST:

iex(1)> quote do def sum(a, b), do: a + b end
{:def, [context: Elixir, import: Kernel],
 [
   {:sum, [context: Elixir], [{:a, [], Elixir}, {:b, [], Elixir}]},
   [
     do: {:+, [context: Elixir, import: Kernel],
      [{:a, [], Elixir}, {:b, [], Elixir}]}
   ]
 ]}

It is a macro with two arguments: the function definition (in this case, sum is being defined) and a do: expression.

Therefore, our defc (which should take the same data) will be a macro that takes two things:

1. A function definition, which consists of the function name, context, and supplied arguments.
2. A do: expression, which consists of everything that should be done with these arguments.

defmodule Curry do
  defmacro defc({name, ctx, arguments} = clause, do: expression) do
  end
end

We want the macro to define two functions:

1. The function defined in defc.
2. A 0-argument function that returns the 1st function, curried.

  defmacro defc({name, ctx, arguments} = clause, do: expression) do
    quote do
      def unquote(clause), do: unquote(expression)
      def unquote({name, ctx, []}), do: unquote(body)
    end
  end

That’s more or less the macro. Now, we need to generate the main part of it, the body.

To do that, we need to go through the whole argument list and, for each argument, wrap the expression in a lambda.

  defp create_fun([h | t], expression) do
    rest = create_fun(t, expression)

    quote do
      fn unquote(h) -> unquote(rest) end
    end
  end

  defp create_fun([], expression) do
    quote do
      unquote(expression)
    end
  end

Then, we assign the variable body in the macro to be the result of create_fun, applied to the arguments and the expression.

  defmacro defc({name, ctx, arguments} = clause, do: expression) do
    body = create_fun(arguments, expression)

    quote do
      def unquote(clause), do: unquote(expression)
      def unquote({name, ctx, []}), do: unquote(body)
    end
  end

That’s it! 🥳

To try it out, let’s define another module with a sum function in it.

defmodule Example do
  import Curry

  defc sum(a, b), do: a + b

end

iex(1)> Example.sum(2,2)
4
iex(2)> Example.sum.(2).(2)
4
iex(3)> plustwo = Example.sum.(2)# Function<5.100981091/1 in Example.sum/0>
iex(4)> plustwo.(2)
4

You can see the full code here.

In our example, the macro provides only the sum() and sum(a,b) functions. But from here, it’s easy to extend our macro to generate partial functions for all arities.

In the case of sum, we can make it so that the macro generates sum(), sum(a), and sum(a,b), modifying the function definition to account for the missing arguments.

Since it is an awesome exercise to try on your own, I will not spoil the answer. 😊

Further learning

If you want to learn more about macros in Elixir, here are a few resources I suggest to check out:

• Metaprogramming in Elixir. Chris McCord’s book gives a detailed introduction to macros with awesome and practical code examples.
• Understanding Elixir Macros. This article series by Saša Jurić covers the topic in much more detail than I do, and it’s a nice read if you don’t have time to read Chris’s book.
• Don’t Write Macros But Do Learn How They Work. If you prefer to watch videos, this is a nice talk by Jesse Anderson.