Writing an Erlang Port using OTP Principles

1. Overview

Introduction 

There are several different mechanisms that are available when trying to make Erlang interact with the external world. These are described in the official Erlang documentation in the Interoperability Tutorial. This is a tutorial on how to use one of those techniques, the Erlang port. The source code for the tutorial can be found here: port-howto.tar.gz

The tutorial demonstrates how to communicate to an external program using a standard Erlang port. A simple echo server, written in Python, will be used in the example. The port will enable us to send Erlang messages to the echo server via its standard input and standard output. The echo server will read data from standard input and echo it back, along with a time stamp, to our Erlang port.

In addition, the example utilizes several standard OTP behaviors such as gen_servers, supervisors, and applications for completeness. We want to ensure that our application is robust in the event that the external echo server crashes or does not respond in a timely manner. The use of standard OTP behaviors enables us to do so with minimal effort.

The Big Picture 

A high-level overview of the components is presented in this section. It is intended to be the road map for the rest of the document. Subsequent chapters provide additional detail on each of the components as well as the source code for each. Lets begin with a diagram illustrating the big picture.

Figure 1.1: The Big Picture

The diagram depicts two OS-level processes, an Erlang virtual machine and a Python interpreter, communicating with each other using standard OS pipes via stdin and stdout. These two processes are completely independent of each other in all other respects and are treated individually by the OS. On the other hand, all of the Erlang processes reside within the same OS-level process, the Erlang virtual machine.

Within the Erlang virtual machine, there are three client processes, a supervisor, a gen_server, and a port to the external Python process. You'll also notice that three of the components have been co-located with each other in the box labeled echo_app. These components represent the OTP application that we are going to be build. The use of an OTP application allows us to start and stop all of the processes associated with our echo server as a single entity. In addition, it'll enable others to incorporate our application in other systems with minimal effort.

Because we are implementing a centralized server that is going to handle and respond to client requests, the use of the OTP gen_server behavior will greatly simplify our implementation. The gen_server behavior provides a framework for modeling client-server relationships where a server is managing a resource that is going to be shared between one or more clients. In this case, the gen_server will be responsible for managing access to the port that is communicating with the external Python process.

Ports can be viewed as external Erlang processes. They can be used to communicate with external OS-level processes or access any opened file descriptors in use by the Erlang VM. Messages can be sent and received from ports just as if they were any other Erlang process. In this tutorial, messages sent to the port will be sent to our external Python process. Likewise, when the Python process sends data to the Erlang VM, a message will be sent back to the port's controlling process, the process that opened the port. In this implementation, the gen_server is responsible for the opening and closing of the port. This will ensure that all messages sent from the port arrive at the gen_server process.

By default, external processes spawned via the opening of an port communicate to the Erlang VM via standard input and standard output. Our Python process will read lines of data via its stdin, and then write responses back via stdout.

Clients will use a specified API, echo/1, to make requests to our echo server. By using this API, clients will be insulated from the implementation details of our echo server. The client does not need to know it is communicating with another process. This allows us to change our implementation later if we choose.

Finally, in order to provide a reliable service, we will employ the use of a supervisor to monitor our gen_server process. Within the OTP framework, supervisors monitor the behavior of worker processes, and can restart them in the event something goes wrong. As you will see, our gen_server process is linked to the port, and as a result, if the port (or the external Python process) terminates abnormally, our gen_server process will also terminate, forcing the supervisor to restart the gen_server and thus the external Python process.

Subsequent chapters will explore each of these components and their implementations in detail. Before moving on, we should take a brief moment to discuss the standard directory layout of an OTP application and where each of the components that are discussed in this tutorial should reside. Here is the layout for this project:

`-- echo_app-1.0
    |-- ebin
    |   |-- echo.beam
    |   |-- echo_app.app
    |   |-- echo_app.beam
    |   `-- echo_sup.beam
    |-- priv
    |   `-- echo.py
    `-- src
        |-- echo.erl
        |-- echo_app.erl
        `-- echo_sup.erl
        

All of the Erlang source code should reside in the src directory, the Python program should reside in the priv directory, and the compiled sources and application definition file should be in the ebin directory. It's now time to start digging into the details of the tutorial.

2. External Python Echo Server

Design 

Lets start with the design of the external Python echo server. As the previous chapter already mentioned, the Erlang VM will communicate with this process via an Erlang port, which by default will use the standard input and standard output of the Python process to communicate to the port. Requests will be processed one at a time, and will yield a response before the next request is processed. Communication is therefore synchronous.

There are a few things to consider when implementing our external Python process. First, we have decided to use the standard IO mechanisms of Python which utilize the stdio library. Therefore, we must be aware that stdin and stdout will be fully buffered because we are communicating with Erlang via standard OS-level pipes. For those that do not recall, the stdio library uses line buffering on file objects associated with a terminal device, and fully buffers everything else with the exception that stderr is never buffered.

The second item to consider is the communication protocol used between the Erlang port and the Python process. Although we have already decided that communication will occur over stdin and stdout, we have yet to discuss the exact format of the requests that will arrive on stdin, and the exact format of the responses that will be sent back via stdout. We will explore this topic in the next section.

Third, what should we do in the event of an error? Do we try and write a robust Python server that catches every single possible error? Or do we let our Python process terminate upon any error? For the purpose of this tutorial, we will allow our Python process to terminate upon any error because we will design our Erlang port to terminate our gen_server process which will then force the supervisor to restart everything including the external Python process. I'm new to Erlang, but it seems that this is the most common approach to programming within Erlang, let things fail, and let your supervisors correct the problem.

In the next section, the communication protocol between the external server and port is discussed in detail.

Communication Protocol 

The communication protocol used between our Erlang application and this external Python process will be line oriented. Requests will arrive on stdin as a single line per request. The server will read and process one request at a time. After the request has been processed, one or more lines will be sent back via stdout. The Erlang process must continue reading data from the process until a line with a single OK has been returned. This will indicate the end of the current response.

For the purpose of this tutorial, the Python server will simply respond with three lines of data: the original request, the current time, and a single OK. It should be noted that we are not limited to three lines of output, and that the protocol could be used with some other process that adheres to it. Here is a sample interaction with the Python echo server (the emphasized lines are user input):

kaz@coco:lib/echo_app-1.0/priv$ ./echo.py
Hello there!
Received Hello there!
Current time is Mon Jan 23 16:51:43 2006
OK
Is anyone home?
Received Is anyone home?
Current time is Mon Jan 23 16:51:48 2006
OK
^D  # Control-D was pressed to close stdin
        

As you can see, the Python echo server lives up to its name. It simply echoes back whatever it received along with some additional data to make the example a little more interesting by forcing us to read an arbitrary amount of response data. With that said, lets move on to the implementation of the echo server.

Implementation 

Here is the full implementation of the Python echo server. Do not forget to place this source file in the priv directory of your directory layout. Although Python was chosen as the implementation language, even those that are not familiar with Python should not have any problem understanding this trivial program.

#!/usr/bin/python

import sys
import time

while 1:
    line = sys.stdin.readline()
    if not line: break

    # Send back the received line
    print "Received", line,

    # Lets send back the current time just so we have more than one
    # line of output.
    print "Current time is", time.ctime()

    # Now send our terminating string to end the transaction.
    print "OK"

    # And finally, lets flush stdout because we are communicating with
    # Erlang via a pipe which is normally fully buffered.
    sys.stdout.flush()
        

As you can see, the Python process sits in a loop forever reading one line at a time from stdin. The line is then echoed back along with a time stamp followed by our terminator string. There are two items of importance, both of which deal with buffering issues.

First, note the call to sys.stdout.flush(). This forces the stdio library to flush the current stdout buffer. It ensures that Erlang will receive a response to the request. If this was not provided, our Erlang server may hang indefinitely if we did not code it in a robust manner (which we will do to protect ourselves from this scenario should it happen).

Second, which is a bit more subtle, and only of interest to those that are familiar with Python's idiom of reading lines for a file object. The normal way to process one line at a time is with the following idiom:

#!/usr/bin/python

import sys

for line in sys.stdin:
    # Do something with line here
    print line
        

Python uses an internal buffering when reading from file-object iterators such as the one above. What does this mean? Even though the Erlang VM may have sent a line, the Python process may be buffering it internally, thus blocking forever. This internal buffer can be avoided by using the technique in our implementation of the Python echo server. This ensures that Python has access to the data as soon as Erlang sends it across the pipe. Again, this is an implementation note to those that may already know Python.

An Aside 

Although the Python echo server is trivial, it was used as an example because it could be easily replaced with any number of other programs that follow the same communication protocol. One such example is RRDTool which happens to be the author's original motivation for writing this tutorial. In fact, as you'll see, which external program to be spawned can be specified as a system configuration parameter to our OTP application.

3. Client API

Design 

Clients will access the server through an API call to echo:echo/1 instead of directly sending messages to the gen_server. The function takes a single string argument which will eventually be processed by our external Python echo server. By providing an API call, we provide a layer of indirection that will hide implementation details the client need not be aware of.

This function will return a list of lists. Each line of output generated by our Python echo server will yield a list of one or more strings depending on whether the line of output exceeds the buffer size specified when the port is opened. This will be explained in more detail in the next chapter. Recall that the Python server sends back a line containing the original request, as well as a line containing the current time stamp. Lets look at some output from the use of echo:echo/1:

kaz@coco$ erl -boot echo -boot_var MYAPPS ~/port_example
Erlang (BEAM) emulator version 5.4.6 [source] [threads:0]

Eshell V5.4.6  (abort with ^G)
1> echo:echo("This is a test of the emergency broadcast system\n").
[["Received This is a test of the emergency broadcast system"],
 ["Current time is Wed Jan 25 11:13:40 2006"]]
2>
        

You may be wondering why the function simply does not return a list of strings instead. The reason, as hinted above, is due to the buffer size associated with the port. If the Python server sends back a line that is longer than this buffer, the gen_server will send back multiple strings per line. Lets take a look at the same example, but this time we'll start our application with a different buffer size:

kaz@coco$ erl -boot echo -boot_var MYAPPS ~/port_example -echo_app maxline 20
Erlang (BEAM) emulator version 5.4.6 [source] [threads:0]

Eshell V5.4.6  (abort with ^G)
1> echo:echo("This is a test of the emergency broadcast system\n").
[["Received This is a t","est of the emergency"," broadcast system"],
 ["Current time is Wed ","Jan 25 14:15:36 2006"]]
2>
        

And now you can see why a list of lists is returned instead of a list of strings. Why return an unflattened list of strings? It appears that this is a common idiom in Erlang for performance reasons. This is not a problem because most of the IO routines can operate on them as is. For example, to print out the responses sent back from the server:

2> Result = echo:echo("This is a test of the emergency broadcast system\n").
[["Received This is a t","est of the emergency"," broadcast system"],
 ["Current time is Wed ","Jan 25 14:27:33 2006"]]
3> Print = fun(Line) -> io:format("~s~n", [Line]) end.
#Fun<erl_eval.6.43886099>
4> lists:foreach(Print, Result).
Received This is a test of the emergency broadcast system
Current time is Wed Jan 25 14:27:33 2006
ok
5>
        

Although we hide the implementation of our server from the client by providing an API, there are a few details that we must enumerate. First, what happens if our server process does not send a response in a timely manner? If we do nothing, the client could end up waiting indefinitely. Clearly, this is a less than ideal situation. For this reason, it is imperative that we use a timeout when waiting for a response. As it turns out, the OTP folks have already thought of this and have provided the appropriate mechanism in gen_server:call/3, which is used to make a synchronous call to a gen_server. The last argument of that function specifies the amount of time to wait for a response before a timeout occurs. We'll see this in the next section.

And second, what happens if the client sends a request that is not terminated with a newline? The external Python server reads data from its standard input one line at a time. If a request arrives that is not properly terminated, the Python server will hang until another request eventually arrives from another client that is terminated with a newline. And, what about when a request contains more than one newline? The Python server will send back multiple responses, which will arrive at the gen_server which is expecting only a single response. We will deal with both of these issues before the request is ever sent to the gen_server.

Lets move on to the implementation and see how all of these issues are dealt with in the code.

Implementation 

The implementation of the client API is provided in the same module as our gen_server code (this is considered good coding style within the Erlang community). Rather than present the entire module here, which we have not discussed yet, only the relevant portions will be extracted. The rest of the module will be presented in the next chapter when the implementation of the gen_server is discussed.

-module(echo).
-author('pete-trapexit@kazmier.com').

%% API functions
-export([echo/1]).

echo(Msg) ->
    ValidMsg1 = case is_newline_terminated(Msg) of
                    true  -> Msg;
                    false -> erlang:error(badarg)
                end,
    ValidMsg2 = case count_chars(ValidMsg1, $\n) of
                    1     -> ValidMsg1;
                    _     -> erlang:error(badarg)
                end,
    
    gen_server:call(?MODULE, {echo, ValidMsg2}, get_timeout()).

get_timeout() ->
    {ok, Value} = application:get_env(echo_app, timeout),
    Value.

is_newline_terminated([])    -> false;
is_newline_terminated([$\n]) -> true;
is_newline_terminated([_|T]) -> is_newline_terminated(T).
    
count_chars(String, Char) ->
    length([X || X <- String, X == Char]).
        

Both issues described above have been addressed in this implementation. First, the argument to echo:echo/1 is checked to ensure that it is valid, containing only a single terminating newline. If the argument is not valid, an error is thrown with a reason of badarg. This prevents the possibility that the external Python server will hang forever waiting for a terminating newline, and will prevent the server from sending multiple responses for a single request.

Second, the call to gen_server:call/3 is passed a timeout value, which is pulled from our application's configuration variables (the same mechanism that let us change the buffer size on the command line in the previous section). These variables are described in more detail in a subsequent chapter on the OTP application implementation. The timeout value ensures the client does not block forever while waiting for a response from the server. Note the format of the gen_server request, {echo, Msg}, as it will be used later when matching function clauses in the implementation of the gen_server.

That completes the implementation of the client API. Clients can now use it to invoke the services of our external Python echo server. The next chapter describes the heart of our architecture, the implementation of the gen_server and the port.

4. Gen_server and the Port

Design 

It's now time to turn our attention to the crux of the architecture, the gen_server process. Erlang's gen_server behavior module provides a framework to simplify the development of client-server applications. In the context of this tutorial, clients make requests to the server in order to obtain access to the external Python echo server. It is the responsibility of the gen_server process to mediate this interaction.

By implementing this behavior, the programmer is agreeing to provide implementations for several callback functions that are invoked during the life cycle of the gen_server process. These callback functions are described below:

• init/1
  This function is called whenever the gen_server process has been started by calling either gen_server:start or gen_server:start_link. It should be noted that this function is executed in a new gen_server process that is spawned, and not the process that called the previous two functions.
• handle_call/3
  This function is called to handle the arrival of a synchronous request whenever a client makes a call using gen_server:call which we do in our implementation of echo/1. The gen_server is expected to return a response to the requester. It is this callback where we will process client requests.
• handle_cast/2
  This function is called to handle the arrival of an asynchronous request whenever a client makes a call using gen_server:cast. The gen_server process does not send a response to the client in this callback. This callback is not used in our implementation, but we'll provide an implementation anyways to play nicely with the OTP framework.
• handle_info/2
  This function is called to handle the arrival of any other messages to the gen_server. In our implementation, we will take advantage of this callback to monitor the status of our port and external Python server. Should the external server fail, a message will be sent to the controlling process, our gen_server. This is the callback that will process that message.
• terminate/2
  This function is called whenever the gen_server is about to terminate as long as the gen_server has been set to trap exit signals. In our scenario, termination might be initiated by the gen_server itself should it detect an error with the external Python server, or it might be terminated in response to our supervisor instructing it to shutdown.
• code_change/3
  This function is used during release upgrades and downgrades. We do not implement this feature of the OTP library and thus we only provide a default implementation to play nicely with the rest of the framework.

When our gen_server is initialized, it will open the port to the external Python server. By opening the port in the gen_server process, we are ensuring that all errors that arise with the port are sent back to our gen_server process (which are handled via the gen_server:handle_info/2 callback) so it can take appropriate action. Once the port is opened in init/1, it is passed around as the state of the gen_server. This enables us to access the port in any of the callback functions.

Ports can be opened with a list of options that control the behavior of the port. All of these are described in the erlang:open_port/2 man page. We will only require the use of two of them (and one of those is actually a default):

• stream
  This option specifies that output messages are sent without packet lengths. By using this option, we must define our own protocol between Erlang and the external process. This was described in detail in a previous chapter. It should be noted that this option is default when opening a stream.
• {line, L}
  This option specifies that messages are delivered on a per-line basis. L is the maximum line length. Lines that exceed this length will be delivered in multiple messages. The format of the messages is {data, {Flag, Line}}, where Flag is either eol or noeol depending on whether or not the maximum line length has been exceeded. If the line length has been exceeded, all but the last message will set Flag to noeol.

After the initialization of the gen_server and the port is completed, processing of client requests entails the sending of a command to the port along the argument that was passed to echo/1 as data. The command is then sent to the external Python server via stdin. While the Python server is processing the request, the gen_server will block until a response has been sent back from the port, or until a timeout occurs. Once the Python process sends a response to stdout, the Erlang VM delivers it to our gen_server as one or more messages from the port.

In reality, at the minimum, it requires three messages: one for the line containing the echoed response, one for the line with the time stamp, and one for the final OK line. It could be more if any of the lines exceed the maximum line length. Lets look at the message flow from a port that has been opened with {line, 45}. If the Python process sends the following data:

Received some data
Fri Jan 27 09:55:52 EST 2006
OK
        

The port will send the following three messages:

{data, {eol, "Received some data"}}
{data, {eol, "Current time is Fri Jan 27 09:55:52 EST 2006"}}
{data, {eol, "OK"}}
        

However, if the Python process sends:

Received some data as well as some extraneous garbage for illustration
Fri Jan 27 09:55:52 EST 2006
OK
        

The port will send the following four messages:

{data, {noeol, "Received some data as well as some extraneous"}}
{data, {eol, "garbage for illustration"}}
{data, {eol, "Current time is Fri Jan 27 09:55:52 EST 2006"}}
{data, {eol, "OK"}}
        

An extra message is sent this time because the line length exceeded the maximum size of the buffer. We must code for this scenario.

Now that we have discussed the message passing between the port and the gen_server, lets discuss what happens in the event of an error. There are two error conditions that must be addressed. First, what should be done in the event the Python process terminates? And second, what should we do if the Python process takes too long to respond? The answer is surprisingly simple. In both cases, the error condition in detected in the gen_server process, and then the process is terminated.

That may not sound very robust at first glance. However, our implementation utilizes another standard OTP behavior: the supervisor. It is the job of the supervisor to monitor the livelihood of the gen_server process. Should the process terminate for any reason, the supervisor will simply start a new gen_server process to replace the old one. And after the gen_server has initialized, a new Python server would have been spawned and ready to process requests sent by clients via the gen_server.

In this section, you have learned how the gen_server communicates with clients and the callbacks used to process the requests. Likewise, you also learned how the gen_server communicates with the Python process via a standard Erlang port. Finally, we discussed what happens when something goes wrong. In the next section, the implementation of the gen_server is presented.

Implementation 

Now that you have read about the design of the gen_server process, it is now time to move on to the implementation. Rather than present the entire module as a whole, it is broken down into functional pieces. In addition, the client API implementation has been left out as it was already discussed. If you prefer, the entire source can be found in the tarball of the entire source tree.

Without further delay, lets start with the module attributes and data structures used throughout the gen_server process:

-module(echo).
-behavior(gen_server).

%% External exports
-export([start_link/1]).

%% API functions
-export([echo/1]).

%% gen_server callbacks
-export([init/1, 
         handle_call/3,
         handle_cast/2, 
         handle_info/2,
         code_change/3,
         terminate/2]).

%% Server state
-record(state, {port}).
        

As you can see, nothing complicated here. The only part worth commenting on is the record that is used to represent the server state. Since there is only a single field in the record, one could argue that the usefulness of the record is zero. However, by using the record, it becomes trivial to add an additional piece of state later to the implementation without the need to change code in several places.

The next set of functions are used when starting the gen_server process:

start_link(ExtProg) ->
    gen_server:start_link({local, ?MODULE}, echo, ExtProg, []).

init(ExtProg) ->
    process_flag(trap_exit, true),
    Port = open_port({spawn, ExtProg}, [stream, {line, get_maxline()}]),
    {ok, #state{port = Port}}.

get_maxline() ->
    {ok, Value} = application:get_env(echo_app, maxline),
    Value.
        

Although we have not discussed the supervisor in detail yet, start_link/1 is called by the supervisor to start the gen_server process. Then gen_server:start_link/4 eventually invokes the init/1 callback in our gen_server after a new process has been spawned by the supervisor. This new process is linked to the supervisor, which allows the supervisor the ability to monitor the status of the gen_server.

The port is created when init/1 is called during the start up of the gen_server. The name of the external program to start is passed as an argument. This will be specified by the application module which is discussed later. The list of port options should look familiar. The maximum line length is supplied by querying the application configuration parameters (again, discussed later in the application chapter).

It is important to note that the gen_server is trapping exits. This enables terminate/2 to be invoked when the gen_server is about to terminate. If the process does not trap flags, it will not be called when the supervisor orders it to terminate.

Now, lets move on to the heart of the gen_server, the processing of client requests. Here is the code:

handle_call({echo, Msg}, _From, #state{port = Port} = State) ->
    port_command(Port, Msg),
    case collect_response(Port) of
        {response, Response} -> 
            {reply, Response, State};
        timeout -> 
            {stop, port_timeout, State}
    end.

collect_response(Port) ->
    collect_response(Port, [], []).

collect_response(Port, RespAcc, LineAcc) ->
    receive
        {Port, {data, {eol, "OK"}}} ->
            {response, lists:reverse(RespAcc)};

        {Port, {data, {eol, Result}}} ->
            Line = lists:reverse([Result | LineAcc]),
            collect_response(Port, [Line | RespAcc], []);

        {Port, {data, {noeol, Result}}} ->
            collect_response(Port, RespAcc, [Result | LineAcc])

    %% Prevent the gen_server from hanging indefinitely in case the
    %% spawned process is taking too long processing the request.
    after get_timeout() -> 
            timeout
    end.

get_timeout() ->
    {ok, Value} = application:get_env(echo_app, timeout),
    Value.
        

If you recall, handle_call/3 is invoked when the gen_server receives a request from a client. The appropriate clause is matched based on the request that was sent by gen_server:call/3 in the client API. In this case, the request from echo:echo/1 is {echo, Msg}. Upon receipt of the request, the gen_server sends a command to the port via the built-in function port_command/2, and then collects the response, if any, that was sent back from the port. Alternatively, one could send a message of the form {command, Data} to the port using ! syntax instead of calling port_command/2. Only the handling of errors is different between the two forms. See the online documentation for port_command/2 for additional details..

The port's various types of messages are collected into a single response that can be sent back to the client. If a response is returned, the callback returns {reply, Response, State} which instructs the gen_server to send the response to the client that had originally invoked gen_server:call/3. However, if a timeout has occurred, then the callback returns {stop, port_timeout, State}. This instructs the gen_server to terminate and invoke the terminate/2 callback function with an argument of port_timeout which is defined next:

handle_info({'EXIT', Port, Reason}, #state{port = Port} = State) ->
    {stop, {port_terminated, Reason}, State}.

terminate({port_terminated, _Reason}, _State) ->
    ok;
terminate(_Reason, #state{port = Port} = _State) ->
    port_close(Port).
        

Recall, if the Python server terminates for any reason, the port sends an exit message back to the gen_server. We handle this message with the handle_info/2 callback which returns {stop, {port_terminated, Reason}, State}. Again, this instructs the gen_server to terminate and call the appropriate terminate/2 clause. Notice that two clauses have been specified in our implementation of terminate/2. The first clause matches if the external Python process terminated. There is no need to close the port in this case as it's already been closed. However, if a timeout occurred or any other unforeseen error occurred, we explicitly close the port thereby terminating the external Python process as well. And when the gen_server terminates, its supervisor will simply start a new one (more details on this later).

Finally, for completeness, here are the last two callback functions that are not utilized in our tutorial. Default implementations are provided for each:

handle_cast(_Msg, State) ->
    {noreply, State}.

code_change(_OldVsn, State, _Extra) ->
    {ok, State}.
        

That concludes the implementation of the gen_server. In the next section, the role and implementation of the supervisor is discussed.

5. Supervisor

Design and Implementation 

In the previous chapter, you learned that our gen_server process may terminate for a variety of reasons. Should the process die, it is the responsibility of the supervisor to detect and take appropriate action such as restarting it. Due to the simplicity of our application, a single supervisor will be sufficient to monitor the lone worker, the gen_server process. In terms of a hierarchy, the supervisor sits above the worker overseeing its behavior. A supervisor may in turn monitor another supervisor, and so forth. This hierarchical structure of supervisors and workers is referred to as a supervision tree. Supervision trees allow us to design fault-tolerant and robust software systems.

A supervisor is a behavior, and we must implement its callback functions just as we did with the gen_server behavior in the previous chapter. Fortunately, this behavior only requires the implementation of a single callback function, init/1. This callback is called by supervisor:start_link which is used to start and create the supervisor by the OTP application (as you'll see in the next section). The return value of the callback specifies the supervisor's restart strategy, maximum restart frequency, and the list of children to be supervised. This is represented by the the following tuple: {ok, {{RestartStrategy, MaxR, MaxT}, [ChildSpec]}}. The details of which can be found in the OTP Design Principles guide.

Here is the implementation of the supervisor:

-module(echo_sup).
-behavior(supervisor).

%% External exports
-export([start_link/1]).

%% supervisor callbacks
-export([init/1]).

start_link(ExtProg) ->
    supervisor:start_link(echo_sup, ExtProg).

init(ExtProg) ->
    {ok, {{one_for_one, 3, 10},
          [{echo, {echo, start_link, [ExtProg]},
            permanent, 10, worker, [echo]}]}}.
        

For this tutorial, we will use a one_for_one restart strategy. This strategy instructs the supervisor to restart only the process that has terminated. Other strategies exist that instruct the supervisor to restart all processes if one terminates. In our supervision tree, there is only a single worker, so the choice of restart strategy is not as important as it would be if there were multiple workers. As for the maximum restart frequency, a safeguard to prevent the supervisor from continuously spawning a misbehaving worker, a frequency of no more than 3 times (MaxR) in 10 seconds (MaxT) will be used.

The return value of init/1 also includes a list of child specifications for each of the workers it is supposed to supervise. The specification is a tuple that includes an id, start function (tuple of module, function, and initial arguments), restart type (permanent, temporary, or transient), shutdown timeout, role (worker or supervisor), and a list containing the name of the callback module.

We'll specify a a restart type of permanent for our gen_server. This instructs the supervisor to restart the process each time it exits unless it has exceeded the maximum restart frequency. If you recall the gen_server implementation, echo:start_link/1 was provided for the supervisor to invoke, and thus is specified as the start up function. Also notice the name of the external program that the port will start, ExtProg, is passed by the caller of echo_sup:start_link/1 which originates from the application as you'll see in the next chapter.

6. Application

Design and Implementation 

Within the OTP framework, the standard method of bundling one's code into a reusable component which can be started and stopped as a single entity is called an application. Not all applications require starting and stopping, it is quite possible to have an application that only provides a library of functions. This type of application is called a library application. In the context of this tutorial, we do require the starting and stopping of our application which consists of the supervisor process, gen_server process, and the port. By creating an application, it will be easy for others to incorporate our application into their own OTP applications. For more information on applications, please refer to the OTP Design Principles guide.

Again, like the supervisor and gen_server before that, an application is implemented as a callback module. This callback module is relatively trivial compared to that of the gen_server. It requires the implementation of two callback functions: start/2 and stop/1. When the application is started, it is expected to return a state value and the pid of the topmost supervisor of the supervision tree. The arguments to start/2 and stop/1 are not used in this tutorial so we can safely ignore them. Lets take a look at the implementation of the application:

-module(echo_app).

-behavior(application).

%% application callbacks
-export([start/2, 
         stop/1]).

start(_Type, _Args) ->
    PrivDir = code:priv_dir(echo_app),
    {ok, ExtProg} = application:get_env(echo_app, extprog),
    echo_sup:start_link(filename:join([PrivDir, ExtProg])).

stop(_State) ->
    ok.
        

The implementation of both callbacks is trivial. In order to start the topmost supervisor, our application callback module invokes echo_sup:start_link/1 which returns {ok, Pid}. This is passed back as the return value of start/2 since it adheres to the contract of that callback function. As you may have also noticed, the name of the external program to be spawned is obtained by querying the application's environment (or configuration variables). These variables can be defined in several places: an application resource file (discussed below), a system configuration file (not discussed), or passed on the command line (discussed in the next chapter). In addition, the path to the executable is created by retrieving the path to our application's priv directory using the code:priv_dir/1 built-in function.

The final step in creating the application is to define an application resource file which is a file that contains one tuple of the form {application, ApplicationName, [Options]}. This tuple is used to specify metadata about the application such as dependencies, version, description, configuration variables, etc. This is the application resource file used for our application.

{application, echo_app,
 [{description, "Echo Port Server"},
  {vsn, "1.0"},
  {modules, [echo_app, echo_sup, echo]},
  {registered, [echo]},
  {applications, [kernel, stdlib]},
  {mod, {echo_app, []}},
  {env, [{extprog, "echo.py"}, {timeout, 3000}, {maxline, 100}]}
 ]}.
        

Here is the list of options that were defined and a brief explanation of each:

• description
  A brief description of the application. This is used when querying the system about loaded and started applications.
• vsn
  A version number as a string.
• modules
  A list of modules that are part of this application. We created three separate modules, one for the gen_server, one for the supervisor, and one for the application. All must be listed.
• registered
  A list of all registered processes used by our application. It is used by the system to help identify naming conflicts between processes.
• applications
  A list of all applications that our application is dependent upon. At the very least, all applications must list kernel and stdlib as dependencies. Aside from the standard applications, no others are required for our application.
• mod
  Specifies the name of the application callback module and arguments to be passed to the start/2 callback function. The module name does not have to be the same name as the name of the application; however, in this example, the application callback module does use the same name as the application itself.
• env
  A list of configuration parameters or variables to be used as defaults. These parameters can be queried via application:get_env/2. In addition, they may also be overridden by a system configuration file or command line arguments. You should recognize the three parameters specified here as they were used throughout the implementation.

With the application callback module defined, and the application resource file created, you are now ready to build and run the application.

7. Running the Application

Building the Application 

Obviously, before we can run the application, we must make sure that it has been compiled. Assuming you've already unpacked the tarball in a directory, precompiled beam files already exist in the ebin directory. However, if you do want to build the code yourself, you might do so as follows:

kaz@coco:~/port_example/lib/echo_app-1.0/src$ erlc -W -o ../ebin *.erl
        

It is important to make sure that all compiled files end up in the ebin directory and not the src directory otherwise Erlang will not be able to find them at runtime. If the code built without errors (it should), you are now ready to start and test the application.

Starting the Application 

As stated earlier, the starting and stopping of our application can be done as a single unit. This is achieved by using application:start/1 and application:stop/1. These functions search for the specified application in the default system path. If your application does not reside in the system path, then Erlang will not be able to locate the code or the application resource file. Rather than pollute the system path with our non-system code, we can specify an additional search path when starting the Erlang VM using the -pa command line argument. Lets take a look and see how all of this fits together (remember that we called our application echo_app).

kaz@coco:/tmp$ erl -pa ~/port_example/lib/echo_app-1.0/ebin
Erlang (BEAM) emulator version 5.4.6 [source] [threads:0]

Eshell V5.4.6  (abort with ^G)
1> application:start(echo_app).
ok
2> application:loaded_applications().
[{kernel,"ERTS  CXC 138 10","2.10.7"},
 {stdlib,"ERTS  CXC 138 10","1.13.7"},
 {echo_app,"Echo Port Server","1.0"}]
3> echo:echo("Testing\n").
[["Received Testing"],["Current time is Mon Jan 30 17:15:43 2006"]]
4> application:stop(echo_app).

=INFO REPORT==== 30-Jan-2006::17:15:50 ===
    application: echo_app
    exited: stopped
    type: temporary
ok
5>
        

As you can see, starting and stopping the application is very easy. Once the application has been loaded, we can use the client API that was defined to send requests to the gen_server, which then sends the request to the external Python program for processing. Even more interesting is testing what happens if the external Python process is killed.

5> os:cmd("pkill echo.py").

=ERROR REPORT==== 30-Jan-2006::17:39:59 ===
** Generic server echo terminating
** Last message in was {'EXIT',#Port<0.96>,normal}
** When Server state == {state,#Port<0.96>}
** Reason for termination ==
** {port_terminated,normal}
[]
6> echo:echo("Testing\n").
[["Received Testing"],["Current time is Mon Jan 30 17:40:11 2006"]]
7>
        

After the external Python process was killed, the gen_server process terminated abnormally with {port_terminated, normal} just as it was designed to do. When the supervisor received the termination notice, it started another gen_server process to replace the old one, which in turn started up a new Python process ready to process requests. We were then able to use the client API to access the services of the Python process again without having to restart the application.

In the next chapter, you will learn how to start the application using a boot script. Upon startup of the Erlang VM, our application will be available immediately for use.

8. Release and Boot Script

Overview 

An OTP release is the bundling of one or more applications together to form a single comprehensive system. Creating a release involves the creation of a boot script and/or a tarball of the release that can be installed on another target system. With a boot script, you can start the Erlang VM using the -boot command line argument. Upon start up, all applications that are part of the release are started and immediately available for use. The boot script will ensure that the correct ordering of start up among the applications for you based on the dependencies that are listed in the application definition files.

Creating a release for this tutorial is not very interesting because we have no other applications to bundle in the release that actually utilize the services our echo server. However, building a release does enable us to create a boot script to make the starting of our application even easier because we will not have to call application:start(echo_app) once the VM has started. This will be done by the boot script for us as we'll see in a later section.

In this chapter, we will only generate a boot script and provide some notes on how to start the application using the boot script. Creating a tarball or package of the release is not discussed because I have yet to figure out how to install one on a target system. As for the brevity of this chapter, the discussion is kept as brief as possible because trapexit.org already contains a tutorial on how to create an Erlang releases by Ulf Wiger.

Creating the Boot Script 

In order to create a release, and thus our boot script, a release resource file must first be created. This file specifies all of the applications and versions of those applications that should be included in the release. It may seem as though our application has no dependencies; however, this is not true. All applications have a minimum dependency on both the kernel and stdlib applications so the release file must include both of them as dependencies.

The release resource file should reside in a separate release directory for your project. If you look in the tarball of this tutorial, you'll find the standard Erlang directory layout that is used for releases. In the port_example/releases/1.0 directory, you'll find the the release file called echo.rel, and the boot script that will be generated from it. Here are the contents of echo.rel:

{release, {"Example Port Server", "1.0"}, {erts, "5.4.6"},
 [{kernel, "2.10.7"},
  {stdlib, "1.13.7"},
  {echo_app, "1.0"}]}.
        

Once the release file is in place, it's a simple matter to create the boot script. Start an Erlang session in the following directory, port_example/releases/1.0, and type the following:

kaz@coco:~/port_example/releases/1.0$ erl
Erlang (BEAM) emulator version 5.4.6 [source] [threads:0]

Eshell V5.4.6  (abort with ^G)
1> Dir = "/home/kaz/port_example".
"/home/kaz/port_example"
2> Path = [Dir ++ "/lib/*/ebin"].
["/home/kaz/port_example/lib/*/ebin"]
3> Var = {"MYAPPS", Dir}.
{"MYAPPS","/home/kaz/port_example"}
4> systools:make_script("echo", [{path, [Path]}, {variables, [Var]}]).
ok
5> halt().
        

This will create two files in the current directory: echo.boot and echo.script. These are the compiled and uncompiled boot scripts respectively. We can now use the compiled boot script to start an Erlang VM that will automatically load our application and all of its dependencies. We'll use the -boot and -boot_var command line arguments to do so. Type the following to start the application:

kaz@coco:~/port_example/releases/1.0$ erl -boot echo -boot_var MYAPPS ~/port_example
Erlang (BEAM) emulator version 5.4.6 [source] [threads:0]

Eshell V5.4.6  (abort with ^G)
1> echo:echo("Testing\n").
[["Received Testing"],["Current time is Wed Feb  1 14:55:04 2006"]]
2>
        

As illustrated above, starting a system with a boot script is simple. Our echo application is immediately available for use without having to manually start it or any of its dependencies.

Before moving on to the next chapter, there are two points of interest to be noted. First, when using the boot script, we can start up our system from any directory as long as the correct paths are used. And secondly, remember those application configuration parameters (extprog, timeout, and maxline) that were specified in the application resource file (echo_app.app)? Well, they may be overridden on the command line. For example:

$ erl -boot echo -boot_var MYAPPS ~/port_example -echo_app maxline 5
Erlang (BEAM) emulator version 5.4.6 [source] [threads:0]

Eshell V5.4.6  (abort with ^G)
1> echo:echo("Testing\n").
[["Recei","ved T","estin","g"],
 ["Curre","nt ti","me is"," Wed ","Feb  ","1 15:","06:46"," 2006"]]
2>
        

For completeness, you should be made aware that the parameters may also be overridden by a system configuration file that can be specified using the -config command line argument.

9. Conclusion

Some Closing Comments 

In this tutorial, you have hopefully learned how to use an Erlang port to communicate to an external system, and along the way learned how one might use some of the features that OTP provides for simplifying development through the use of standard behaviors as well as how to build robust software using a supervisor.

Thanks for reading!