Late last week I wrote a blog entry on how to interface Chicken Scheme and C. It was a short and sweet introduction to writing Scheme that calls into C that in turn calls back into Scheme. I spent some of the weekend adapting the vector_of_pairs code to implement a working echo server using epoll. There's a little more C code involved and a lot more Scheme code than before. If you've seen Tornado's epoll.c wrappers, you'll quickly recognize the similarities between their code and mine, with mine being adapted to Scheme instead of Python.

First, here's the epoll.c gist:

The _epoll_wait function is almost a direct copy of the vector_of_pairs implementation from my last blog post. The basic idea of epoll is to initialize an epoll file descriptor (_epoll_create), add events to epoll (_epoll_ctl), and wait for epoll to tell you of events as they occur (_epoll_wait).

In order to actually implement an echo server using epoll, you need to write some standard boilerplate to set up a server socket, bind it to an address:port, and start accepting connections. Thankfully, the tcp unit in Chicken Scheme handles quite a bit of that.

Here's the test_epoll.scm file:

The first step to setting up a server socket is by calling tcp-listen. That function returns a server socket listener.

(define listener (tcp-listen 6666))

Next, you initialize epoll. To do this, you define a foreign-lambda to bind the foreign function _epoll_create to a Scheme function:

(define ##epoll#epoll_create (foreign-lambda int "_epoll_create"))
(define epfd (##epoll#epoll_create))

Those two lines define the FFI and creates the epoll file descriptor. All events are tied to the epfd. At the bottom of the test_epoll.scm file, there's the main loop. It takes as an argument the socket listener created above and extracts the listener's file descriptor (tcp-listener-fileno). ev-main-loop then adds the server socket to epoll and watches for the read event. An infinite loop is then entered and epoll_wait is called with a timeout of 200 milliseconds.

The function SCM_epoll_wait_cb is the callback that the C _epoll_wait function calls. Right now _epoll_wait always calls back into Chicken, so next time I'll make it dependent on there actually being events ready to operate on. SCM_epoll_wait_cb converts the vec object into a list of pairs and passes that list to the event handler (fd-event-list-handler).

fd-event-list-handler is where the magic happens. If there aren't any fd's to operate on, fd-event-list-handler evaluates to an empty list. When a server processes sockets, it has to differentiate between new connections and old connections. It does this by checking if the socket being operated on is the same as the server socket, which would mean we have a new connection. A new connection has to be passed to the accept function to create the client socket. Old connections are operated on normally.

There are two event conditions that the handler recognizes when dealing with old connections: read and write. I set up two hash tables for handling a client socket's I/O: fd-write-table and fd-read-table. The lookups are based on the client's file descriptor and initialized to empty strings once the client socket is accepted.

When the server reads from a socket, it writes that data into a temporary buffer and writes the modified string to the client's output buffer (using send-to-client). Modified as in splits on the newline character and only writes the string up to and including that point. send-to-client doesn't actually do any writing. It just appends strings to the output buffer. When epoll tells us that a socket is ready to write on, that's when we write the output buffer to the socket. Both buffers are emptied by this point.

After each read/write, the file descriptor is updated in epoll to reflect the new event. After a read takes place, the socket is ready for writing to. After a write, the socket is ready for reading from.

Like the last entry, this code requires Linux and Chicken Scheme. Compile the two files above to get a working server: csc epoll.c test_epoll.scm -o server

Once the server is running you can telnet to localhost on port 6666.