Pattern: Thread per connection
Explanation
This pattern is very similar to the Process Per Connection pattern from the last chapter. The difference? Spawn a thread instead of spawning a process.
Threads and processes both offer parallel execution, but in very different ways. Neither is ever a silver bullet and your choice of which to use depends on your use case.
Spawning. When it comes to spawning, threads are much cheaper to spawn. Spawning a process creates a copy of everything the original process had. Threads are per-process, so if you have multiple threads they’ll all be in the same process. Since threads share memory, rather than copying it, they can spawn much faster.
Synchronizing. Since threads share memory you must take great care when working with data structures that will be accessed by multiple threads. This usually means mutexes, locks, and synchronizing access between threads. Processes need none of this because each process has its own copy of everything.
Parallelism. Both offer parallel computation implemented by the kernel. One important thing to note about parallel threads in MRI is that the interpreter uses a global lock around the current execution context. Since threads are per-process they’ll all be running inside the same interpreter. Even when using multiple threads MRI prevents them from achieving true parallelism. This isn’t true in alternate Ruby implementations like JRuby or Rubinius 2.0.
Processes don’t have this issue because each time a copy is made, the new process also gets its own copy of the Ruby interpreter, hence there’s no global lock. In MRI, only processes offer true concurrency.
One more thing about parallelism and threads. Even though MRI uses a global interpreter lock it’s pretty smart with threads. I mentioned in the DNS chapter that Ruby will allow other threads to execute while a given thread is blocking on IO.
In the end threads are lighter-weight; processes are heavier. Both offer parallel execution. Both have their use cases.
Implementation
require 'socket'
require 'thread'
require_relative '../command_handler'
module FTP
Connection = Struct.new(:client) do
CRLF = "\r\n"
def gets
client.gets(CRLF)
end
def respond(message)
client.write(message)
client.write(CRLF)
end
def close
client.close
end
end
class ThreadPerConnection
def initialize(port = 21)
@control_socket = TCPServer.new(port)
trap(:INT) { exit }
end
def run
Thread.abort_on_exception = true
loop do
conn = Connection.new(@control_socket.accept)
Thread.new do
conn.respond "220 OHAI"
handler = FTP::CommandHandler.new(conn)
loop do
request = conn.gets
if request
conn.respond handler.handle(request)
else
conn.close
break
end
end
end
end
end
end
end
server = FTP::ThreadPerConnection.new(4481)
server.run
This code is subtly different from the previous two examples. It has more-or-less the same methods, but they’re organized differently.
Connection = Struct.new(:client) do
CRLF = "\r\n"
def gets
client.gets(CRLF)
end
def respond(message)
client.write(message)
client.write(CRLF)
end
def close
client.close
end
end
Here we have the same boilerplate methods as before, but now they’re grouped into a Connection class, rather than being defined on the server class directly.
def run
Thread.abort_on_exception = true
loop do
conn = Connection.new(@control_socket.accept)
Thread.new do
conn.respond "220 OHAI"
handler = FTP::CommandHandler.new(conn)
There are two key differences here. The first is that this code spawns a thread where the previous example spawned a process. The second difference is that the client socket returned from accept is passed to Connection.new; each thread gets its own Connection instance.
This is very important when working with threads. If we had simply assigned the client socket to an instance variable, as we did previously, it would be shared among all of the active threads. Since the threads are spawned in a shared instance of the FTP server, they share the internal state of the instance.
This is a stark difference to programming with processes, where each process gets its own copy of everything in memory. This sharing of state is one reason why some developers say that programming with threads is hard. There’s a simple rule of thumb when you’re doing socket programming with threads: each thread gets its own connection object. This will save you headaches.
Considerations
This pattern shares many of the same advantages as the previous one: very little code changes were required, very little cognitive overhead added. Although using threads can introduce issues with locking and synchronization, we don’t have to worry about any of that here because each connection is handled by a single, independent thread.
One advantage of this pattern over Process Per Connection is that threads are lighter on resources, hence there can be more of them. This pattern should afford you more concurrency to service clients than you can have using processes.
But wait, remember that the MRI GIL comes into play here to prevent that from becoming a reality. In the end, neither pattern is a silver bullet. Each should be considered, tried, and tested.
This pattern shares a disadvantage with Process Per Connection: the number of threads can grow and grow until the system is overwhelmed. If your server is handling an increased number of connections, then your system could possibly be overwhelmed trying to maintain and switch between all the active threads. This can be resolved by limiting the number of active threads. We’ll see this when we look at the Thread Pool pattern.