The Run Loop

I’ve recently decided to do some research into what exactly async/await does under-the-hood. I’ve been bitten several times with C#’s async/await before, getting deadlocks in some code. It’s very frustrating sometimes. (I’ve been watching Haikyu! recently, and can compare it very easily to the Iron Wall of blockers, and how a spiker repeatedly blocked loses their composure and confidence).

So, let’s fix that! It may just be my experience, but I don’t think run loops, event loops, synchronization contexts, or whatever name they’re given get a lot of discussion.

The journey of a programmer

When a fledgling programmer begins their journey into programming, they typically start out with:

int main(int argc, char* argv[]) {
	printf("Hello, world!");
	return 0;
}

We can all parse this easily:

  1. The OS/kernel loads the program into memory
  2. Every program has a main(int argc, char* argv[]) function which marks the entry point of the line of execution.
  3. The kernel constructs the single process for the program, with the process' minimum of one thread of execution.
  4. The kernel calls that function.
  5. The program executes its code, in this case writing some bytes to the stdin file descriptor, which terminals and command prompts handle by rendering to the screen
  6. The program returns from the main function, the kernel dumps the program from memory, destroys the thread and process, and ends.

And programmers learn to do a lot more advanced and fun things to do in the main function. Maybe they do some math, maybe they get risqué and make a couple of functions to call.

But, you’ve no doubt decided “Okay, I think I got this whole programming thing down. Let’s make a GUI!” Woah now, hold up. That’s a big task! Let’s take the easy route and use a framework someone else made.

C# WinForms:

using System;
using System.ComponentModel;
using System.Drawing;
using System.Windows.Forms;

public class Form1: Form
{
	public Form()
	{
		button1 = new Button();
		button1.Size = new Size(40, 40);
		button1.Location = new Point(30, 30);
		button1.Text = "Click me";
		this.Controls.Add(button1);
		button1.Click += new EventHandler(button1_Click);
	}
}

Swift UIKit:

import Foundation
import UIKit

@UIApplicationMain
class AppDelegate: UIResponder, UIApplicationDelegate {
	var window: UIWindow?
	var pushHelper = PushHelper()
	var appComponents: AppComponents?

	func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplicationLaunchOptionsKey: Any]?) -> Bool {

	}
}

Wait…where did our main functions go?? It’s the core of our being, the entry point into both our application and our very soul and existence. You’ve blasphemed by getting rid of it…or, hiding it? Indeed, GUIs tend to hide the main function from us. It tends to be in an auto-generated or hidden file, in Visual Studio’s/C#’s example:

[STAThread]
public static void Main()
{
	Application.EnableVisualStyles();
	Application.Run(new Form1());
}

Or via a tag like Swift’s @UIApplicationMain, which translates to the following on compilation:

int main(int argc, char *argv[])
{
	@autoreleasepool {
		return UIApplicationMain(argc, argv, nil, AppDelegate.self);
	}
}

And even if this were more visible (they weren’t always so well hidden, they used to just be auto-generated files when making a project, which you could change at will), you likely wouldn’t touch it much. It looks foreign to you; magical even. You think to yourself “I don’t get it, wouldn’t the function just end immediately? What does this ‘run’ function even do?” You search the internet for a bit, are told that it only returns on an error or if the user closes the window, and then you call it a day, safe with the knowledge that it’s handling everything for you, but never quite sure what it’s actually doing.

Continuing the journey

With all that trouble now behind you, you’ve got your GUI up and running! Things are going well, with a text box and a button, which computes the Fibonacci sequence based on the number in the textbox.

void btnCalculate_Click() {
	indicator.Visible = true;
	Messagebox.Show(Fibonacci(txtField1.Value));
	indicator.Visible = false;
}

Satisfied with your hard work and calling it a day, you put in 999 and hit enter to revel in your own prowess, only to stare in disbelief as your entire GUI locked up! Your loading indicator that spins isn’t spinning, or didn’t even show up in some cases, you can’t click to exit the window, and nothing is working. Heart racing, blood pumping, face sweating, you fear for the worst. And then, the popup appears with the number. The GUI carries on like normal, as if the world didn’t just stop. You grasp your monitor, cry out to your app, begging it to realize what happened, to recognize the incredulity of its own….okay I’ll stop with the narrative.

What happened is we’ve blocked the thread. This is something we’re all told at one point. We’ve written blocking, synchronous code that locked up the main thread, preventing the run loop from processing more events, such as rendering or other clicks. It’s not fun, but we all learn one way or another. Thus begins the rabbit hole that is this post.

The solution

When you happen upon this type of problem, the usual fix is something like the following:

Swift:

indicator.Visible = true
DispatchQueue.global(qos: .userInitiated).async { [weak self] in
	// Do work on bg thread
	let overlayImage = self.faceOverlayImageFrom(self.image)

	// Change UI on main thread
	DispatchQueue.main.async { [weak self] in
		indicator.Visible = false
		self?.fadeInNewImage(overlayImage)
	}
}

C#:

indicator.Visible = true;
var overlayImage = await this.faceOverlayImage();
this.fadeInNewImage(overlayImage);
indicator.Visible = false;

In both instances, the tasks being performed are:

  1. On the main thread
    1. Change the UI to show the loading indicator
    2. Queue up a block of execution on a background thread
    3. Return the main thread to the run loop
  2. On the background thread
    1. Perform the work, that may take a while to complete
    2. Queue up a block of execution on the main/UI thread
    3. Return the background thread to its own run loop (in some instances; sometimes it’s creating and destroying a thread for this purpose, sometimes it’s a generic thread with its own run loop to handle background tasks)
  3. On the main thread
    1. Change the UI to hide the loading indicator and show the results of the work

In Swift, these tasks are a bit more explicit. We see the specific queuing of resources onto specific threads. C# uses the async/await pattern, present in many other languages, to make this a bit simpler to perform. There are proposals for Swift to gain async/await as well, but we do not have that yet.

For the C# example, I might be a little wrong1; it’s possible that it just breaks that function into 2 or 3 chunks which get queued into the run loop one after another (to allow regular UI events to happen in-between), and that network or other IO operations are non-blocking. If we put a heavy computational load into that await call, then it might still run on the main thread, necessitating an explicit call to a new thread. This is why it’s important to know the details of your specific run loop model, and its intricacies!

So how does this all work?

I think Apple’s2 documentation says it best:

  1. Notify observers that the run loop has been entered.
  2. Notify observers that any ready timers are about to fire.
  3. Notify observers that any input sources that are not port based are about to fire.
  4. Fire any non-port-based input sources that are ready to fire.
  5. If a port-based input source is ready and waiting to fire, process the event immediately. Go to step 9.
  6. Notify observers that the thread is about to sleep.
  7. Put the thread to sleep until one of the following events occurs:
    • An event arrives for a port-based input source.
    • A timer fires.
    • The timeout value set for the run loop expires.
    • The run loop is explicitly woken up.
  8. Notify observers that the thread just woke up.
  9. Process the pending event.
    - If a user-defined timer fired, process the timer event and restart the loop. Go to step 2.
    • If an input source fired, deliver the event.
    • If the run loop was explicitly woken up but has not yet timed out, restart the loop. Go to step 2.
  10. Notify observers that the run loop has exited.

This is specific to the RunLoop in Obj-C/Swift, of course, but I think it’ll work very similarly to C#’s SynchronizationContext as well, and to whatever JavaScript uses for setTimeout and such.

Basically, a network or other IO result are sent to the thread via epoll3 or kqueue4, which wakes up the thread and tells it to read the data, before the run loop puts its thread back to sleep. Or, you can add one or more blocks of execution (Swift: selectors via perform(_:target:argument:order:modes:); C#: tasks, via async/await; JS: functions, via setTimeout or with Promises) from the main thread, or another thread which also wakes up the run loops' thread to tell it to process the event. If multiple events have been queued up, then it keeps processing them one at a time until the queue is exhausted and it can put itself back to sleep.

Takeaways

The thread has transformed from your Computer Science days. The fundamentals are the same: it still is a single thread of execution. Linear, unending, and fast. But a layer has topped it like an umbrella, molding it into a new way of thinking.

A thread is broken up into a series of tasks or blocks to execute, by essentially calling queued functions, and the thread goes to sleep when it has nothing to do, freeing up system resources. That’s basically it.

When working on client-side GUIs or server-side applications, you have to realize that blocking or long-running code is sharing that thread with other events or requests. You have to break up your code into smaller chunks and utilize the run loop more effectively, or use a pool of background threads to do CPU-heavy work. If not, you risk stalls in system performance.

I didn’t get into some more detailed aspects of C# or Swift like I wanted to. Using async/await in C# can be tricky in older applications where not everything is awaitable from the get-go. This can cause deadlocks and it always tricky to understand the solution to. Even after writing all of this I would still need to look up the reason why, but I think it’s because you can mix different run loops or sync contexts, or something similar, with older ASP framework code. However, I think this somewhat more broad discussion helps a lot for understanding this problem. If you need more specific answers, do research on your language or API, since they’re all pretty different in minute ways.

How I feel

In the Swift example earlier, it’s definitely a bit more cumbersome to write out. It has nested calls to specific threads. Nesting can get ugly at times. Promises are the general solution to this, where we chaining together different blocks of code in a more linear fashion. It looks like PromiseKit can specify which thread to queue events on, which is nice. And I think Combine is meant to help some of that as well. I looked at Combine a while ago, but should look into it more with this better knowledge of run loops in mind.

C# with async/await feels more magical to me. It took me a while to even find SynchronizationContext which powers it all in the background. I think the older ASP/WinForms roots and hybrid code doesn’t help a lot, with their insistence on backwards compatibility. From my research, it’s harder to get a straight answer about which thread something runs on, or if await uses new thread or not. I read somewhere that the newer ASP uses a pool of threads to handle requests, and if you use non-blocking IO, your request might jump around to different threads that are free, which is nice. But it feels perhaps over-engineered (at least from my brief research).

I feel like I have a better grasp on the concept but truly mastering a specific system will be more difficult, especially with differences in use, in APIs, or in implementation details.

Go

Curiously, I don’t think Go uses run loops much, at least in server/service/daemon scenarios. In a UI, sure, it most likely needs one. But for processing incoming HTTP requests? It has a different solution.

In the old days, Apache would start up and immediately spawn a set number of child processes. The main thread would listen on port 80, and pawn off the requests to the different child processes to handle themselves, before returning the results to the parent to send to the client. A process can have a bit more importance than thread within a process, but spawning a process can be slow and take a lot of time. But, each process (as a guess) would only be able to handle one request at a time, blocking its main thread.

Instead, Go uses green threads, called goroutines, which doesn’t register any new thread within the process. Go has a runtime which makes new lightweight threads at will. So, when a new socket connection is made on port 80, simple make a new goroutine to handle it. When 1,000 new connections come in? Make 1,000 new goroutines! We don’t have to worry about blocking anything because each and every request gets its own goroutine in which to handle tasks, that it doesn’t really need to worry about blocking something else. And you might argue that the overhead of a run loop goes away with this model as well.

That certainly is a bit easier to remember than “I need to make sure I’m not blocking this thread, so I should probably perform this work asynchronously, but I need to return back to the new thread to use the results, and…d’oh! I caused a deadlock.”

It’d be interesting to see a Go UI that uses the same methodology. Instead of executing each incoming IO or user event on the UI sequentially on a main thread, process them all in goroutines! Obviously all UI code would need to be thread-safe, which poses the next problem. But I wonder what kind of solutions might come about from this.

  1. https://stackoverflow.com/questions/37419572/if-async-await-doesnt-create-any-additional-threads-then-how-does-it-make-appl ↩︎

  2. https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/Multithreading/RunLoopManagement/RunLoopManagement.html ↩︎

  3. https://en.wikipedia.org/wiki/Epoll ↩︎

  4. https://en.wikipedia.org/wiki/Kqueue ↩︎