Mon. Apr 22nd, 2024

This is a quick getting started guide to using Tokio, an async runtime for Rust. To understand this article, you need to:

  • understand at least the basics of Rust programming
  • understand at least some of the concepts behind parallel and concurrent programming
  • understand asynchronous programming with async/await

What is Tokio?

Tokio is an async runtime for rust. Rust supports async programming, but in the spirit of keeping the standard library small, it does not offer a great async runtime out the box. Instead, 3rd party tools like Tokio may be used instead. The obvious downside here is you need to learn about a 3rd party dependency, but the upside is you have choice, and if Tokio does not meet your needs, you can always use something else.

Tokio is probably the most popular async runtime at the time of writing this article (February 2022).

Concurrent vs. parallel

The short version

Concurrent means running multiple tasks in such a way that they look like they are all running at once, even though only one task is running at a time. A task is just a piece of code to run, like a function, or a lambda, or even an async block.

Parallel means running more than one task simultaneously e.g. running 2 threads at the same time, each on a different CPU core.

If you understand this, skip to the next section. If you are still a little confused, keep reading.

The longer version

Suppose you have 3 functions you want to run at the same time. Let’s call them:

  • async fn A()
  • async fn B()
  • async fn C()

Traditionally you would use threads. Each function will be started in a different thread, and the operating system will typically allocate each thread to a different CPU core if possible. So, if you have a 4-core CPU, it’s possible A(), B(), and C() are all running at exactly the same time. This is parallel processing.

Of course if you have a 2-core CPU, only 2 threads will run in parallel, and the operating system scheduler will eventually context switch to the remaining thread when possible (i.e. it will swap one of the running threads for the thread that’s not running).

If you have a single core CPU, then only 1 thread will be able to run at a time, so nothing will be running in parallel. That is, even though it will look like all 4 threads are running at once, the OS scheduler is really just switching between them really quickly to make it seem as though they are all running at once, even though they are running one after the other.

Another way of saying what happens to threads on a single core CPU is to say the threads are running concurrently. They can’t run in parallel because there’s only one CPU core!

Concurrency has the effect of parallel processing, without actually doing parallel processing.

Historically, it has been the job of the OS scheduler to achieve this effect, and the effect was achieved with threads. But more recently, people have created started doing multiple pieces of work concurrently on the same thread; sometimes called green threads.

The reason is simple: threads are expensive. Starting a thread is much faster and less resource intensive than starting a new application/process. But starting a green thread is much much faster, and even lighter on resources, than running a thread. So if you need to service thousands of network calls, for instance, running them on green threads places much less of a strain on the operating system, and you have huge performance gains.

The operating system does not handle green threads. And it would be silly if every programmer had to implement a green threading solution for themselves. Which is where async runtimes like Tokio fit it.

Tokio to the Rescue

Tokio is a runtime and once started, you can simply hand it tasks to complete. A task, as mentioned above, it a piece of code to run. Tokio calls these pieces of code tasks as well. Tokio will accept the task and run it in the background until it is complete. Tokio can accept multiple tasks, and can be configured to run on multiple threads. This means you can have X tasks running on Y threads, with almost no additional effort on your part.

Coding time

Setup:

Add the following to your Cargo.toml:

[dependencies]
tokio = { version = "1", features = ["full"] }
rand = "0.8.4"

Sample Code

Here’s some heavily commented sample code. The code will run 10 tasks in the background. In this case, I am calling the same function 10 times. Each task does the following:

  • prints a message saying for how long it plans to sleep
  • sleeps for that amount of time
  • prints a message saying it is complete
use std::thread::sleep;
use std::time::Duration;
use tokio::task;
/*
 * Grab a random number, say that you're going to sleep
 * for that number of ms, and say when the sleep is complete
 */
async fn random_print(num: i32) {
    let r = rand::random::<u64>() % 500u64;
    println!("msg{} will sleep for {}ms", num, r);
    sleep(Duration::from_millis(r));
    println!("msg{} complete", num);
}
/*
 * mark main() as the main function to use with the tokio runtime.
 * It's normally ok to just use #[tokio::main], but here, I am
 * explicitly saying I want to configure the runtime to use
 * multiple threads, and I want 8 threads configured.
 */
#[tokio::main(flavor = "multi_thread", worker_threads = 8)]
async fn main() {
    /* create a vector for the join handles. This is a nice to have, but not required */
    let mut handles = vec![];
    /* let's start a function 10 times */
    for j in 0..10 {
        /* start a task */
        let handle = task::spawn(random_print(j));
        /*
         * save the handle for later use. This handle can be used
         * to check if the task is complete later.
         */
        handles.push(handle);
    }
    /*
     * the next loop is useful if you want to explicitly wait for all tasks to complete.
     * A more complex implementation might require that you only wait for certain
     * tasks to complete
     */
    for handle in handles {
        let _ = handle.await;
    }
}
Here’s some sample output. If you run this code, your output will almost certainly be different, but similar.
msg0 will sleep for 190ms
msg1 will sleep for 458ms
msg2 will sleep for 260ms
msg3 will sleep for 345ms
msg4 will sleep for 321ms
msg5 will sleep for 109ms
msg6 will sleep for 4ms
msg7 will sleep for 188ms
msg6 complete
msg8 will sleep for 155ms
msg5 complete
msg9 will sleep for 379ms
msg8 complete
msg7 complete
msg0 complete
msg2 complete
msg4 complete
msg3 complete
msg1 complete
msg9 complete
Explanation

First, you need to start the Tokio Runtime. There are several ways of doing this, but by far the easiest is to just use a macro attached to fn main() and tell Tokio to start automatically.

So:

fn main() {
}

Becomes:

#[tokio::main]
async fn main() {
}

Now, when your main function starts, the Tokio runtime will start as well. Note I used the async keyword on the main function. As mentioned above, you need to understand async/await for this tutorial to make sense, so please ensure you are familiar with these concepts.

In my example, I used a slightly more configurable version of this macro when I wrote:

#[tokio::main(flavor = "multi_thread", worker_threads = 8)]

Tokio can schedule your tasks in several different ways. I chose the “multi_thread” scheduler, and I asked for Tokio to use 8 threads. This is overkill for this example, but it is included to show some of the available configuration options.

It is also possible to start the runtime manually, as I will show later.

The important line in the code is

let handle = task::spawn(random_print(j));

This is called 10 times in a loop. spawn() creates a Tokio task and the code to run in this case is the async function random_print().

Tokio will move the task to the background to some thread, and when possible it will run that task concurrently with any other tasks allocated to the thread. You don’t need to pick the thread, or decide when the task runs. Tokio takes care of everything.

Finishing task processing

It is important to know when a task completes. The last thing to you want is for your application to exit while a background task is still completing. task::spawn() returns a JoinHandle that can be used to communicate with the running task. Note that in my second loop, I call JoinHandle.await. This ensures all tasks are completed before the application exits.

Starting Tokio from a non-main functions

The example above is the typical scenario covered in most examples. However it is possible to start multiple Tokio runtimes, and to also start those runtimes in a non-main function. See the code below:

use std::thread::sleep;
use std::time::Duration;
use tokio::runtime;
async fn random_print(num: i32) {
    let r = rand::random::<u64>() % 500u64;
    println!("msg{} will sleep for {}ms", num, r);
    sleep(Duration::from_millis(r));
    println!("msg{} complete", num);
}
fn main() {
    not_a_main_fn();
}
fn not_a_main_fn() {
    let rt = runtime::Builder::new_multi_thread()
        .worker_threads(8)
        .build()
        .unwrap();
    for j in 0 .. 10 {
        rt.spawn(random_print(j));
    }
    /*
     * I'm being lazy here, and just delaying to let all tasks complete.
     * Using JoinHandle.await is much better. Also, comment out this line
     * and you will notice the program most likely ends before all 10 tasks
     * complete correctly!!! This is why JoinHandle.await is important!
     */
    sleep(Duration::from_millis(5000));
}

The code is similar to the original example but not the same. Here, main() is not an async function, and all it’s doing is calling

fn not_a_main_fn()

not_a_main_fn() is responsible for starting a Tokio runtime manually, configuring it, and then using it. Notice, for instance, that task::spawn() has been replaced with rt.spawn(), meaning that the newly spawned task will run on the runtime referenced by “rt”. Note also that the

#[tokio::main]

macro is not used at all.

Lastly, a sleep statement was included here instead of cleanly waiting for every task to complete. This version of the code will typically take around 5 seconds to execute which is wasteful. But commenting out the sleep statement will most likely result in the application not working correctly. My sample output when commenting out the sleep statement is shown below:

msg1 will sleep for 243ms
msg0 will sleep for 207ms
msg0 complete
msg1 complete

That’s it. 10 tasks were created but only 2 ran. This is because the main application exited before the tasks completed. This is why it is important to collect JoinHandles from tasks and ensure tasks exit cleanly.

Summary

Tokio is an async runtime for Rust. It is very easy to use in its default configuration, and not much more complex when configuring it manually. With minimal effort, you can schedule multiple tasks on multiple threads, without the effort of implementing any low level details.

The End



Leave a Reply

Your email address will not be published. Required fields are marked *