Intro
Dear Readers!
In this post I’m going to continue a previous post I wrote about C++ concurrency facilities, which mostly dealt with data sharing and thread management. As there are other topics beyond the mentioned ones in the field of concurreny, a second post is warranted, which will discuss facilities supporting waiting for events, inserting tasks into existing threads, and creating lockless synchonized data bridges between threads. Before going further however, I’d like to say that the same disclaimer applies to this post as to the previous one, so please read that before proceeding. After having said all this, let’s get started.
Condition variables
Condition variables were introduced into the C++ threading library to avoid expensive ways of waiting for some events to happen. Let’s see the different ways of waiting and why some of them are better than others.
Figure 1: Thread ‘A’ is continuously polling f to check if it becomes true.
This is the most basic way to wait for a particular condition to become true, which can be implemented with a simple loop. Thread ‘A’ polls f continuously until it becomes true, which is resource hungry operation. Because of that, this approach is also called a busy-wait.
Figure 2: Thread ‘A’ polls f continuously, but some sleeping periods are inserted between asserts to free computing resources.
In this second case thread ‘A’ sleeps between asserts, so some computing resources are now freed during waits. Such a solution can be created using a loop and calling sleep_for() on std::this_thread, but the problem with such an approach is that it is really hard to guess how much time is needed between asserts. If the time is too short, than the actual gain is of going to sleep periodically is quite minimal because of the extra resource consumed by the context switches themeselves. If the time is too long on the other hand, than there is a high chance that the waiting thread will waste huge amounts of time if the condition becomes true quite early in the wait period. A similar situation is also depicted on the image above: f becomes true in the middle of a sleeping period, so half of the waiting time is actually wasted. If the sleep time were to be set to 200 ms and f would become true at e.g. 40 ms into the sleep cycle, then Thread ‘A’ would be waiting for 160 ms completely unncessarily. In some situations this might not be that terrible, but in case of a GUI application or a heavily used database it would simply translate to horrible user experience and inefficiency.
Figure 3: Thread ‘A’ simply sleeps until f becomes true. Computing resources are free to be utilized by other tasks in the meantime.
The execution pattern above seems to mitigate all previously mentioned problems, as Thread ‘A’ consumes resources only when absolutely necessary and waits precisely as much as need and not more. The creators of the Thread Library fortunately thought of this and as such std::condition_variable and std::condition_variable_any is provided to facilitate such needs. The image below depicts it in action:
Figure 4: Utilizing a std::condition_variable to wait for some condition to be met.
The way the wait() function works is that if the callable object provided for it (in this case a lambda) returns true (aka. condition is met), than the lock is not released and the mutex protected code is executed. If the function returns false, it unlocks the mutex and puts its own thread to sleep until somebody (from another thread) calls notify_one() or notify_all(). Once a notify function has been called, the thread is woken up, it relocks the mutex and then finally checks the condition again. At least theoretically this is what should happen, but in reality whether there is sleep or not greatly depends on the implementation.
Finally, there is an important aspect of all of the above approaches: the conditions on which the threads operate can become valid or can be reset any number of times. Consequently, this means that there is always (at least) one thread that makes changes to make the condition(s) valid, while other threads will always do work to invalidate it/them somehow (see image below). All this translates into a very important property of conditional variables: they do not contain any state related to the condition, instead they just execute the check provided to them (through the constructor) after they get notified, and as such these constructs are just means of communication rather than a magic boolean of some sort.
Figure 5: A usual workflow that utilizes condition variables.
Futures and async
As discussed in the previous section some conditions can become valid then become invalid again, but what if someone needs something to happen only once, and not only that, it should also happen irreversibly (shouldn’t be able to invalidate itself along the way). The good news is that C++ supports this sort of events through futures. Let’s examine them a bit closer.
The C++ concurrency library provides not only one, but two futures: std::future and std::shared_future, which are said to be modeled after std::unique_ptr and std::shared_ptr:
Figure 6: Similarities between futures and smart pointers.
But what are those exactly? Futures are basically placeholder objects for values that will only be available sometime in the future, hence the name. Because futures can convey data (but they are not required to do so), they are templated just the same as smart pointers. The available two types of futures model smart pointers yet again, as with a non-shared version state can only be accessed from a single location, while with a shared version access from multiple threads is possible.
Figure 7: Using std::thread and std::async to launch new threads.
As seen on the image above, a std::async is used to supply a std::future for the requesting thread, while the function launched through async calculates the result in a separate thread. One could argue that std::async holds some similarity to a std::thread, and he/she would be right. They are both used to outsource an operation to a new thread, but the major difference between the two is, that returning data from a function launched with std::thread is far more complicated than it is with a std::future, as there is no direct way to transfer data out of the new thread with the former one. Of course, one needs to pass data in to get some results out, which is quite possible by providing data as additional arguments to std::future and std::thread too, a fact that has conveniently been omitted in the first post. There are other differences as well. In case of std::async it is the future that we have has an object referring to the new thread, while in case of std::thread it is the named thread object itself.
As it is also visible on the image above, the data from the future can be retreived with the get() member function. The image also shows, that if the computation of the result is not ready by the time get() is called, the calling thread will wait until the result becomes available, or in other words the future becomes ready. It is important to note here, that whether a new thread is launched or not for a task passed to a std::async is up to the implementation, but can also be chosen by the user with an argument of std::launch type passed to async.
Stashing tasks away for later
The mechanism described in the previous section works pretty well when we would like to launch the task without caring on which thread it gets launched. If one wants to launch a task on a particular thread however, one needs a facility that is capable of packaging and transfering a function to a particular thread, while it is also capable of returning a result back later through a future. An observant reader might have already noticed some hidden information in the previous sentence hinted by the ‘getting the results back later’ part: a task that can be packaged away doesn’t necessarily needs to be launched immediately, and it can be moved around just like any other variable. Such features are provided by the std::packaged_task:
Figure 8: A task gets moved to another thread for later execution.
The figure above is somewhat similar to the figure depicting the workings of std::async and std::thread found in the previous section, but there is major difference: unlike a task passed to a std::thread (or async), the task here gets transferred into an existing thread and the execution doesn’t necessarily start immediately. The figure above shouldn’t confuse the reader though, as it is not a requirement to pass a task into an existing thread, but it is also quite possible to launch a new one with the facilities described earlier.
Futures and promises
Up until now, futures were tied to specific functions to fill them up with data later in the program. Not only that, but the programmer also had to know which function will do that at the future’s time of creation. Thus, to provide more flexibility, the thread library has means to make futures without such knowledge by supplying std::promise. The general idea behind a std::promise / std::future pair is to create a synchonized data bridge between threads, where the promise is a one-time data sink, while the future is a simple data outlet. One can think of these as a single variable divided into two parts that has a read-only part in one thread (a std::future), and a write-only part sent off to another thread (a std::promise):
Figure 9: A std::future gets filled with data with a function unknown to it at the time of creation.
If we compare the image above with the one depicting the workings of packaged task, it becomes apparent that there is no func1 that needs to be known beforehand to be send off to another thread in order to fill the future, but instead, we just simply send a write-only variable over without caring who will fill it with data. There is another important difference between a std::promise and ordinary variables: it can be written into once only. If set_value() is called multiple times, then another very interesting feature of futures gets used: an exception is stored in it. Once get() is called on that particular future, the exception gets ‘released from the jar’ and the exception has to be dealt with normally from that point. This is not the only circumstance where an exception is stored in future though, as when a promise is broken (a promise is destroyed before setting a value) or when a function stored in a packaged task is never executed (it is destroyed before calling the function), the same mechanism is used. This exception storing property of futures is more general than this however, as is it possible to store all sorts of exceptions in a future for later use with the help of the set_exception() member function of a std::promise.
As all major (and currently supported) facilities of the thread library have been discussed, it is time to conclude this post. This does not mean that more could not be said about threading in general, quite the contrary, but since I’m also just familiarizing myself with the concepts, I’ll leave in depth discussions to pros.
As always, thanks for reading.