Linux:: how to synchronize the main with a thread in C - Page 1
Forum section: electronics / general

author: 0db
author_rank: Frequent Contributor
timestamp: March 30, 2020, 06:30:16 pm
content: hi guysI am still studying Multithreading in C. I know how to synchronize two or more threads via Mutex and Conditions, but I don't know how to stop a thread until the main program has reached certain conditions.Is there any full guide with examples?

author: SiliconWizard
author_rank: Super Contributor
timestamp: March 30, 2020, 06:36:24 pm
content: Use semaphores.You can then wait on a semaphore in a thread, and post it in another thread (including the main thread) to wake the thread up again and exit the wait.

author: PlainName
author_rank: Super Contributor
timestamp: March 30, 2020, 06:54:28 pm
content: QuoteWhat OS are you using or thinking about? If it's FreeRTOS you should download the'hands-on tutorial'.Chapter 6.4 describes interrupt synchronization which should cover what you want to know. The pukka FreeRTOS API guide details the functions with examples, but this tutorial covers why you might want to use them.Edit: just noticed this is the Linux topic. Duh! But the principle remains the same whatever you're using.

author: 0db
author_rank: Frequent Contributor
timestamp: March 30, 2020, 07:18:13 pm
content: Quote from: SiliconWizard on March 30, 2020, 06:36:24 pmYou write "main thread", but in my case, I am in the int main() { ... } body of a C program and I need to wait until a thread, which will never terminate, reaches a particular state.Can I sill use semaphores, mutes, condition variables, &C in this case?

author: 0db
author_rank: Frequent Contributor
timestamp: March 30, 2020, 07:33:24 pm
content: Code:// C program to demonstrate working of Semaphores#include <stdio.h>#include <pthread.h>#include <semaphore.h>#include <unistd.h>sem_t mutex;void* thread(void* arg){//waitsem_wait(&mutex);printf("\nEntered..\n");//critical sectionsleep(4);//signalprintf("\nJust Exiting...\n");sem_post(&mutex);}int main(){sem_init(&mutex, 0, 1);pthread_t t1,t2;pthread_create(&t1,NULL,thread,NULL);sleep(2);pthread_create(&t2,NULL,thread,NULL);pthread_join(t1,NULL);pthread_join(t2,NULL);sem_destroy(&mutex);return 0;}basically, I have 16 threads, and their synchronization looks like the above code. However, in main() {} I have this lineCode:int main(){.../** wait untill tip has completed its process*/wait_for_tip_completed(tip);wait_for_tip_completed is an active waiting loopCode:void wait_for_tip_completed( ....while (1){if (tip.completed) return;}This somehow works, but ... it doesn't look the right way.

author: Karel
author_rank: Super Contributor
timestamp: March 30, 2020, 07:36:56 pm
content: http://man7.org/linux/man-pages/man3/pthread_join.3.html

author: 0db
author_rank: Frequent Contributor
timestamp: March 30, 2020, 07:56:03 pm
content: Quote from: Karel on March 30, 2020, 07:36:56 pm"pthread_join" is for terminated thread, mine never terminates.

author: Karel
author_rank: Super Contributor
timestamp: March 30, 2020, 10:59:18 pm
content: If your worker thread never terminates, then you can use an (atomic) (global) variable (e.g. an int).The worker thread writes the status of that thread to that global variable and the main thread reads it repeatedly to check the status.Use sleep() or usleep() during the repeat loop in the main thread. As long as one thread only writes and only one thread reads from thatglobal variable, it 's safe.Or use a pthread_mutex, they are designed for that.

author: nctnico
author_rank: Super Contributor
timestamp: March 30, 2020, 11:49:35 pm
content: Quote from: Karel on March 30, 2020, 10:59:18 pmReading a variable works but you have to poll it. If you use a semaphore you can wait until the semaphore gets signalled and continue immediately so performance is maximal.

author: SiliconWizard
author_rank: Super Contributor
timestamp: March 31, 2020, 12:25:25 am
content: Quote from: 0db on March 30, 2020, 07:18:13 pmOf course. It's still running inside its own thread just like any other.

author: Nominal Animal
author_rank: Super Contributor
timestamp: March 31, 2020, 10:00:46 am
content: POSIX threads, my favourite!Let's say you want the created threads to block at a specific point, until main thread has reached that point. The simplest way to achieve this is to use a shared global mutex, that the main thread locks before creating the threads, and unlocks when it reaches the specified point. The created threads simply grab that mutex and release it immediately. If the main thread has not reached that point yet, then the threads will block (wait at that point, without consuming CPU time) until the main thread unlocks the thread.Example:Code:#define _POSIX_C_SOURCE 200809L#include <stdlib.h>#include <inttypes.h>#include <stdio.h>#include <pthread.h>#include <limits.h>#include <string.h>#include <time.h>#include <errno.h>#ifndef NTHREADS#define NTHREADS 5#endif#ifndef MSDELAY#define MSDELAY 500#endifstatic pthread_mutex_t main_point = PTHREAD_MUTEX_INITIALIZER;static void *worker(void *payload){const int id = (intptr_t)payload;struct timespec now;clock_gettime(CLOCK_REALTIME, &now);printf("%ld.%09ld: Thread %d grabbing mutex.\n", (long)(now.tv_sec), now.tv_nsec, id);fflush(stdout);pthread_mutex_lock(&main_point);pthread_mutex_unlock(&main_point);clock_gettime(CLOCK_REALTIME, &now);printf("%ld.%09ld: Thread %d continuing.\n", (long)(now.tv_sec), now.tv_nsec, id);fflush(stdout);return NULL;}static unsigned long millisleep(unsigned long msecs){struct timespec req, rem;req.tv_sec = msecs / 1000;req.tv_nsec = (msecs % 1000) * 1000000;if (nanosleep(&req, &rem) == -1 && errno == EINTR)return 1000 * (unsigned long)rem.tv_sec + rem.tv_nsec / 1000000;return 0;}int main(void){pthread_t ptid;pthread_attr_t attrs;struct timespec now;int i, result;if (pthread_attr_init(&attrs)) {fprintf(stderr, "Cannot initialize pthread attributes: %s.\n", strerror(errno));return EXIT_FAILURE;}/* These worker threads do not need more than the minimum stack.Create an attribute to tell the C library about this setting. */pthread_attr_setstacksize(&attrs, PTHREAD_STACK_MIN);/* Lock the main_point mutex. */pthread_mutex_lock(&main_point);clock_gettime(CLOCK_REALTIME, &now);printf("%ld.%09ld: Main thread started.\n", (long)(now.tv_sec), now.tv_nsec);fflush(stdout);for (i = 0; i < NTHREADS; i++) {result = pthread_create(&(ptid), &attrs, worker, (void *)(intptr_t)(i + 1));if (result) {fprintf(stderr, "Failed to create thread %d of %d: %s.\n", i + 1, NTHREADS, strerror(errno));return EXIT_FAILURE;}}clock_gettime(CLOCK_REALTIME, &now);printf("%ld.%09ld: Main thread sleeping for %d milliseconds.\n",(long)(now.tv_sec), now.tv_nsec, MSDELAY);fflush(stdout);millisleep(MSDELAY);clock_gettime(CLOCK_REALTIME, &now);printf("%ld.%09ld: Main thread woken up; releasing mutex...\n", (long)(now.tv_sec), now.tv_nsec);fflush(stdout);pthread_mutex_unlock(&main_point);clock_gettime(CLOCK_REALTIME, &now);printf("%ld.%09ld: Main thread mutex released. Reaping worker threads...\n", (long)(now.tv_sec), now.tv_nsec);fflush(stdout);for (i = 0; i < NTHREADS; i++) {result = pthread_join(ptid, NULL);if (result) {fprintf(stderr, "Warning: Could not reap thread %d of %d: %s.\n", i + 1, NTHREADS, strerror(result));}}clock_gettime(CLOCK_REALTIME, &now);printf("%ld.%09ld: All done.\n", (long)(now.tv_sec), now.tv_nsec);fflush(stdout);return EXIT_SUCCESS;}Compile this using e.g.gcc -Wall -O2 -DNTHREADS=8 -DMSDELAY=250 example.c -lpthread -o exampleand run using e.g../example | sort -nTheNTHREADSdefines the number of threads, andMSDELAYhow long the main thread sleeps after creating the threads. The output timestamps are in seconds since Unix Epoch, real time. Thesort -ncommand sorts the output according to the timestamps.On my system, this outputsCode:1585648512.613949989: Main thread started.1585648512.614027646: Thread 1 grabbing mutex.1585648512.614046279: Thread 2 grabbing mutex.1585648512.614060944: Thread 3 grabbing mutex.1585648512.614078104: Thread 4 grabbing mutex.1585648512.614093435: Thread 5 grabbing mutex.1585648512.614136953: Main thread sleeping for 250 milliseconds.1585648512.614142279: Thread 6 grabbing mutex.1585648512.614146957: Thread 7 grabbing mutex.1585648512.614157421: Thread 8 grabbing mutex.1585648512.864261296: Main thread woken up; releasing mutex...1585648512.864304643: Main thread mutex released. Reaping worker threads...1585648512.864377347: Thread 1 continuing.1585648512.864460867: Thread 2 continuing.1585648512.864515911: Thread 3 continuing.1585648512.864538436: Thread 4 continuing.1585648512.864579944: Thread 5 continuing.1585648512.864601765: Thread 6 continuing.1585648512.864620224: Thread 8 continuing.1585648512.864649748: Thread 7 continuing.1585648512.864709077: All done.which shows that all eight threads continued within 388 microseconds (0.000388 seconds) of the main thread releasing the mutex.This works for situations where you have a "checkpoint" where you want threads to wait, until a specific thread lifts that "checkpoint".

author: 0db
author_rank: Frequent Contributor
timestamp: March 31, 2020, 11:04:03 am
content: Perfect!

author: Nominal Animal
author_rank: Super Contributor
timestamp: March 31, 2020, 12:04:57 pm
content: Let me know if you want to see some other example, like thread pool/work pool, etc. You could say I'm a POSIX C enthusiast.In practice, I try to design my locking/synchronization schemes so that a given thread holds at most one lock at a time, to avoid the possibility of deadlocks.When multiple concurrent locks need to be held (say, solving thedining philosophers problem), I write small test programs to stress-test the corner cases of my locking scheme. These require a lot of work to design correctly, so looking at the literature and other resources to examine the patterns others have used, is a good idea.

author: thinkfat
author_rank: Unknown
timestamp: March 31, 2020, 12:18:04 pm
content: For waiting for a particular state, you best use pthread conditional variables. IOW, pthread_cond_wait() and pthread_cond_broadcast() combined with a mutex.

author: Nominal Animal
author_rank: Super Contributor
timestamp: March 31, 2020, 12:49:00 pm
content: Condition variables can be thought of as a parking lot, with the associated mutex being the single entry-exit gate or bottleneck -- the kind you need to grab/own to get through.In my example, I only have the gate mutex. Because the other threads pile up without issues while the main thread holds the gate, there is no need for a condition variable.A thread pool or producer-consumer example would show how condition variables are used in practice. Typically, you have a mutex, a chain (linked list) or queue or heap or array of pending work, and one or two condition variables. One condition variable is signaled on whenever a new work item is added (and whenever a thread that grabbed the next work item notices there is more work left). If the number of work items is fixed/limited, then the second condition variable is signaled or broadcast on, whenever work items are completed.In my experience, when there is some variable that reflects the state somehow, protected by a mutex, you end up needing one or more condition variables, for threads to efficiently (without wasting CPU time needlessly) wait for a specific state or just for state changes; with those who change the variable signaling or broadcasting on the corresponding condition variable.When there is no such variable, as in my example, a simple mutex or semaphore tends to suffice.Consider that a practical result of theKISS principle, and not a computer science axiom!

author: 0db
author_rank: Frequent Contributor
timestamp: March 31, 2020, 06:09:28 pm
content: Quote from: Nominal Animal on March 31, 2020, 12:04:57 pmThank you, you are very kind. I have already applied your tip to my code, and it works marvelously, but it also would be great to see more examples.I am developing a straight packet sniffer like tcpdump, with a couple of additional threads that also implement packet logger since it is useful for network traffic debugging.

author: Nominal Animal
author_rank: Super Contributor
timestamp: April 01, 2020, 01:14:02 pm
content: Quote from: 0db on March 31, 2020, 06:09:28 pmThat sounds very much like a producer-consumer setup.The main problem I've found in such cases is handling memory use sensibly. Logging a packet can take a relatively long time, and they tend to occur in bursts, so you do want to use a buffer to cache them.Each IP packet can be up to 65535 bytes long, but typically they are much shorter. Depending on your read/receive mechanism, you almost always will do a memory move from the receive buffer to a queue item structure (that will include additional fields,CLOCK_REALTIMEtimestamp at minimum). In some cases, you have a classifier thread, that examines each received packet, and either discards them, or forwards them to a processing queue/heap. (A heap is often useful if the packets have different logging/processing priorities; a priority or arrival-time min-heap is surprisingly powerful for this.)The memory problem is that since some of the packets need to be kept longer than others, you get severe memory fragmentation. (In particular, it is important to remember that in most operating systems, chunks of memory of up to say 120k, allocated viamalloc(), are not returned to the OS whenfree()d, just kept in the allocation pool for that particular process. This excarberates the memory fragmentation and memory use for a logger that just mallocs and frees each packet.)There are several approaches used to mitigate this effect. You can use memory "caches" or "slabs", that contain fixed-size units. For example, you could have 1.5k (covering typical Ethernet 1500-byte payload packets), 3k, 6k, and so on, with earliest unused unit in each slab used first, with a separate index-to-slab slot mapping, so that an unit can be moved within the slab if necessary to significantly shrink it. (Yes, to access a slab, you'll end up dereferencing a "pointer" twice, but that cost is acceptable considering the memory use efficiency. IIRC, I first saw this in classic Mac OS, version 7.5, I believe.)(Many developers have powerful workstations with SSD storage, and often miss how their applications behave on a more typical machine, especially one with a spinny-disk HDD; and their application can become a swap-happy memory hog if nothing happens for long enough to the HDD to spin down, and then a stream of logged packets come in -- and due to the memory fragmentation, the OS cannot keep up with the memory use and I/O load at the same time, bogging everything down, and in the worst case, invokes the OOM killer... Yuk.)Compared to the memory management details (to make the logger truly robust), the threads are almost trivial!If you are implementing a transparent filter/logger, say on a small Linux SBC with two 1GbE interfaces, then surprisingly the memory management becomes simpler, as the underlying protocol is allowed to drop packets due to congestion. (When logging, you really should try to avoid that happening.) Such bridge gadgets are especially useful in analyzing exactly what internet traffic some devices have, and also to act as hyper-paranoid firewalls/loggers for experimental networks (say, isolating a subnet with devices that use customized IP stacks, or IP stacks you developed yourself).This is exactly the kind of task where software development really becomes software engineering. You cannot rely on standards and assumptions on what "should" happen or almost always happens; it needs to be careful and robust. For debugging reasons, you need to check for errors even if those errors rarely occur in practice; knowing where reality first differed from expectations when things go b0rk is important. Otherwise, your service is like a voltmeter that draws a significant current from what it is measuring, affecting the system being logged so badly that the results are completely irrelevant.On the other hand, this is an excellent opportunity to exercise proper engineering principles, from source control (are you using Git yet?), to unit testing. For example, when you implement the packet management stuff (as header and source files), you can write a test program that stress-tests it using generated packets, and measures the memory use/fragmentation, and speed. If you write a new one, you can use that test program to verify the new one is actually better. Similarly for the thread management and other stuff.Do not forget to keep your documentation up to date! In the code, the comments should describe your intent, the overall purpose of the code, and not what the code does. We can read the code, so we can tell what it does; but we humans won't knowwhy, unless the developer explained the reasons or described the algorithm in comments.Again, if you run into problems or out of ideas on how to implement some specific scenario, do ping me; I'd be happy to help anyone interested in writing robust POSIX C code.

author: magic
author_rank: Super Contributor
timestamp: April 02, 2020, 09:06:45 am
content: The main thread is a thread like any other and all pthread functions work in it.I also recommend having a look at the _timedwait versions of pthread wait functions, this can be handy when things go wrongAnd newest versions of C and C++ have built-in standard threading functions, which work on all platforms.

author: guenthert
author_rank: Frequent Contributor
timestamp: April 21, 2020, 11:31:28 pm
content: Quote from: magic on April 02, 2020, 09:06:45 amThat isn't quite the case. Ok, this is perhaps nitpicking, but there are subtle differences between the main thread and others in Linux, most significant perhaps being that the process terminates, if the main thread does.

author: Nominal Animal
author_rank: Super Contributor
timestamp: April 22, 2020, 12:52:10 am
content: Quote from: guenthert on April 21, 2020, 11:31:28 pmActually, it is even more complicated than that. When the main thread exits in Linux, the C library calls theexit_groupsyscall, which causes the process to terminate. If the main thread were to exit calling theexitsyscall, the other threads would continue as normal. The kernel only terminates the process (in the sense that a parent can reap its exit status) after the final thread in the thread group exits (or any thread calls the exit_group syscall, or the process is terminated by a signal).So, strictly speaking, it is the C library that treats the thread that runsmain()specially in Linux; to the kernel, it is just one thread in the thread group.This is not the only Linux kernel wrinkle the C library hides. For example, the Linux kernel treats credentials (user and group) as per-thread attributes, not per-process as POSIX states. When one callssetuid(),setgid(),setresuid(),setresgid(), etc., the C library actually uses an internal real-time signal (viart_tgsigqueueinfo()) to each thread in the thread group to synchronize them.(Edited to add: So, if you ever write code where one thread changes the user/group, and releases say a mutex or signals on a condition variable to let another thread go on with some input/output thing that needs the new credentials, funnn-ky race conditions await. These are rare things, however, and at least us oldtimers keep a very suspicious eye on those, because bugs in that kind of code has bitten us before: it once lead to an exploitable root hole via FTP, for example. Just had to send a Ctrl+C at the right moment, and you got root on the server. But anything fiddling with capabilities or credentials needs several suspicious pairs of eyes look-through it, anyway.)Unless you play withseccomp(), or write freestanding C to run directly under the Linux kernel without the standard C library, this does not matter; and it is more practical to just think of the main thread as "slightly special", yes.

author: SiliconWizard
author_rank: Super Contributor
timestamp: April 22, 2020, 03:50:03 pm
content: Quote from: Nominal Animal on April 22, 2020, 12:52:10 amIndeed. For most practical matters, it doesn't make much difference. Obviously there are some things you can't do with the "main thread", such as "join" it - it wouldn't make sense.