Concurrency

Introduction Definition According to Wikipedia: Structured concurrency is a programming paradigm aimed at improving the clarity, quality, and development time of a computer program by using a structured approach to concurrent programming. The core concept is the encapsulation of concurrent threads of execution (here encompassing kernel and userland threads and processes) by way of control flow constructs that have clear entry and exit points and that ensure all spawned threads have completed before exit. Such encapsulation allows errors in concurrent threads to be propagated to the control structure’s parent scope and managed by the native error handling mechanisms of each particular computer language. It allows control flow to remain readily evident by the structure of the source code despite the presence of concurrency. To be effective, this model must be applied consistently throughout all levels of the program – otherwise concurrent threads may leak out, become orphaned, or fail to have runtime errors correctly propagated. ...

Concurrency in Python In Python, early concurrency methods were dominated by traditional multi-processing and multi-threading, similar to Java. At the same time, there were many third-party asynchronous solutions (gevent/tornado/twisted, etc.). In the Python 3 era, the official asyncio library and async/await syntax were introduced as Python’s official coroutine implementation, which gradually became popular. Processes Example of multi-processing programming: from multiprocessing import Process def f(name): print('hello', name) if __name__ == '__main__': p = Process(target=f, args=('bob',)) p.start() p.join() The API of multiprocessing is close to threading, making it relatively easy to create multi-process programs. It is a solution recommended by Python officially to bypass the GIL restriction of multi-threading. However, it should be noted that the parameters for creating a process need to be pickle-serializable. It is best to use process-safe data structures like Pipe and Queue (Official Programming guidelines). ...