There is a saying among programmers that reminds us of the importance of multi threading in applications, “Parallel computing is our future. And that's forever!” It gives us a perspective to what’s involved in parallel computing and hints at its complexity. We need to add it, and once it is in, we can never remove it.
Work on implementing multi-threaded computing in our C3D Toolkit began several years ago, and continues to this day. Support for multi-threading in the toolkit includes the following functions:
- Employing multi-threaded calculations by the kernel
- Guaranteeing data thread safety in the kernel, and so providing security to user applications that interface with the kernel during parallel computing
OpenMP is an open standard that adds parallelization to software products through programming languages like С, С++, and FORTRAN. Applications that use OpenMP also solve cross-platform and compatibility issues. We use OpenMP technology for the multi-threaded mechanism in our C3D Toolkit.
It turns out, however, that compilers support OpenMP to varying degrees. Intel’s current C++ compiler implements OpenMP v4.5 and only some features of OpenMP v5.0, while Microsoft’s C++ compiler supports only OpenMP v2.0.
Parallel Computing In The C3D Toolkit
In the C3D Toolkit, we implemented multi-threaded operations primarily for the following actions:
- Building flat projections
- Calculating polygon meshes
- Calculating mass-inertial properties
- Converting file formats
Multi-threaded operations are also available in the C3D Toolkit for functions in addition those in the list.
Implementing parallel processing of independent data is, as a rule, relatively simple and effective. However, sometimes issues can be encountered when objects that are working in different threads depend on one another, such as when they all use the same data set, or when one object takes part in another thread calculation. Then the thread safety problem of accessing data comes first.
The easiest solution is to use locking (mutual exclusion or "mutex"), in which a thread gets exclusive access to shared data by blocking other threads from accessing it. Such a straightforward method often leads to increased waiting times, which may reduce the otherwise potential benefits of parallel computing, and sometimes even slow down the overall computing speed.
In the C3D kernel, we take a different approach by providing a special mechanism known as “multi-threaded caching,” as described next.
Thread Safety for Data Structures
One optimization method used in computing is caching. The idea behind caching is that certain important parameters, on which calculations are performed, are not arbitrary, yet obey some predefined or statistically predictive rules. This allows for repeated use of pre-calculated results.
Using cached data is usually effective for consequent calculations under parallel computing, and so can provoke so-called data racing when several threads compete to access the shared cached data.
How Multi-thread Caching Works
The multi-thread caching mechanisms provides thread-safe access to data structures in our kernel, and so enables effective parallelization in which data is processed by several threads simultaneously. Each thread creates its own copy of the cached data. This prevents competition for data between threads, and minimizes the use of locks.
A cache manager controls each object’s multi-thread caches and is responsible for creating, keeping, and retrieving cached data for the object of the current thread. The multi-thread cache mechanism is managed by the multi-threading modes of the C3D Toolkit.
Multi-threaded caching works effectively for both sequential computing and parallel computing, and so it provides an indisputable benefit when transitioning to multi-threaded caches as it requires minimal changes to existing code.
Multi-threading Modes of the C3D Toolkit
The C3D Toolkit offers several multi-threading modes to manage the thread safety of structures. The modes determine which operation will run in parallelized mode:
- Off mode – multi-threading in the C3D Toolkit is turned off and all operations are performed sequentially; the thread-safety mechanism is also disabled
- Standard mode – this is the standard multi-threading mode, which enables limited parallelism in the C3D Toolkit. Parallelization is performed only on operations that process independent data; thread-safety of data structures are disabled in the C3D Toolkit
- SafeItems mode – like Standard mode, but thread safety mode with multi-thread caching mechanism is enabled; this option was developed to support multi-threaded operations in user applications
- Items mode – maximum multi-threading mode, where the mechanism for multi-thread caching is on, and all operations that can be carried out in parallel are done so by the C3D Toolkit
Our C3D Toolkit provides users with an ability to dynamically change the multi-threading mode. A description of the multi-thread modes of the toolkit’s management of the multi-thread cache can be seen in our online documentation at https://c3d.ascon.net/doc/math/tool__multithreading_8h.html.
Support for Multi-threading in User Applications
In the SafeItems multi-threaded mode and above, all geometric objects in the C3D Toolkit are thread-safe when the multi-thread caching mechanism is enabled. As well, most operations (including those which do not appear to be multi-threaded) are thread-safe and so can be used by several threads when SafeItems or above mode is used.
Locks are implemented in the C3D Toolkit on system-synchronization mechanisms, such as the WinAPI on Windows and the pthread API on Linux. These provide data security to the user interface in user applications with different parallelization mechanisms.
The C3D Toolkit provides interfaces for notifying the toolkit about the start and the end of parallel calculations. Important! Custom applications that make calls to the C3D interface from several threads must notify the C3D Toolkit when starting and finishing parallel calculations.
A description of the synchronization interface of the C3D kernel can be found through our online documentation at https://c3d.ascon.net/doc/math/tool__mutex_8h.html.
Managing the C3D Toolkit’s Multi-thread Modes
The C3D Toolkit allows users to dynamically change the multi-thread modes, as follows:
- Managing thread safety of the C3D Toolkit’s data objects by switching on and off multi-thread caching
- Determining which C3D Toolkit operations will parallelize – all, or only ones that process independent data
These switches are useful when internal paralleled C3D cycles have an impact on external parallel performance in user applications. Important! It is important that you analyze the necessity of switchable multi-thread C3D modes for every particular case.
A description of the interface for viewing and managing modes in the multi-thread toolkit can be seen at our online documentation: https://c3d.ascon.net/doc/math/class_math.html.
Using the C3D interface in Multi-threaded Processing
Before you start to use the C3D interface in several threads, it is necessary to be sure that the toolkit is set to multi-thread mode of at least SafeItems or above. By default, the toolkit operates in the maximum multi-thread mode.
When parallel mechanisms different from OpenMP are used, the user application is obliged to notify the toolkit when starting a parallel session and when finishing it. This can be applied in any of the following ways:
- class ParallelRegionGuard protects parallel regions within a scope
- functions EnterParallelRegion and ExitParallelRegion
- Macros ENTER_PARALLEL and EXIT_PARALLEL
Examples of the C3D Toolkit’s notifications in entering and leaving the parallel regions.
Conclusion
The C3D Toolkit can be configured for specific customer needs for using multi-threading in custom applications. The toolkit supports multi-threading in user applications, provides thread-safety of operations in the toolkit, and allows thread-safe data access to data objects of the toolkit.
In conclusion, we note that the C3D Toolkit actively uses its internal mechanism of multi-thread computing. The primary feature of this toolkit is your ability to choose between different multi-thread modes for your mathematical calculations, but we also allow users to dynamically change between internal multi-threading modes.
Author:
Tatiana Mitina
Head of Programming Department at C3d Labs
