Every PKCS#11 integration starts the same way: load the module, call C_Initialize, pass CKF_OS_LOCKING_OK because the application is multi-threaded. The flag reads like a declaration that the library is safe to call from several threads. It is a request, and the standard is explicit about what a library that cannot honour it must do.

What the standard requires

PKCS#11 v2.40 sets out four cases for C_Initialize, by the flag and by whether the application supplies its own mutex callbacks. Almost everybody uses the second:

If the flag is set, and the function pointer fields aren’t supplied (i.e., they all have the value NULL_PTR), that means that the application will be performing multi-threaded Cryptoki access, and the library needs to use the native operating system primitives to ensure safe multi-threaded access. If the library is unable to do this, C_Initialize should return with the value CKR_CANT_LOCK.

And the return code has its own definition:

CKR_CANT_LOCK: This value can only be returned by C_Initialize. It means that the type of locking requested by the application for thread-safety is not available in this library, and so the application cannot make use of this library in the specified fashion.

The contract is clear: ask for OS locking, and a library that cannot provide it says so. What it must not do is accept the request and then fall over.

What a current provider does instead

We measured a commercial provider that reports itself via C_GetInfo as Aktiv Co., Rutoken ECP PKCS #11 library, library version 2.21, Cryptoki 2.40 — the vendor’s current macOS package, released two weeks before this article. Token: Rutoken ECP. Host: macOS.

The reproducer depends on nothing but the cryptoki crate. Two threads meet at a barrier and call C_Initialize at the same moment, both passing CKF_OS_LOCKING_OK with no mutex callbacks — case 2 of the standard.

use cryptoki::context::{CInitializeArgs, CInitializeFlags, Pkcs11};
use std::sync::{Arc, Barrier};
use std::thread;
fn main() {
    let path = std::env::var("PKCS11_MODULE_PATH").expect("set PKCS11_MODULE_PATH");
    let barrier = Arc::new(Barrier::new(2));

    let threads: Vec<_> = (0..2)
        .map(|_| {
            let path = path.clone();
            let barrier = Arc::clone(&barrier);
            thread::spawn(move || {
                let ctx = Pkcs11::new(&path).expect("load module");
                barrier.wait();
                ctx.initialize(CInitializeArgs::new(CInitializeFlags::OS_LOCKING_OK))
            })
        })
        .collect();

    for t in threads {
        match t.join().expect("thread panicked") {
            Ok(()) => println!("initialized"),
            Err(e) => println!("refused: {e}"),
        }
    }
}

Point PKCS11_MODULE_PATH at your provider and loop it; each iteration must be a fresh process, since a crashed one reports nothing.

What runsResult over 10–20 runs
Two C_Initialize calls in sequenceCorrect every time: first returns OK, second returns CKR_CRYPTOKI_ALREADY_INITIALIZED
Two threads, serialized by a mutex in our own codeCorrect every time, same pair of results
Two threads, unserialized, flag passedProcess death, 20 out of 20

Not one clean run. The death is not even consistent: nine runs ended in SIGABRT with libc++abi: terminating on stderr, five in SIGSEGV, six in SIGTRAP. That spread is the signature of corrupted state rather than a single bad pointer.

The library never returned CKR_CANT_LOCK. It accepted the request it could not fulfil.

We first hit this a year ago on version 2.14.1, on two token models, where it appeared as an uncaught C++ exception on one and a segmentation fault on the other. Seven minor releases later, the reproducer still works.

It is not one vendor

The same defect has a history across the ecosystem, and the open-source cases are public. OpenSC carried it until 2021; the pull request that fixed it describes the mechanism exactly:

C_Initialize may be called by multiple threads. But while trying to setup an OpenSC context, setup global_locking and detect cards, it is possible that multiple threads may be trying to do this, as the only test is “if (context == NULL)” but this may not be set until it is too late, and things may be overwritten such as existing context, causing additional problems, with pcsc.

The fix was a mutex around initialization, merged in January 2021. tpm2-pkcs11 had a related failure where C_OpenSession deadlocked precisely when C_Initialize had been called with CKF_OS_LOCKING_OK, reported in 2018.

A guard written as if (context == NULL) is the recurring root cause: it looks like initialize-once and it is a race. The flag makes it worse — an application that passes it believes concurrency is handled, so it calls from several threads, which is the precondition for the bug.

Why this matters more in an authentication module

In a browser or a server, a crash in a PKCS#11 path is an outage. In a PAM module it is worse. pam_tessera is a shared library loaded into someone else’s process — sshd, login, a display manager. A process death there is not a failed operation with an error code; it is the authentication process disappearing. On an unattended device where the same mechanism is the way in, that is a machine nobody can log into until someone physically attends to it.

That asymmetry is why we treat provider concurrency as hostile by default.

What we do about it

Three rules.

Serialize C_Initialize process-wide. Not per handle, not per backend instance — the provider’s defect is global to the process, so the guard has to be too. Our context registry keeps one initialized context per module path and hands out references; a second caller adopts the existing context instead of racing to create one.

Keep asking for OS locking anyway. The flag is still passed on every C_Initialize, because a conforming library needs to hear the request. Asking costs nothing; believing the answer is what costs.

Default to serializing every call, and resolve disagreements upward. Where two components in one process want different locking modes, the stricter one wins: serialization that half the callers opt out of serializes nothing. The cost is one uncontended mutex per call — tens of nanoseconds against a hardware signature that takes milliseconds.

The test that hid it for three years

The uncomfortable part is not the provider. Our own suite called this area healthy the whole time: everything ran against SoftHSM2, which survives the race, and the test named for the concurrent case made no PKCS#11 calls at all — inside it were two threads and a thread::sleep. It measured our own mutex and passed, year after year, propping up a source comment asserting that modern providers handle concurrency correctly.

A green test that never reaches the dependency it is named after is worse than none: it converts an unexamined assumption into apparent evidence. When the suite finally ran against real hardware, the assumption died in the first minute.

If you integrate PKCS#11 yourself: serialize initialization process-wide whatever your provider claims, and make sure at least one test calls the vendor library on the hardware you ship with, not a software token that is more forgiving than what your users will hold.

Honest boundaries

One provider, one host operating system, one token family, one build. We have not retested the Linux build of 2.21, other vendors’ current releases, or calls besides C_Initialize. The result is binary: the process either survives twenty concurrent initializations or it does not.

Nothing here is exploitable from outside — reaching this code means already running inside the process. It is a robustness defect, of the kind that decides whether a fleet of unattended machines stays reachable.

Sources