How RDP smart card logon works

2025-11-24T00:00:00+00:00

Intro</h1>
Around two years ago, FreeRDP got smart card logon support: Smartcard logon with FreeRDP</a>. Theoretically (and sometimes practically), it is possible to use a smart card (scard) for RDP logon in FreeRDP. In this post, I will explain how RDP scard logon works and how to set it up.
This post has two main parts: theoretical and practical. I was thinking about two separate posts, but then I decided that I don't want to split theory and practice.
After reading this article, you should have enough understanding and skills to set up and troubleshoot your own RDP smart card logon.
Note: It is much easier to do smart card logon on Windows, so I assume we run FreeRDP on macOS and our target machine runs Windows.
You may be surprised (sarcasm), but it is not easy to do scard logon on macOS when connecting to the Windows machine. It became easier with recent improvement</a> in `sspi-rs</code>. I have a lot to say, let's dive in.`

`How it works</h1> Each section explains the theoretical aspects of RDP smart card logon and the software component we will use for it. Sometimes it is not trivial because not all Microsoft components have exact replacements on macOS.`RDP NLA</h2> Now we need to understand how the scard authorization works. Without this knowledge, we can't move further. The RDP is very complicated. So, I'll focus my attention only on scard-related things. So, the first thing you need to know is Network Level Authentication</a>: Network Level Authentication (NLA) is a feature of RDP Server or Remote Desktop Connection (RDP Client) that requires the connecting user to authenticate themselves before a session is established with the server. ... Network Level Authentication delegates the user's credentials from the client through a client-side Security Support Provider and prompts the user to authenticate before establishing a session on the server. </blockquote> Basically, NLA says: "Let's authenticate the user (and the server, if the Kerberos protocol is used) before opening any GUI windows at the beginning of the RDP connection". And only when the user passes the authentication, the NLA will transfer credentials to the target server. In turn, the RDP server will use these credentials to open the session. NLA is not a concrete protocol but a general principle. In RDP, the CredSSP</a> protocol is responsible for the NLA. flowchart LR subgraph CREDSSP["CredSSP"] subgraph SPNEGO["SPNEGO"] subgraph ApplicationProtocol["Negotiated Application Protocol (NTLM, Kerberos, etc)"] end end end </div> The CredSSP protocol performs the user authentication (using NTLM/Kerberos/SPNEGO protocols inside</a>) and transfers the user's credentials to the target server. The CredSSP protocol is responsible only for credential delegation. It doesn't authenticate clients or servers. An actual authentication is performed by the inner application protocol, like NTLM or Kerberos (preferably). But Microsoft would not be Microsoft if it did not add another layer of abstraction. The CredSSP uses the SPNEGO framework</a>: (quote src</a>) SPNEGO provides a framework for two parties that are engaged in authentication to select from a set of possible authentication mechanisms. ...The CredSSP Protocol uses SPNEGO to mutually authenticate the CredSSP client and CredSSP server. It then uses the encryption key that is established under SPNEGO to securely bind to the TLS session. </blockquote> Let's summarize it. The SPNEGO selects (negotiates) an appropriate authentication protocol and performs authentication using it. As a result, we'll have an established security context with a secure encryption key. In turn, the CredSSP will use this key to encrypt the credentials and pass (delegate) them to the target CredSSP (RDP) server. FreeRDP implements the CredSSP protocol, so there are no pitfalls. Its CredSSP implementation can rely on either the built-in NTLM implementation, the MIT KRB5 implementation</a>, or an external /sspi-module:</code> — a dynamic library that implements the SSPI interface</a>. Spoiler: We will use an external library. Good. Now you have a brief overview of the NLA. Now it's time to move forward. Kerberos</h2> To log on with a smart card, we need SPNEGO to select Kerberos as the authentication protocol. Kerberos is the only authentication protocol that can be used for scard logon. To support password-less logon, Kerberos includes the PKINIT extension: Public Key Cryptography for Initial Authentication in Kerberos (PKINIT)</a>. The only difference between password-based Kerberos and scard-based is in the AS exchange (The Authentication Service Exchange</a>). During password-based logon, the AS exchange session key is derived from the user's password: // https://github.com/Devolutions/picky-rs/blob/628bbcab3100a782971261022f0ec91b4f4549f5/picky-krb/src/crypto/aes/key_derivation.rs#L39-L51 pub fn derive_key_from_password<P: AsRef<[u8]>, S: AsRef<[u8]>>( password: P, salt: S, aes_size: &AesSize, ) -> KerberosCryptoResult<Vec<u8>> { let mut tmp = vec![0; aes_size.key_length()]; pbkdf2_hmac::<Sha1>(password.as_ref(), salt.as_ref(), AES_ITERATION_COUNT, &mut tmp); // For AES encryption (https://www.rfc-editor.org/rfc/rfc3962.html#section-6): // > random-to-key function `` identity function let temp_key = random_to_key(tmp); // A Key Derivation Function (https://www.rfc-editor.org/rfc/rfc3961.html#section-5.1). derive_key(&temp_key, KERBEROS, aes_size) }</code></pre> Essentially, it means that if someone knows the user's password, they can decrypt the AsRep</code> encrypted part, extract a session key, and decrypt-and-extract all other sequential keys in the Kerberos authentication (e.g., TGS exchange session key). Thus, always enforce a strong password policy 🙃. However, I am not here to talk about Kerberos attack vectors and vulnerabilities. So, what is different in the scard-based logon? During the scard-based logon, the AS exchange session key is derived using the the Diffie-Hellman Key Exchange</a>. PKINIT also defines the Public Key Encryption</a>, but I haven't seen it in action ever. I always see the Diffie-Hellman Key Exchange in the Wireshark capture of the mstsc</code>. Diffie–Hellman (DH) key exchange</a> is a mathematical method of securely generating a symmetric cryptographic key over a public channel. </blockquote> It means that two parties can securely establish an encryption key over a public network. There are tons of articles explaining how the Diffie-Hellman Key Agreement works. Let me paste only a small description and move on. The client generates DH parameters and a private key.</li> Using the DH parameters and private key, the client generates the public key.// https://github.com/Devolutions/picky-rs/blob/628bbcab3100a782971261022f0ec91b4f4549f5/picky-krb/src/crypto/diffie_hellman.rs#L145-L149 /// [Key and Parameter Requirements](https://www.rfc-editor.org/rfc/rfc2631#section-2.2) /// y is then computed by calculating g^x mod p. pub fn compute_public_key(private_key: &[u8], modulus: &[u8], base: &[u8]) -> DiffieHellmanResult<Vec<u8>> { generate_dh_shared_secret(base, private_key, modulus) }</code></pre></li> The client sends DH parameters and public key to the server (KDC in our case).</li> The server generates its own private and public keys.</li> Both parties can generate the shared secret (symmetric encryption key) using the DH algorithm:// https://github.com/Devolutions/picky-rs/blob/628bbcab3100a782971261022f0ec91b4f4549f5/picky-krb/src/crypto/diffie_hellman.rs#L94-L112 /// [Using Diffie-Hellman Key Exchange](https://www.rfc-editor.org/rfc/rfc4556.html#section-3.2.3.1) /// let DHSharedSecret be the shared secret value. DHSharedSecret is the value ZZ /// /// [Generation of ZZ](https://www.rfc-editor.org/rfc/rfc2631#section-2.1.1) /// ZZ = g ^ (xb * xa) mod p /// ZZ = (yb ^ xa) mod p = (ya ^ xb) mod p /// where ^ denotes exponentiation fn generate_dh_shared_secret(public_key: &[u8], private_key: &[u8], p: &[u8]) -> DiffieHellmanResult<Vec<u8>> { let public_key = BoxedUint::from_be_slice_vartime(public_key); let private_key = BoxedUint::from_be_slice_vartime(private_key); let p = Odd::new(BoxedUint::from_be_slice_vartime(p)) .into_option() .ok_or(DiffieHellmanError::ModulusIsNotOdd)?; let p = BoxedMontyParams::new_vartime(p); // ZZ = (public_key ^ private_key) mod p let out = pow_mod_params(&public_key, &private_key, &p); Ok(out.to_be_bytes().to_vec()) }</code></pre></li> Using the derived encryption key, the server encrypts all needed data and sends it to the client alongside the DH public key.</li> </ol> Of course, I've omitted many technical details, but I'm here to convey the overall idea, not to teach you cryptography. The astute reader may notice that the smart card is not required for the Diffie-Hellman exchange. The smart card is needed ONLY for signing the digest of all these public authentication parameters: DH parameters, DH public key, nonce, etc (source code</a>). The KDC will verify the signature using the client's certificate. That's all 🙃. Someday I will write a detailed post about Kerberos scard auth, but today is not that day 😬. The FreeRDP itself does not implement the Kerberos protocol. It uses the external library for that instead. FreeRDP's CredSSP implementation uses the SSPI interface</a> to perform the authentication. Here we have a few options: Enable the WITH_KRB5</code> and WITH_KRB5_MIT</code> flags and provide MIT KRB5</a> library for FreeRDP (see FreeRDP/wiki/Compilation</a>). FreeRDP has a wrapper over MIT KRB5 to export its functionality via the SSPI interface: FreeRDP/winpr/libwinpr/sspi/Kerberos/krb5glue_mit.c</a> + FreeRDP/winpr/libwinpr/sspi/Kerberos/kerberos.c</a>.</li> Provide our own SSPI implementation using the /sspi-module</code> arg. It can be any library that implements the SSPI interface.</li> </ol> Surprisingly, we will follow the second path. We will use the following open-source SSPI implementation: github/Devolutions/sspi-rs</a>. It supports both Kerberos password-based and scard-based logon and has various other pros: Easier to set up and configure.</li> Easier logs reading and configuring.</li> Do not need to install new packages (e.g., libkrb5-dev</code>) and link them with FreeRDP.</li> Implemented in Rust 🙃.</li> </ul> Scard credentials</h2> During password-based logon, the CredSSP client transfers the user's username, password, and domain to the target server. In turn, the target server tries to log in the user using these credentials. But we cannot send the smart card to the target server 🙃. According to the specification, smart card credentials are defined as follows: -- https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-cssp/94a1ab00-5500-42fd-8d3d-7a84e6c2cf03 TSCredentials ::= SEQUENCE { credType [0] INTEGER, credentials [1] OCTET STRING } -- credType = 2. Credentials contains a TSSmartCardCreds structure that defines the user's smart card credentials. -- https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-cssp/4251d165-cf01-4513-a5d8-39ee4a98b7a4 TSSmartCardCreds ::= SEQUENCE { pin [0] OCTET STRING, cspData [1] TSCspDataDetail, userHint [2] OCTET STRING OPTIONAL, domainHint [3] OCTET STRING OPTIONAL } -- https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-cssp/34ee27b3-5791-43bb-9201-076054b58123 TSCspDataDetail ::= SEQUENCE { keySpec [0] INTEGER, cardName [1] OCTET STRING OPTIONAL, readerName [2] OCTET STRING OPTIONAL, containerName [3] OCTET STRING OPTIONAL, cspName [4] OCTET STRING OPTIONAL }</code></pre> I want to go through every field and explain its meaning: credType</code> = 2</code> (according to the specification). </li> pin</code> - smart card PIN code. </li> userHint</code> - username. Important: the username must be only username. The target server may reject</a> the FQDN. t2@example.com</code> ❌ -> t2</code> ✅. </li> domainHint</code> - domain. </li> keySpec</code> = AT_KEYEXCHANGE</code> = 1</code>. (AD FS and certificate KeySpec property information</a>): The KeySpec property identifies how a key, that is generated or retrieved using the Microsoft CryptoAPI (CAPI) from a Microsoft legacy Cryptographic Storage Provider (CSP), can be used. </blockquote> Typically, this value equals AT_KEYEXCHANGE</code>. Since this value is not encoded in the certificate or smart card, we can set it manually in the credentials. </li> cardName</code> - smart card name. It can be any name. </li> cspName</code>. Cryptographic Service Provider</a>: An independent software module that actually performs cryptography algorithms for authentication, encoding, and encryption. </blockquote> In our case, it is always equal to Microsoft Base Smart Card Crypto Provider</code></a>. It supports smart cards and implements algorithms for hashing, signing, and encrypting data. We do not need to know internals, but only that it uses smart cards for crypto operations. </li> readerName</code>. (quote src</a>): smart card reader name: The friendly, human-readable name of the smart card reader. Also referred to as a Reader Name.</li> smart card reader: A device used as a communication medium between the smart card and a Host; for example, a computer. Also referred to as a Reader.</li> </ul> </blockquote> The system automatically generates the Reader Name on Windows. But we need to take it from somewhere on macOS. In short, the PKCS11 slot description is used as the reader name. </li> containerName</code>. (quote src</a>): key container - A part of the key database that contains all the key pairs (exchange and signature key pairs) belonging to a specific user. Each container has a unique name used when calling CryptAcquireContext</code> to obtain a handle to the container. </blockquote> Therefore, the container name is a unique string that represents the smart card certificate and its private key. And, of course, it is internally generated during scard certificate enrollment on Windows. We cannot precalculate or extract it from somewhere. A small trick is used to generate this value: // https://github.com/Devolutions/sspi-rs/blob/dea60b5ef60932efdbf22e0806954d8c236fbb78/ffi/src/winscard/piv.rs#L134C1-L212 fn chuid_to_container_name(chuid: &[u8], tag: [u8; 3]) -> Result<String> { // ... (some lines are omitted) ... // `chuid` - smart card Card Holder Unique Identifier. let guid = &chuid[BYTES_TO_SKIP..BYTES_TO_SKIP + 16 /* GUID length /]; // Construct the value Windows would use for a PIV key's container name. let container_name = format!( "{:02x}{:02x}{:02x}{:02x}-{:02x}{:02x}-{:02x}{:02x}-{:02x}{:02x}-{:02x}{:02x}{:02x}{:02x}{:02x}{:02x}", guid[3], guid[2], guid[1], guid[0], guid[5], guid[4], guid[7], guid[6], guid[8], guid[9], guid[10], guid[11], guid[12], tag[0], tag[1], tag[2] ); Ok(container_name) }</code></pre> This is how FreeRDP constructs</a> a container name for the certificate. It takes a smart card CHUID, extracts a GUID value, and combines it with the certificate PIV tag to construct a unique container name. Since every smart card CHUID is unique and there is only one certificate per slot, the constructed value is guaranteed to be unique. Moreover, the uniqueness is not the critical part. The crucial part is that the container name format must meet Windows expectations. Last year, in one of my posts, I explained and provided an example of how a bad container name can cause issues in smart card logon: tbt.qkation.com/posts/scard-container-name/#what</a>. </li> </ul> Scard driver</h2> As explained above, the Kerberos protocol does client and server authentication. And, as we can logically assume, it relies on some APIs to communicate with the smart card. Let's first discuss how it works on Windows, then move on to macOS. Look at this diagram from the MSDN</a>: The Kerberos implementation uses the Windows high-level cryptographic API to communicate with the smart card. Using the certificate hash</a>, it can get the user's certificate, container name, and all other smart card credentials. Windows allows using vendor-specific CSPs, but the default basecsp.dll</code> is used during RDP smart card auth. In turn, the basecsp.dll</code> uses the smart card minidriver to perform cryptographic operations. Smart Card Minidrivers</a>: The smart card minidriver provides a simpler alternative to developing a legacy cryptographic service provider (CSP) by encapsulating most of the complex cryptographic operations from the card minidriver developer. Applications use CAPI for smart card–based cryptographic services. CAPI, in turn, routes these requests to the appropriate CSP to handle the cryptographic requirements. Although Base CSP can use the RSA capabilities of a smart card only by using the minidriver... </blockquote> Smart Card Minidriver Overview</a>: The card-specific minidriver is the lowest logical interface layer in the Base CSP/KSP. This minidriver lets the Base CSP/KSP and applications interact directly with a specific type of card by using SCRM. </blockquote> In simple words, the smart card minidriver is a small dll provided by the card vendor that implements the Smart Card Minidriver specification</a>. This dll is used by BaseCSP to perform basic cryptography operations. This allows card vendors to reuse Microsoft's cryptography stack and not implement their own CSP: I have the YubiKey 5 Nano</a> smart card and will use it in this article. Of course, YubiKey has its own smart card minidriver implementation</a>, which must be installed on the target machine: Deploying the YubiKey Smart Card Minidriver to workstations and servers</a>. But what about macOS? We do not have minidrivers here... FreeRDP (and sspi-rs</code>) relies on the PKCS11 module (a dynamic library) for executing cryptographic operations. In cryptography, PKCS #11 is a Public-Key Cryptography Standard that defines a C programming interface to create and manipulate cryptographic tokens that may contain secret cryptographic keys. It is often used to communicate with a Hardware Security Module or smart cards. (Wiki</a>) </blockquote> Fortunately, YubiKey also distributes its own PKCS11 library: libykcs11.2.7.2.dylib</code></a>. However, that's not the end: each smart card minidriver communicates with the smart card through a unified API: WinSCard API</a>. ⬇️ WinSCard</h2> The basic parts of the smart card subsystem are based on PC/SC standards (see the specifications at https://pcscworkgroup.com</a>). (src</a>) </blockquote> The WinSCard API enables communications with smart card hardware. It is the lowest API from the software perspective. The WinSCard API is based on the PC/SC (Personal Computer/Smart Card) specification - a globally implemented standard for cards and readers (https://www.smartcardbasics.com/pc-sc/</a>). The advantage of PC/SC is that applications do not have to acknowledge the details corresponding to the smart card reader when communicating with the smart card. This application can function with any reader that complies with the PC/SC standard. (src</a>). </blockquote> On Windows, we have WinSCard API, but on macOS, we have pcsc-lite</a>: PC/SC is the de facto cross-platform API for accessing smart card readers. It is published by PC/SC Workgroup</a>, but the "reference implementation" is Windows. Linux and Mac OS X use the open source pcsc-lite</a> package. (src</a>) </blockquote> Everything looks good at first sight: libykcs11</code> uses pcsc-lite</code>, both of them are available on macOS. Seems like nothing stops us from scard logon. Unfortunately, there is one thing. The target machine tries to open a new session using the transferred smart card credentials. Internally, the same high-level APIs are used except for the WinSCard API. All WinSCard API calls are redirected to the client machine (via [MS-RDPESC]: Remote Desktop Protocol: Smart Card Virtual Channel Extension</a>). Soooo? Obviously, we will call the pcsc-lite</code> API on macOS instead of WinSCard because we do not have WinSCard here. The problem is that the pcsc-lite</code> API does not match the WinSCard API identically. There are differences: Most of them are listed in the documentation: https://pcsclite.apdu.fr/api/group__API.html#differences</a>.</li> pcsc-lite</code> API does not contain some functions as WinSCard. For example: SCardReadCache</code>.</li> Some function has -A</code> and -W</code> variations. For example: SCardConnectA</code> and SCardConnectW</code>.</li> </ul> A pcsc-lite</code> wrapper needed to be implemented to match the original WinSCard API. FreeRDP's implementation</a> is not complete. It lacks a proper WinSCard cache implementation. The target machine expects the client-side to have the required WinSCard smart card cache items. It is a real problem: the target machine will not authenticate the client without a properly working smart card cache. Otherwise, the target machine will display a message such as "This smart card could not be used. Additional detail may be available in the system log. Please report this error to your administrator." or "The requested key container does not exist on the smart card.". If you are interested in smart card cache items, then check my old post: tbt.qkation.com/posts/win-scard-cache/</a>. To my knowledge, the sspi-rs</code>'s FFI module is the only open-source library that implements proper smart card cache items: github/Devolutions/sspi-rs/dea60b5e/ffi/src/winscard/system_scard/context.rs#L283-L564</a>. Now, with this final software component, we can finally set up FreeRDP scard logon 🤩 🥵. Let's put it all together</h2> software component</th> meaning</th> Windows</th> macOS</th></tr></thead> RDP client</td> no comments</td> mstsc.exe</code></td> FreeRDP</td></tr> CredSSP protocol</td> Performs NLA. Transfers user credentials to the target server</td> credssp.dll</code></td> FreeRDP</td></tr> Kerberos protocol</td> Authenticates the user and the server. Established the security context</td> Implemented inside Windows</td> libsspi.dylib</code></a> - Kerberos client implementation with scard logon support</td></tr> Smart card (mini)driver</td> Transforms high-level crypto operations into low-level PCSC commands</td> YubiKey Smart Card Minidriver</a></td> libykcs11.2.7.2.dylib</code></a> - a PKCS#11 module</td></tr> PC/SC</td> Enables communications with smart card hardware</td> WinSCard</td> libsspi.dylib</code></a> - WinSCard-compatible pcsc-lite</code> wrapper with proper smart card cache implementation</td></tr> </tbody></table> At this point, I hope it all makes sense to you 🤞. Setting up</h1> Finally! We've lived enough to start configuring this machinery 🤩. If you would like to replicate the configuration below in your own environment, I have some prerequisites for you: You know how to configure, and even more, you have them configured: Windows Server Active Directory and Certificate Authority.</li> Domain joined-machines to make the RDP connection.</li> </ul> </li> You possess a YubiKey, a PIV-compatible smart card device. The list of suitable devices</a>: YubiKey 5 NFC, YubiKey 5 Nano, YubiKey 5C, and YubiKey 5C Nano provide Smart Card functionality based on the Personal Identity Verification (PIV) interface specified in NIST SP 800-73, “Cryptographic Algorithms and Key Sizes for PIV.” </blockquote> For example, I own a YubiKey 5 Nano. </li> You enrolled in the smart card certificate. </li> </ul> Smart card</h2> I already have a ready-to-go smart card. The user certificate is enrolled and valid: ykman piv info</code></pre> You should have something like that: PIV version: 5.4.3 PIN tries remaining: 3/3 PUK tries remaining: 0/3 Management key algorithm: TDES WARNING: Using default PIN! PUK is blocked Management key is stored on the YubiKey, protected by PIN. CHUID: 3019d4e739da739ced39ce739d836858210842108421c84210c3eb34101df30c281471d9a216caed2291f8e494350832303330303130313e00fe00 CCC: No data available Slot 9A (AUTHENTICATION): Private key type: RSA2048 Public key type: RSA2048 Subject DN: Issuer DN: CN=tbt-WIN-956CQOSSJTF-CA,DC=tbt,DC=com Serial: 00:1c:00:00:00:11:a3:a9:8c:0c:7d:22:25:df:00:00:00:00:00:11 Fingerprint: bbb2ff52231e831d4d7b3c575c8f7bb8c50c941ac2121a0000d68d4d540255e5 Not before: 2025-10-23T17:17:44+00:00 Not after: 2027-10-23T17:27:44+00:00 Slot 9D (KEY_MANAGEMENT): Private key type: RSA2048</code></pre> And here is the certificate itself: t2@tbt.com.cer</code></a>. You can export it from your smart card using the following command: ykman piv certificates export ${SLOT_ID} ${PATH_TO_FILE} # Example: ykman piv certificates export 9a ~/delete_it/t2@tbt.com.cer</code></pre> And the last step in configuring the smart card is installing libykcs11.2.7.2.dylib</code></a>. YKCS11 is automatically built as part of yubico-piv-tool </blockquote> git clone https://github.com/Yubico/yubico-piv-tool.git cd yubico-piv-tool mkdir build; cd build cmake .. make sudo make install</code></pre> When working with smart cards, I like to check if the smart scard works well using the following command: echo "TheBestTvarynka" | pkcs11-tool --module "/usr/local/lib/libykcs11.2.7.2.dylib" -m RSA-PKCS -p 123456 -s --id 01 | base64 # You should get the base64-encoded signature: # SRA+CTXAYJRO3Wz42K9W/4qeHlV2WmY/t6bDumdSOf7+KL1/32E7Cuq55GIdkDnZnCfHZp89Eyq/ # TEcV2GAnbyiMiUqlZb1iBEYyYOf5ovaJ7xy4dPIuEvkJLMVJPpCCJ7Cr9zolpTEmfZ7FMZVbtprS # 04ho4P2lz60eJ00Bykrerwc4FcmIPs7tNX//1EJVNLrlXdEBYlxH8ZOFnYtMUggf80EJJlWxTHjT # ZTz8/+KNZRSFwHR+lZikzxMDaVx4Kiq+2w5NMrSA0MQlFA7fvHOBRh4LBAdHPHzXmgCMWNM1cAbm # h9PrXdMlo24Xdj+EHZRRLsD7HfeVYPWYNhCvoC0tW1RCVF0tPjogR2V0RnVuY3Rpb25MaXN0IHN0 # YXJ0IQotLVtUQlRdLT46IEdldEZ1bmN0aW9uTGlzdCBzdWNjZXNzIQo=</code></pre>WinSCard</h2> As I said above, we will use sspi-rs</code></a> to provide FreeRDP with the custom WinSCard API implementation. # The `sspi-rs` was in commit `ffd920b7b396739b60a6e1bae701f31e735deeef` at the time of writing this article. git clone https://github.com/Devolutions/sspi-rs.git cd sspi-rs/ffi cargo build --features scard file ../target/debug/libsspi.dylib # You should see something like this: # ../target/debug/libsspi.dylib: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=eb4b2d386823f59cbd4bd7ed08055f887db6c45b, with debug_info, not stripped</code></pre> You can confirm that the libsspi.dylib</code> library exports the WinSCard API by running: readelf -Ws --dyn-syms ../target/debug/libsspi.dylib | grep -E " SCard"</code></pre> To configure the WinSCard API implementation, we need to set the following environment variables: name</th> meaning</th> example</th></tr></thead> WINSCARD_USE_SYSTEM_SCARD</code></td> Tells the sspi-rs</code> to use the system-provided (real) smart card instead of an emulated one</td> true</code></td></tr> </tbody></table> The last step in this section is to ensure that the pcsc-lite</code> is installed on your system and running: # I use Arch bwt sudo pacman -S pcsclite sudo systemctl enable pcscd sudo systemctl start pcscd sudo systemctl status pcscd # ● pcscd.service - PC/SC Smart Card Daemon # Loaded: loaded (/usr/lib/systemd/system/pcscd.service; indirect; preset: disabled) # Active: active (running) since Sat 2025-11-08 12:28:03 EET; 5s ago # Invocation: e7d44e3ded3c4a19a388887891599d6b # TriggeredBy: ● pcscd.socket # Docs: man:pcscd(8) # Main PID: 265031 (pcscd) # Tasks: 5 (limit: 18705) # Memory: 1.4M (peak: 2.7M) # CPU: 65ms # CGroup: /system.slice/pcscd.service # └─265031 /usr/bin/pcscd --foreground --auto-exit # Nov 08 12:28:03 tbt systemd[1]: Started PC/SC Smart Card Daemon.</code></pre>Kerberos</h2> As I mentioned above, we will use sspi-rs</code></a> to provide FreeRDP with a Kerberos implementation. sspi-rs</code> exports both SSPI and WinSCard APIs from the same library file. Thus, at this point, we already have the Kerberos implementation exported via the SSPI interface inside libsspi.dylib</code> from the previous step. You can confirm that by checking the exported functions. For example: readelf -Ws --dyn-syms ../target/debug/libsspi.dylib | grep -E "SecurityContext(A|W)?$" # You should see simething like this: # 153: 00000000003066f0 2841 FUNC GLOBAL DEFAULT 12 AcceptSecurityContext # 197: 00000000003bfe30 2900 FUNC GLOBAL DEFAULT 12 InitializeSecurityContextA # 223: 0000000000308f60 2248 FUNC GLOBAL DEFAULT 12 ImpersonateSecurityContext # 233: 00000000003c2850 2272 FUNC GLOBAL DEFAULT 12 ImportSecurityContextA # 234: 00000000003c0990 2900 FUNC GLOBAL DEFAULT 12 InitializeSecurityContextW # 235: 000000000030bb40 2271 FUNC GLOBAL DEFAULT 12 ExportSecurityContext # 257: 00000000003c3130 2272 FUNC GLOBAL DEFAULT 12 ImportSecurityContextW # 269: 0000000000309830 2248 FUNC GLOBAL DEFAULT 12 RevertSecurityContext # 273: 0000000000307c60 2595 FUNC GLOBAL DEFAULT 12 DeleteSecurityContext # 105839: 00000000003c2850 2272 FUNC GLOBAL DEFAULT 12 ImportSecurityContextA # 105866: 00000000003c0990 2900 FUNC GLOBAL DEFAULT 12 InitializeSecurityContextW # 105872: 0000000000307c60 2595 FUNC GLOBAL DEFAULT 12 DeleteSecurityContext # 105878: 00000000003bfe30 2900 FUNC GLOBAL DEFAULT 12 InitializeSecurityContextA # 105887: 00000000003c3130 2272 FUNC GLOBAL DEFAULT 12 ImportSecurityContextW # 105896: 0000000000308f60 2248 FUNC GLOBAL DEFAULT 12 ImpersonateSecurityContext # 105929: 00000000003066f0 2841 FUNC GLOBAL DEFAULT 12 AcceptSecurityContext # 105957: 000000000030bb40 2271 FUNC GLOBAL DEFAULT 12 ExportSecurityContext # 105978: 0000000000309830 2248 FUNC GLOBAL DEFAULT 12 RevertSecurityContext</code></pre> We need to set the following environment variables to configure Kerberos properly: name</th> meaning</th> example</th></tr></thead> SSPI_LOG_PATH</code></td> Path to the logfile</td> ~/delete_it/sspi.log</code></td></tr> SSPI_LOG_LEVEL</code></td> Logging level</td> trace</code></td></tr> SSPI_KDC_URL</code></td> Url to the KDC server or KDC Proxy server. You can omit this value if the DNS is properly configured</a>.</td> tcp://192.168.1.104:88</code></td></tr> SSPI_SCARD_TYPE</code></td> sspi-rs</code> also implements the emulated smart card. We need to specify it to use the system-provided (real) smart card</td> system</code></td></tr> SSPI_PKCS11_MODULE_PATH</code></td> Path to the PKCS11 module</td> /usr/local/lib/libykcs11.2.7.2.dylib</code></td></tr> </tbody></table> FreeRDP</h2> Now to the hard part. Compiling the C/C++ project was always a nightmare for me. Time passes, and I become more skilled in it, but it remains unpleasant. Fortunately, FreeRDP has an excellent compilation guide: FreeRDP/wiki/Compilation</a>. Unfortunately, versions of some dependencies are outdated (at the moment), so I used the macOS bundling script (FreeRDP/scripts/bundle-mac-os.sh</a>) to help me with compilation. Demo</h1> export SSPI_LOG_PATH=sspi.log export SSPI_LOG_LEVEL=trace export SSPI_KDC_URL="tcp://192.168.1.104:88" export SSPI_SCARD_TYPE=system export SSPI_PKCS11_MODULE_PATH=/usr/local/lib/libykcs11.2.7.2.dylib export WINSCARD_USE_SYSTEM_SCARD=true ./sdl-freerdp /v:DESKTOP-QELPR32.tbt.com /u:t2 /d:tbt.com /p:123456 /smartcard-logon /sec:nla \ /cert:ignore /log-level:TRACE \ /sspi-module:/Users/user/delete_it/sspi-rs/target/debug/libsspi.dylib \ /kerberos:pkcs11-module:/usr/local/lib/libykcs11.2.7.2.dylib \ /winscard-module:/Users/user/delete_it/sspi-rs/target/debug/libsspi.dylib </code></pre> It was quite a short demo 😅. But it works! Doc, references, code</h1> Smartcard logon with FreeRDP</a>.</li> Network Level Authentication</a>.</li> CredSSP</a>.</li> Public Key Cryptography for Initial Authentication in Kerberos (PKINIT)</a>.</li> Smart Card Minidrivers</a>.</li> Smart Card Architecture</a>.</li> WinSCard API</a>.</li> </ul> Announcing Dataans v.0.3.0 2025-08-03T00:00:00+00:00 GitHub/TheBestTvarynka/Dataans/releases/v.0.3.0</a>. Intro</h1> The Dataans app has adopted a local-first approach since the very beginning of its existence. After some time, I started to use the app on many devices and inside my Windows/Linux VMs. I caught myself thinking that it would be good to be able to transfer data from one device to another. Fast forward two months, and suddenly I realized that it is a critical feature for me. After some considerations and small research, I understood that I wanted the multi-device synchronization feature. Decentralized Internet, P2P networks, etc, are good. But the current implementation of the data synchronization feature relies on the central sync (backup) server. The data synchronization using the P2P communication may be implemented in the future. It depends on my needs. This post turned out to be quite large. Alternatively, you can read the shorter version: github/TheBestTvarynka/Dataans/6c898a01/doc/sync_server.md</a>. Demo</h1> Before explaining how it works and how I implemented it, I want to show you the demo. Better to see something once than hear about it a thousand times. </blockquote> Asian Proverb</a>. So, first, the user needs to log in. Open the App-info</code> window by clicking on the app version number in the bottom left corner of the main window. Then, type the sync server address and click log in: Now the user needs to authorize: And yes, as you can see, I decided to delegate all authentication-related tasks to Cloudflare. I did not want to implement it manually, and, honestly, it was one of the best decisions I made. Thanks to Cloudflare Zero Trust Access</a>, I only need to enter email and the code from email. After the successful sign in, the user needs to enter the password and secret passphrase. At this point, two cases are possible: If it is the first sign-in among all users' devices (sync server's database is empty), then the user needs to fill in ONLY the password input. The passphrase will be automatically generated by the app. But if the user wants to use a custom passphrase, then they can type it.</li> If it is a new sign-in on another device and the sync server's database is not empty (there was a sync from other devices), then the user must type both password AND passphrase. The user can read the passphrase on any signed-in device in the profile.json</code>.</li> </ol> Why does the app require the user to have a password if all auth has been delegated to Cloudflare? </blockquote> It is explained in detail in the Auth</a> section. But, in short, the data encryption key is derived from the user's password and passphrase. This is what the user sees after successful sign-in: From the screenshot above: Button that triggers data synchronization.</li> Synchronization mode. Currently, only manual mode is implemented. It means that whenever the user wants to sync the data, they need to press the button 🙃.</li> Sign out button.</li> </ol> Now, the user is able to sync app data. The synchronization process happens in the background. So, the user can freely use the app without any limitations during synchronization. Once the sync is complete, a small pop-up message will appear. How it works</h1> Now let's talk about interesting things: how it works 😊. Below are detailed technical explanations. You can jump right to the interesting part. There is no need to read it in order. Infrastructure</h2> This section is boring, because I don't like to describe infrastructure in detail. So, if you are not interested in such things, then skip it 😉. graph TD subgraph Internet Client[Dataans App] end subgraph Cloudflare Client -->|HTTPS Request| CloudflareZeroTrust[Cloudflare Zero Trust Access] CloudflareZeroTrust -->|cf-access-jwt-assertion Injected| FlyApp[Sync Server] subgraph Fly.io FlyApp -->|User files| ObjectStorage[(Tigris Object Storage)] end FlyApp -->|Verifies JWT| CF_PubKey[Cloudflare Public Key Endpoint] end subgraph Neon.com FlyApp -->|User data| Postgres[(PostgreSQL Database)] end </div> The overall infrastructure is straightforward. I use Cloudflare</a> for endpoint protection, auth, DDoS protection, and domain hosting. I use Fly.io</a> as a main cloud provider. I like the microVM approach that they use. It is my first app deployed on fly.io. The deployment process was simple and fast. It reminded me of the old, almost forgotten feeling "it just works". I use Neon.com</a> PostgreSQL provider to store user data. I did not like fly.io's Postgres integration because it does not shut down automatically when it is not in use. So, I decided to try Neon. They have a very limited free tier, but it is ok for me (at least for now). I am very satisfied with the database start time. I think it is a great choice for small personal projects. I use Tigris</a> object storage to store user files. Mainly because it is integrated into the fly.io, so I decided to use it. Its free tier is more than enough for me. Auth</h2> Initially, I implemented the authentication manually. It was a simple password-based authentication with a primitive session-based authorization implementation. However, I then decided to throw it out</a>. I pondered on it and decided that I do not want to maintain a ton of code dedicated to auth. I wanted to delegate it to some external service. I explored the most popular solutions and chose Cloudflare Zero Trust Access</a>. It works very simply (as I wanted). Cloudflare works as a proxy and intercepts requests to my web service. If the request does not have an authentication token inside, then Cloudflare does the auth. I configured the allowed emails list, and Cloudflare sends a code to the entered email during sign-in. Cloudflare injects a token into the request after successful sign-in and sends the request to my web server. sequenceDiagram participant Client as Client participant Cloudflare as Cloudflare participant Server as Web-Service Client->>Cloudflare: HTTP Request Cloudflare->>Client: Redirect to Zero Trust login page Client->>Cloudflare: Sign in Cloudflare->>Client: Redirect back with session cookie (CF_Authorization cookie) Client->>Cloudflare: HTTP Request (with CF_Authorization cookie) Cloudflare->>Server: Proxies request with cf-access-jwt-assertion header Server-->>Cloudflare: Response Cloudflare-->>Client: Web server response </div> The only thing I need to do on my web server's side is to validate Cloudflare's JWT token. The validation algorithm is pretty straightforward: Extract the cf-access-jwt-assertion</code> header.</li> Get Cloudflare's signing keys.</li> Validate JWT using any common library.</li> </ol> You can read the full code here: github/TheBestTvarynka/Dataans/6c898a01/crates/web-server/src/routes/mod.rs</a>. There is an important moment: By default, Access rotates the signing key every 6 weeks. This means you will need to programmatically or manually update your keys as they rotate. (src</a>) </blockquote> Therefore, if you decide to hardcode signing keys, ensure that you rotate them regularly. I decided to follow another path: the web server requests the signing keys every time it needs to verify the JWT token. Yes, it is not the best option, but it is enough for me (Worse Is Better</a>). Great. Simple, secure, easy to set up 🚀. I asked ChatGPT for the step-by-step guide, which worked from the first try. I tried to access the protected route from the browser, and it worked perfectly. However, there is a small, tiny problem: I am not writing a website, I am writing a desktop app using Tauri. I cannot rely on cookies themselves because I want to make requests from the app backend using the request</code></a> HTTP client. Therefore, I need to extract the CF_Authorization</code> token somehow and save it somewhere in the app backend. Additionally, the app needs to ask the user password and passphrase. I was thinking for some time about the easiest way to implement. I came up with the idea of sending the cookie token, password, and passphrase in one Tauri command to the app backend. When the user passes the Cloudflare, the web server responds with the authorize.html</code></a> page. It is a simple form with an embedded script. When the user submits the form, the script extracts the CF_Authorization</code> token, reads the typed password and passphrase, and sends all this data to the app backend: async function extractAndSendToken() { const cookie = document.cookie .split('; ') .find(row => row.startsWith('CF_Authorization=')); if (!cookie) { console.error("CF_Authorization cookie not found."); return; } const token = cookie.split('=')[1]; const password = document.getElementById("password").value; const salt = document.getElementById("salt").value; let args = { token, url: window.location.origin }; if (password && password.length > 0) { args.password = password; } if (salt && salt.length > 0) { args.salt = salt; } try { await window.TAURI.core.invoke("plugin:dataans|sign_in", args); console.log("Token sent successfully:", token); } catch (e) { console.error("Failed to send token:", e); } }</code></pre> Fortunately, Tauri injects the window.TAURI</code> object into every spawned webview. Thus, even when the webview is created from an external URL</a> (not the direct app frontend</a>), it can still send Tauri commands. That's all. The rest is easy to implement. When the app backend receives the data, it generates an encryption key (from the password and passphrase), tests the auth token, and saves it... There is a corresponding piece of code</a>, if you want to see it. Data encryption</h2> The local state is unencrypted and saved in a SQLite database. But the app encrypts the data before sending it to the sync server. Dataans follows one rule regarding user data: we trust the user's computer and the user is responsible for keeping the local state safe, but we do not trust any remote servers and always encrypt data before sending it. We need to derive a key to encrypt the data. The app uses the PBKDF2</code></a> function and needs the user's password and passphrase to derive the key. // Pseudocode let password = sha256(password); let salt = sha256(passphrase); let key = pbkdf2_hmac(password, salt, 1_200_000 / iterations /);</code></pre> The derived key is stored in plaintext in the profile.json</code> file. There is no need to hide this key, because all data is local and unencrypted. As I wrote above, all data is encrypted before sending to the sync server. The app uses the AES256</a>-GCM</a> algorithm and appends an HMAC</a>-SHA256</a> checksum. // Pseudocode let nonce = random(); let cipher_text = aes_gcm.encrypt(key, nonce, plaintext); let checksum = hmac_sha256(key, nonce + plaintext); let result = nonce + plaintext + checksum;</code></pre> AES256-GCM</code> provides strong encryption, and HMAC-SHA256</code> provides data integrity. The same for decryption, but in reverse: // Pseudocode let (nonce, cipher) = result.split_at(NONCE_LEN); let (cipher_text, checksum) = cipher.split_at(cipher.len() - HMAC_LEN); let plaintext = aes_gcm.decrypt(key, nonce, cipher_text); let calculated_checksum = hmac_sha256(key, nonce + plaintext); if checksum != calculated_checksum { return Err("data is altered"); } Ok(plaintext)</code></pre> That is all for the encryption part. The overall encryption process is simple, secure, and, most importantly, boring. I like the fact that it took a bit more than 100 lines of code to implement all the needed cryptography functions in Rust: github/TheBestTvarynka/Dataans/6c898a01/dataans/src-tauri/src/dataans/crypto.rs</a>. Synchronization</h2> (If you do not want to read the whole section, then you can read the code: github/TheBestTvarynka/Dataans/6c898a01/dataans/src-tauri/src/dataans/sync/mod.rs</a>. It is well-documented. I spent around half of the year trying different approaches in synchronization. It was hard to choose a suitable sync algorithm. I had never implemented such features and had been overthinking it for months. First, I read the rsync algorithm</a> documentation and tried to apply it. Then, I tried to store the hash of every object in the database and compare these hashes to determine the state diff. At some point, I went even further and tried to apply Merkle trees</a> 🤪. Everything looked overcomplicated. But I did not give up and decided to make things even more complicated: CRDT</a>s. I was disappointed again because it wasn't what I needed. At some point, I paid attention to the Operation-Based CRDTs</a>. Synchronizing the operations log instead of the full app state looks simple. I elaborated this idea further and finally decided to use it for the synchronization. Every time a user does any action that alters the local state, the app logs this action. It is called a user operation, or an operation. When the user starts the sync, the app and the sync server synchronize the operation lists. Now the task is a bit simpler: there are two lists and the app needs to synchronize their content. The naive solution is to take local (app state) and remote (sync server state) operation lists and compare them operation by operation. But it can be slow, operations number can be large, and I still want to overengineer things. Thus, I introduced blocks and block hashes. A block is a collection of N</code> sequential operations sorted by timestamp. // `operations` are sorted by timestamp. let block_1 = operations[0..N]; let block_2 = operations[N..N 2]; // ... let block_n = operations[N (n - 1)..N n];</code></pre> I am sure you got the idea. All operations are divided into chunks by N</code>, which I call blocks. Nothing more, nothing less. A block hash is a hash of sequential hashes of block operations. let block_hash = hash(hash(block[0]), hash(block[1]), ..., hash(block[N - 1]));</code></pre> This is the core of my small, simple optimization. During the first synchronization step, the app requests the server's block hashes. And calculates the same block hashes on local operations. Blocks with the same hashes contain the same operations. Thus, the app can safely discard such blocks. Conflicts? The last write wins. Every object in the local database has an updated_at</code> timestamp. When the app detects conflict during synchronization, it compares two timestamps and last (latest) write wins. Let's summarize the synchronization algorithm. I prepared a scheme for you to make it easier to understand: sequenceDiagram participant App as Dataans participant Server as Sync Server Note over App,Server: Step 1: Request server's blocks and calculate local ones App->>Server: GET /data/blocks Server-->>App: [hash_1, hash_2, ...] Note over App: Step 2: Compare block hashes App->>App: Find distinct blocks by comparing hashes Note over App,Server: Step 3: Request server's operations from distinct blocks App->>Server: GET /data/operation?operations_to_skip=<> Server-->>App: [operation_1, operation_2, ...] Note over App: Step 4: Determine operations differences App->>App: Form operations_to_upload and operations_to_apply sets Note over App,Server: Step 5: Sync operations App->>Server: POST /data/operation (operations_to_upload) App->>App: Apply operations_to_apply to local state </div> Step 1. The app requests the server's block hashes and calculates local block hashes concurrently. The app will have two lists of hashes as the result. Step 2. The app compares block hashes from two lists one by one. It stops on the first pair of unequal hashes (or until one of the lists ends) and discards all hashes before this pair. The app will have two lists of block hashes that have different hashes. It does not need to compare hashes after the first unequal pair. All operations are sorted by timestamp before calculating block hashes. So, all the next consecutive blocks will also have different hashes. Step 3. At this point, the app knows how many blocks are equal on the local and remote sides. So, the app requests all operations after equal blocks and queries local operations (also after equal blocks). Step 4. The app compares local and remote operation lists (both of which are sorted by timestamp). The best thing is that the app does not need to do a Cartesian product over these two lists. The app compared operations one by one until the pair of operations with different ids (or until one of the lists ends). We can be sure that all operations after that point also have different ids. Because it is impossible otherwise. If the app is aware of some operation, then it is also aware of all operations before. The app cannot synchronize only later operations and skip earlier ones. The same is true for the server's side. // Step 4: Find the difference between local and remote operations. let mut operations_to_upload = Vec::new(); let mut operations_to_apply = Vec::new(); loop { match (local_operations.next(), remote_operations.next()) { (Some(local_operation), Some(remote_operation)) => { if local_operation.id != remote_operation.id { operations_to_upload.push(local_operation); operations_to_apply.push(remote_operation); for local_operation in local_operations { operations_to_upload.push(local_operation); } for remote_operation in remote_operations { operations_to_apply.push(remote_operation); } break; } } (Some(local_operation), None) => { operations_to_upload.push(local_operation); } (None, Some(remote_operation)) => { operations_to_apply.push(remote_operation); } (None, None) => break, } }</code></pre> The app will have operations_to_upload</code> and operations_to_apply</code> lists as the result. Step 5. The last step is simple. The app uploads the operations_to_upload</code> operations to the sync server and applies operations_to_apply</code> operations on the local database. If everything succeeds, we can be sure that local and remote states are the same ☺️. Files sync</h2> Aren't you forgetting something important? 🙃 Let me remind you. The Dataans supports attaching files to notes. Only file metadata is saved in the local SQLite database. The file contents are saved locally in the file system as files. It means that we also need to synchronize files alongside the SQLite DB state. A registered file is the file that is recorded in the local SQLite database. Every file has an is_uploaded</code> property. The app iterates over all registered files in the local database. Based on this property and the file existence in FS, the app does one of the following: is_uploaded</code> is true</code> and the file exists in FS - then the app does nothing. Everything is good.</li> is_uploaded</code> is true</code> and the file does not exist in FS - then the app downloads the file from the sync server.</li> is_uploaded</code> is false</code> and file exists in FS - then the app uploads the file and sets is_uploaded</code> to true</code>. The file is encrypted before uploading.</li> is_uploaded</code> is false</code> and the file does exist in FS - then the app logs a problem. Ideally, such a situation should not even exist. If it happens, then it means that someone/something has modified the app data directory (or a bug, but come on, it is definitely something else 🙃).</li> </ol> Files synchronization task runs concurrently with the main synchronization task. The synchronization process ends when both tasks finish. The app can discover new files during database sync. For example, when you add a new file on the first device, synchronize the data, and then try to sync the data on the second device. When this happens, the main synchronization task notifies the files synchronization task that there is a new file to download. These two tasks communicate over the channel</a>. // https://github.com/TheBestTvarynka/Dataans/blob/6c898a01afc0942cb94b5dbc822349d8afa924ee/dataans/src-tauri/src/dataans/sync/mod.rs#L133-L138 let (sender, receiver) = channel::<FileId>(CHANNEL_BUFFER_SIZE); let main_sync_fut = synchronizer.synchronize(emitter, sender); let file_sync_fut = synchronizer.synchronize_files(emitter, receiver); let (main_result, file_result) = futures::join!(main_sync_fut, file_sync_fut);</code></pre>What is next?</h1> I am happy I managed to implement it. I was a bit frustrated about it because the path to the final solution was very rough. Such an intuitive and simple feature turned out to be quite complex and hard to implement. Currently, I have implemented most of the major app backend features I needed. I am continuing to develop the Dataans. But I will focus more on the app frontend features. I think the next major releases will contain a lot of UI and UX improvements. I have a long list of amazing features ✨. If you are interested in contributing to or using the Dataans, feel free to ask any questions ☺️. References</h1> GitHub/TheBestTvarynka/Dataans/releases/v.0.3.0</a>.</li> GitHub/TheBestTvarynka/Dataans</a>.</li> Cloudflare Zero Trust Access</a>.</li> Neon</a>.</li> Fly.io</a>.</li> Tigris</a>.</li> Operation-based_CRDTs</a>.</li> github/TheBestTvarynka/Dataans/6c898a01/doc/sync_server.md</a>.</li> </ol> Safety Comments Matter 2025-05-31T00:00:00+00:00 TL;DR: This article is highly inspired by this GitHub comment</a>. Safety Comments Matter</h2> SAFETY</code> comments are always welcome when dealing with unsafe</code> code in Rust. The clippy even has a lint to enforce writing them: rust-lang.github.io/rust-clippy/#undocumented_unsafe_blocks</a>: #![warn(clippy::undocumented_unsafe_blocks)]</code></pre> You can find such comments in many crates and std. Example</a>: // SAFETY: `NonNull` is `transparent` over a `const T`, and `const T` // and `mut T` have the same layout, so transitively we can transmute // our `NonNull` to a `mut T` directly. unsafe { mem::transmute::<Self, mut T>(self) }</code></pre> Such comments help us to understand safety preconditions and invariants of the unsafe operation, and how they are upheld. Devs usually tend to treat them as one another boring thing in the development process. I kinda feel the same, but the benefits of properly written unsafe</code> blocks are almost immeasurable. But what is "a properly written unsafe</code> block"? A properly written unsafe</code> block has the following characteristics: Has a single unsafe operation.</li> Documented using SAFETY:</code> comment. In turn, to write a proper SAFETY:</code> comment, you must follow the following requirements (src</a> of the quote below):</li> </ul> Explain why the code is "sound", detailing the invariants and preconditions that are maintained.</li> Avoid assumptions without justifications. Do not use phrases like "we assume this is safe because…" or "this should be safe".</li> Provide concrete evidence or reasoning that justifies the safety of the operation.</li> </ul> </li> Explicitly reference type invariants and prior checks when safety relies on it.// SAFETY: index is guaranteed to be within bounds due to the prior check. unsafe { array.get_unchecked(index) };</code></pre></li> Address each safety requirement individually, using a bullet list.// SAFETY: // - ptr is non-null and properly aligned. // - The memory region ptr points to is valid for reads of len elements. unsafe { std::slice::from_raw_parts(ptr, len) };</code></pre></li> Document FFI boundaries clearly, but if the only safety requirements are standard (e.g.: non-dangling pointers), for consistency, use the following concise comment verbatim:// SAFETY: FFI call with no outstanding preconditions. unsafe { straightforward_ffi_function(ptr) };</code></pre></li> Do not document how the resulting value is safely used later. Really focus only on why the current unsafe operation is sound.</li> Subsequent operations should be documented as appropriate in the code.</li> </ul> </li> It’s welcomed to document how and why the function is used, but this should not be part of the SAFETY:</code> comment. Write a separate paragraph out of the safety comment, typically above the safety comment.// Passing None to GetModuleHandleW requests the handle to the // current process main executable module // // SAFETY: FFI call with no outstanding preconditions. let h_module: HMODULE = unsafe { GetModuleHandleW(None)? };</code></pre></li> </ul> </li> To keep the comment concise, do not use phrases like "this is safe because…". Go straight to the point (e.g.: "pointer is guaranteed not null due to the prior check").</li> For readability and clarity, keep the unsafe</code> block as small as possible, and untangled from other safe code. If needed, use intermediate variables.</li> </ul> // SAFETY: … let intermediate_variable = unsafe { transform_credentials_handle(credentials_handle) }; match intermediate_variable { … }</code></pre> If a precondition must be upheld by the caller, mark the function as unsafe and document it with a # Safety</code> section listing the invariants.</li> </ul> /// Returns the length in characters of the C-string. /// /// # Safety /// /// - `s` must be a valid, null-terminated C-string. unsafe fn strlen(s: const u8) -> usize { … }</code></pre></blockquote> 😮‍💨. I hope you didn't get tired at this point. It is true that writing a good SAFETY:</code> comment can be hard and exhausting. But it is worth it. For me, the most important point in SAFETY:</code> comments is that writing them requires thinking about preconditions and invariants. Devs seldom think about invariants and what can go wrong. When explaining why the unsafe</code> operation is safe, you check the corresponding unsafe function preconditions and how to uphold them. You can use AI to help you write SAFETY:</code> comments, but remember that it is your responsibility to check that the generated ones list all needed preconditions and invariants. Do not neglect SAFETY:</code> comments. Example</h3> As Linus Torvalds said</a>, Talk is cheap. Show me the code. </blockquote> So, let's show some code. Suppose we have the following unsafe</code> code: unsafe fn do_some_job(context: PCtxtHandle, buf: mut u8, len: mut u32, attrs: mut u32) -> u32 { if buf.is_null() || len.is_null() || attrs.is_null() { return 1; } let out_data = / do the job /; let out_attrs = / Attributes::... /; unsafe { from_raw_parts_mut(buf, out_data.len()).copy_from_slice(&out_data); len = out_data.len().try_into().unwrap(); attrs = out_attrs.bits(); } 0 }</code></pre> The function above is pretty simple. It does some job and saves the resulting buffers and attributes into the out parameters. Now let's make it better. I know we can. /// Does some job. /// /// # SAFETY: /// /// - The `context` pointer must be a valid non-null pointer to the application context and obtained using the [init_context] function. /// - The `len` and `attrs` pointers must be non-null. /// - The `buf` pointer must be non-null. The entire memory range behind it must be contained within a single allocated object, /// be properly initialized and aligned. unsafe fn do_some_job(context: PCtxtHandle, buf: mut u8, len: mut u32, attrs: mut u32) -> u32 { if buf.is_null() || len.is_null() || attrs.is_null() { return 1; } let out_data = /* do the job /; let out_attrs = / Attributes::... /; // SAFETY: // - The `buf` pointer is not null due to the prior check. // - We create only one slice at a time, so slice do not alias any other references. // - The caller must ensure that the memory is properly initialized and aligned. let buf = unsafe { from_raw_parts_mut(buf, out_data.len()) }; buf.copy_from_slice(&out_data); let out_len = out_data.len().try_into().unwrap(); // SAFETY: // The `len` pointer is not null due to the prior check. unsafe { len = out_len; } let out_attrs = out_attrs.bits(); // SAFETY: // The `attrs` pointer is not null due to the prior check. unsafe { attrs = out_attrs; } 0 }</code></pre> Much better, right? Let's recall what has changed: Added function doc comment with # Safety</code> requirements.</li> Split unsafe operations into many unsafe</code> blocks containing exactly one unsafe operation.</li> Every unsafe</code> block now has the SAFETY:</code> comment.</li> Moved safe operation out of the unsafe block (I'm talking about the .copy_from_slice</code> method call).</li> </ul> Now the caller knows that it is not enough to just allocate the memory. The memory needs to be initialized. Now the caller knows that len</code> and attrs</code> pointers cannot be NULL. And more... It sounds obvious, but the unsafe code becomes very dangerous in the blink of an eye. Put A Finger Down</h2> Let's play the Put A Finger Down</a> game. (This section is optional and written mostly for fun). The rule is simple: you put one finger down for every true statement about your unsafe Rust code. If you put down zero fingers, my congratulations 🎊. You are probably aware of what you are doing, and you can sleep peacefully. If you put down all your fingers, then, please, reconsider your life choices 🙂 ☑️ Rust 2018 edition or older. ☑️ unsafe</code> blocks without SAFETY</code> comments. ☑️ Many unsafe operations per one unsafe block. ☑️ Your unsafe code has never been run using Miri</a>. ☑️ Raw pointer <-> int</code> cast is a usual thing. The reader understands that these statements mean something bad for your unsafe code. Let's discuss each of them. I want to make sure that we are on the same page. Rust 2018 edition or older</h3> Why is the Rust edition important in our case? </blockquote> (I already described it here</a>.) The 2024 edition brought many important features related to unsafe</code> code and FFI. unsafe_op_in_unsafe_fn</code> warning</a>.</li> </ul> unsafe fn set_id(id: u8, dest: mut u8) { // warning[E0133]: dereference of raw pointer is unsafe and requires unsafe block dest = id; // Solution: unsafe { dest = id; } }</code></pre> I like it a lot. Because in huge unsafe</code> functions, it's not clear which operations are unsafe and which are not. Unsafe attributes</a>.</li> </ul> // Before Rust 2024 the following code was accepted: #[no_mangle] pub fn tbt() {} // In Rust 2024 you need to add the unsafe attribute: #[unsafe(no_mangle)] pub fn tbt() {}</code></pre> Starting with the 2024 Edition, it is now required to mark these attributes as unsafe. This one applies to export_name</a>, link_section</a>, and no_mangle</a>. It is crucial because previously the app could crash even if it contained only safe code: fn main() { println!("Hello, world!"); } #[export_name = "malloc"] fn foo() -> usize { 1 }</code></pre> Disallow references to static mut</a>.</li> </ul> static mut X: i32 = 23; unsafe { // ERROR: shared reference to mutable static let y = &X; }</code></pre> Merely taking such a reference in violation of Rust's mutability XOR aliasing requirement has always been instantaneous undefined behavior, even if the reference is never read from or written to. </blockquote> Unsafe extern blocks</a>.</li> </ul> Adding the unsafe keyword helps to emphasize that it is the responsibility of the author of the extern block to ensure that the signatures are correct. </blockquote> It produces a warning in Rust 2021, but in Rust 2024 it results in error. Now you understand why the 2024 edition is preferred in unsafe code 😃. unsafe</code> blocks without SAFETY</code> comments</h3> The whole first part</a> of the article is about it. You should already understand the importance of such comments in the code. Many unsafe operations per one unsafe block</h3> It is partially covered by the 2024 edition, but still can violated: // Compiles without any warnings. unsafe fn tbt(buf: mut mut u8, len: mut u32) { unsafe { buf = null_mut(); len = 0; } }</code></pre> It is not recommended (by me and some other dudes 😉) to have multiple unsafe operations in one unsafe</code> block. Because it becomes much harder to keep the code safe and follow all safety preconditions. Every unsafe operation should have its own unsafe</code> block and SAFETY</code> comment above it (you can see an example in the first part</a> of the article). Miri</h3> I already described all pros and cons of Miri here: https://tbt.qkation.com/posts/miri/</a>. Please, read this article so I don't need to repeat myself. TL;DR: if you have many unsafe operations, then consider using Miri. Miri is awesome and will help you to prevent a lot of bugs. Pointer <-> integer cast</h3> Oh, this one deserves a separate article, but I will try to fit it in a few sentences. I'm still convinced that the Rust documentation is the best place to learn about unsafe. To your attention, quotes from the std::ptr</code></a> module level documentation: Pointers are not simply an “integer” or “address”. ...pointers need to somehow be more than just their addresses: they must have provenance. Provenance can affect whether a program has undefined behavior: It is undefined behavior to access memory through a pointer that does not have provenance over that memory.</li> It is undefined behavior to offset</code></a> a pointer across a memory range that is not contained in the allocated object it is derived from, or to offset_from</code></a> two pointers not derived from the same allocated object. Provenance is used to say what exactly “derived from” even means: the lineage of a pointer is traced back to the Original Pointer it descends from, and that identifies the relevant allocated object. In particular, it’s always UB to offset a pointer derived from something that is now deallocated, except if the offset is 0.</li> </ul> </blockquote> and</a>: From this discussion, it becomes very clear that a usize</code> cannot accurately represent a pointer, and converting from a pointer to a usize</code> is generally an operation which only extracts the address. Converting this address back into pointer requires somehow answering the question: which provenance should the resulting pointer have? Rust provides two ways of dealing with this situation: Strict Provenance and Exposed Provenance. </blockquote> There is a lot more to say. But I think you can read the Rust doc by yourself. My point is that you should never cast a pointer to an integer and vice versa. Or if you even dare to do it, then at least use an appropriate API: expose_provenance</code></a> and with_exposed_provenance</code></a>. Conclusion</h2> Pointers are not simply an "integer" or "address" 🙂. Implementing Kerberos RPC encryption over SSPI 2025-04-05T00:00:00+00:00 Intro</h1> Today's world has many unknown and magical things. Proprietary protocols and libraries are among them 🤪. In this blog post, I will explain how RPC encryption works under the hood and how to implement it over the SSPI interface. Goals</h2> Provide a detailed explanation of how RPC PDUs are encrypted.</li> Explain how it is related to SSPI.</li> Implement RPC PDUs encryption/decryption and test it against real RPC traffic 🤭.</li> </ul> Non-goals</h2> I'm not explaining RPC use cases or how to set it up.</li> Do not expect RPC internals explanations.</li> </ul> This blog post is only about encrypting and decrypting RPC PDUs. Getting started</h1> I assume the reader has enough knowledge and experience with RPC and SSPI to read this article. If not, I highly recommend reading the following articles (they should give enough context to understand what I am going to do): RPC Encryption - An Exercise in Frustration</a>.</li> SSPI introduction</a>.</li> </ul> Microsoft frequently uses RPC for local and remote calls. Of course, many remote RPC calls are encrypted, and the caller needs to pass the authentication to communicate with the server. Let's take, for example, [MS-GKDI]: Group Key Distribution Protocol</a>, ...which enables clients to obtain cryptographic keys associated with Active Directory security principals. </blockquote> It specifies only one GetKey</code></a> RPC method. I hope it's obvious that the key is encrypted and cannot be sent over the network as plaintext. (If you want to open and analyze this packet with me, then here is a Wireshark recording: rpc.pcapng</a>.) Captured RPC communication shows us all authentication steps and encrypted GetKey</code> RPC method call. And we will decrypt it at the end of this article 😎. RPC PDU structure</h1> (Alternatively, you can read the RPC Payload section from the RPC Encryption - An Exercise in Frustration</a> article). RPC PDU is usually split into three parts (https://pubs.opengroup.org/onlinepubs/9629399/chap12.htm</a>): name</th> purpose</th></tr></thead> header</td> contains protocol control information</td></tr> body</td> the body of a request or response PDU contains data representing the input or output parameters for an operation</td></tr> security trailer</td> contains data specific to an authentication protocol. For example, an authentication protocol may ensure the integrity of a packet via inclusion of an encrypted checksum in the authentication verifier</td></tr> </tbody></table> But for now, we need to dig a bit deeply. PDU body consists of its header and data. In turn, the PDU security trailer also has its header and auth value. Look at the screenshot below: I think you've got the idea. In our case, the PDU body data and security trailer auth value are encrypted. RPC encryption over SSPI</h1> Why SSPI? There are several reasons why: This is how it is originally done in Windows.</li> Some aspects of the encryption process make sense only with SSPI in mind.</li> RPC and SSPI are interconnected in many places, so why not explain the SSPI part?</li> </ol> RPC PDUs are encrypted by calling the SSPI::EncryptMessage</a> function (and, correspondingly, decryption is done by calling the SSPI::DecryptMessage</a> function). Let's see its interface. // https://learn.microsoft.com/en-us/windows/win32/api/sspi/nf-sspi-encryptmessage SECURITY_STATUS SEC_ENTRY EncryptMessage( [in] PCtxtHandle phContext, [in] unsigned long fQOP, [in, out] PSecBufferDesc pMessage, [in] unsigned long MessageSeqNo ); // https://learn.microsoft.com/en-us/windows/win32/api/sspi/nf-sspi-decryptmessage SECURITY_STATUS SEC_ENTRY DecryptMessage( [in] PCtxtHandle phContext, [in, out] PSecBufferDesc pMessage, [in] unsigned long MessageSeqNo, [out] unsigned long pfQOP ); // https://learn.microsoft.com/en-us/windows/win32/api/sspi/ns-sspi-secbufferdesc typedef struct _SecBufferDesc { unsigned long ulVersion; unsigned long cBuffers; PSecBuffer pBuffers; } SecBufferDesc, PSecBufferDesc; // https://learn.microsoft.com/en-us/windows/win32/api/sspi/ns-sspi-secbuffer typedef struct _SecBuffer { unsigned long cbBuffer; unsigned long BufferType; char pvBuffer; } SecBuffer, PSecBuffer;</code></pre> There are a lot of security buffer types and two security buffer flags defined in SSPI (SecBuffer structure (sspi.h</code>): BufferType</code></a>). But for now, we need only a few of them: buffer type</th> meaning/purpose</th></tr></thead> SECBUFFER_TOKEN</code></td> in the general case, it contains the security token portion of the message. In our case, it contains a signature generated by the security package.</td></tr> SECBUFFER_DATA</code></td> contains input data to be processed by the security package. In our case, it's just data we want to encrypt or decrypt.</td></tr> </tbody></table> Also, we have two possible buffer flags: buffer flag</th> meaning</th></tr></thead> SECBUFFER_READONLY</code></td> the data in this buffer is read-only and is never can be overwritten by the security package. Usually, this flag is used to inform the security package about something.</td></tr> SECBUFFER_READONLY_WITH_CHECKSUM</code></td> The data in this buffer is included in the checksum calculation but not in the encryption/decryption process.</td></tr> </tbody></table> As you can see, the message we want to encrypt is an array of buffers. The whole PDU is split into SSPI security buffers and passed to the SSPI::EncryptMessage</code> function. This process is described in the "Message Protection" section of the RPC Encryption - An Exercise in Frustration</a> article. In short, it happens as follows: </th> security buffer type + flags</th> security buffer value</th> made-up name</th></tr></thead> 1</td> SECBUFFER_DATA</code> + SECBUFFER_READONLY_WITH_CHECKSUM</code></td> contains PDU header + PDU body header</td> Sign1</td></tr> 2</td> SECBUFFER_DATA</code></td> contains PDU body data to be encrypted (in-place)</td> Enc</td></tr> 3</td> SECBUFFER_DATA</code> + SECBUFFER_READONLY_WITH_CHECKSUM</code></td> contains PDU security trailer header</td> Sign2</td></tr> 4</td> SECBUFFER_TOKEN</code></td> will contain PDU security trailer auth value</td> Token</td></tr> </tbody></table> We will refer to these buffers a lot, so I added made-up names to each of them. This can be illustrated in the previous RPC request that we used as the example: All encryption is always in-place. The security package overwrites the original data with encrypted data. This also means that the caller must allocate the last security buffer (4th which has the SECBUFFER_TOKEN</code> type), and the security package will overwrite it. The caller can get the token buffer length by calling the QueryContextAttributes</code></a> function with the SECPKG_ATTR_SIZES</code> parameter. The cbSecurityTrailer</code> field of the SecPkgContext_Sizes</code></a> structure contains a desired value. SSPI::EncryptMessage</h1> Before implementing it, I want to clarify some general function behavior. As I wrote above, it encrypts the input message in-place. Some security packages, besides encryption, also calculate checksum over the provided data. Even more, some security packages can only calculate a checksum over the provided data. Here is a citation from the MSDN</a>: Some packages do not have messages to be encrypted or decrypted but rather provide an integrity hash that can be checked. </blockquote> In our case, both encryption and integrity hash (hmac) calculations happen. We have 4 buffers as an input message. The general encryption process looks like this: // pseudo code // Sign1: message[0] is PDU header + PDU body header. SECBUFFER_DATA + SECBUFFER_READONLY_WITH_CHECKSUM. // Enc: message[1] is PDU body data. we need to encrypt it. SECBUFFER_DATA. // Sign2: message[2] is PDU security trailer header. SECBUFFER_DATA + SECBUFFER_READONLY_WITH_CHECKSUM. // Token: message[3] is PDU security trailer auth value. Currently, this buffer is blank (memory is allocated). SECBUFFER_TOKEN. message[1] = encrypt(message[1]); message[3] = checksum(message[0] | message[1] | message[2]);</code></pre>Kerberos::EncryptMessage</h1> Now that we have enough knowledge about the encryption process, we can talk about concrete encryption algorithms. The client can authenticate using one of the available protocols (security packages) like NTLM or Kerberos. The most secure these days is, of course, Kerberos. Our task is to understand how it works and implement it. First, I should mention that the Kerberos::EncryptMessage</code> function behaves as the GSS_WrapEx</code></a> function. It encrypts the data, constructs a Wrap Token, and writes it in the input message. The wrap token construction and encryption process are defined in RPC 4121</a>. Let's recall the token structure</a>: Octet no Name Description -------------------------------------------------------------- 0..1 TOK_ID Identification field. Tokens emitted by GSS_Wrap() contain the hex value 05 04 expressed in big-endian order in this field. 2 Flags Attributes field, as described in section 4.2.2. 3 Filler Contains the hex value FF. 4..5 EC Contains the "extra count" field, in big- endian order as described in section 4.2.3. 6..7 RRC Contains the "right rotation count" in big- endian order, as described in section 4.2.5. 8..15 SND_SEQ Sequence number field in clear text, expressed in big-endian order. 16..last Data Encrypted data for Wrap tokens with confidentiality, or plaintext data followed by the checksum for Wrap tokens without confidentiality, as described in section 4.2.4.</code></pre> This can look familiar to you because we already saw this structure in the Wireshark: If we want to decrypt the RPC request, then we need to know the meaning of each field: name</th> meaning</th></tr></thead> TOK_ID</code></td> 0x0504</code></td></tr> Flags</code></td> They depend on the authentication process.</td></tr> Filler</code></td> 0xff</code></td></tr> EC</code></td> RFC 4121: EC Field</a>. It represents how many filler bytes we must insert after the plaintext data before encryption. In our case, it is equal to 16 (The sender should set extra count (EC) to 1 block - 16 bytes.</a>)</td></tr> RRC</code></td> In our case, it is equal to 28. The RRC field is 28 if encryption is requested.</a></td></tr> SND_SEQ</code></td> This value is set by the Kerberos implementation.</td></tr> </tbody></table> It might not make much sense to you so far, but soon enough, you will have one picture. RFC 4121 also defines the encryption process</a>: The resulting Wrap token is {"header" | encrypt(plaintext-data | filler | "header")}</code>, where encrypt()</code> is the encryption operation. </blockquote> It looks simple, and I think you got the idea. But there is a tricky moment. The negotiated Kerberos encryption algorithm is AES256-CTS-HMAC-SHA1-96</code>. This algorithm has two specific features: It appends a checksum (hmac) over the plaintext to the ciphertext.</li> It prepends one block of random bytes (named confounder) to the plaintext data before encryption (it is a non-deterministic encryption).</li> </ul> So, the actual encryption scheme is a bit more complex and looks something like this: // pseudocode // I hope you didn't forget the `message` array from the previous pseudo code block. let ec = 16; let filler = ec 0x00; let confounder = rand::<[u8; 16]>(); let data_to_encrypt = message[1] | filler | wrap_token_header; let ciphertext = encrypt(confounder | data_to_encrypt); let data_to_hmac = message[0] | message[1] | message[2] | filler | wrap_token_header; let checksum = hmac(confounder | data_to_hmac); let wrap_token = wrap_token_header | ciphertext | checksum;</code></pre> Cool. Can we finally write the resulting Wrap Token to input buffers? </blockquote> Not yet 🤭. The RFC also defined the right rotation operation after the encryption. RFC 4121: RRC Field</a>: The "RRC" (Right Rotation Count) field in Wrap tokens is added to allow the data to be encrypted in-place by existing SSPI applications. Excluding the first 16 octets of the token header, the resulting Wrap token in the previous section is rotated to the right by "RRC" octets. The net result is that "RRC" octets of trailing octets are moved toward the header. Consider the following as an example of this rotation operation: Assume that the RRC value is 3 and the token before the rotation is {"header" | aa | bb | cc | dd | ee | ff | gg | hh}</code>. The token after rotation would be {"header" | ff | gg | hh | aa | bb | cc | dd | ee }</code>. </blockquote> It means, that we need to do the right-rotation operation on ciphertext | checksum</code>. But pay attention here: we must rotate it by RRC + RC bytes</a>. The final Wrap Token looks like this: // pseudocode let rrc = 28; let ec = 16; let wrap_token = wrap_token_header | rotate_right(ciphertext | checksum, rrc + ec);</code></pre> The very last step is to split the resulting Wrap Token into buffers and write them into corresponding input buffers (input message). Let's recall the task. We have 4 input buffers: Sign1: Data</code> buffer with SECBUFFER_READONLY_WITH_CHECKSUM</code>.</li> Enc: Data</code> buffer.</li> Sign2: Data</code> buffer with SECBUFFER_READONLY_WITH_CHECKSUM</code>.</li> Token: Token</code> buffer.</li> </ul> And now we need to write the Wrap Token buffer into the second and fourth buffers. To do it correctly, we write first cbSecurityTrailer</code> bytes of the Wrap Token into the Token buffer, and the rest of the Wrap Token should be written into the Enc buffer. // pseudocode let cb_security_trailer = 76; let (token, data) = wrap_token.split_at(cb_security_trailer); // Write into the Enc buffer message[1].copy_from_slice(data); // Write into the Token buffer message[3].copy_from_slice(token);</code></pre> Some curious readers may ask me: Why can't we fill the Enc buffer first and then just write the rest of the Wrap Token into the Token buffer? Most likely because the RPC server expects the same buffers, as you just explained. But the question is why it works this way and not another. </blockquote> Some things are done on purpose, while others are simply historical heritage (legacy). If you write down all these data manipulations, you can notice that, in the end, the Enc buffer ciphertext matches the original data. The checksum, encrypted confounder, and Wrap Token header will be in the Token buffer. Look at the scheme below: I tried to show the whole encryption process in the diagram above. As you can see, the encrypted Enc buffer data is written directly into the Enc buffer. But the Wrap Token Header, encrypted confounder, filler, etc., are written exclusively in the Token buffer. Assumptions</h2> Now we know all the details of the encryption process. Of course, we can't know all the reasons behind Microsoft's decisions regarding their protocols and implementations. But for now, I can make a few assumptions: Why do we need to rotate ciphertext | checksum</code>?. To make the encrypted Enc buffer data match the unencrypted one. This way, the encryption will be in-place.</li> Why is the EC value equal to 16?. Just to extend the ciphertext (to make it longer on purpose). Why do we need to make it longer? Kerberos AES256-CTS-HMAC-SHA1-96 encryption algorithm uses AES256 in CTS mode</a>. In short, it changes the last two blocks of ciphertext, so the padding is unnecessary. To ensure that the encrypted Enc buffer data is not affected during CTS (cipher text stealing) (otherwise, it would be impossible for the encrypted Enc buffer data to match the unencrypted one), we need to be sure that the last two blocks of the ciphertext do not contain the Enc buffer data. Thus, we have two additional blocks: the first block is the filler bytes (0x00 * 16</code>) and the second block is the Wrap Token header (16 bytes long).</li> Why is the cbSecurityTrailer</code> value equal to 76? wrap token header len + confounder len + filler len + wrap token header (encrypted) len + checksum len = 16 + 16 + 16 + 16 + 12 = 76.</li> </ul> Alternatively, you can read Microsoft's example of the message encryption. [MS-KILE]: GSS_WrapEx</code> with AES128-CTS-HMAC-SHA1-96</code></a>: The picture above shows the same process but with two buffers to encrypt instead of one. clearhdr</code> - Wrap Token header. padding</code> = ec * 0x00</code>. Phew 😮‍💨 I hope you are not tired because we are going to implement it 🤗. Code: Encryptor</h1> Note 1. The full code is available here: RPC decryptor github/TheBestTvarynka/trash-code/rpc-decryptor</a>. Note 2. The code in this article is not production-ready! The only purpose of the code below is to show the algorithm's correctness. Let's start with simple things. Here is a small SecBuffer</code></a> implementation. We don't need anything more complex. pub const DATA: u32 = 1; pub const TOKEN: u32 = 2; pub const READONLY_WITH_CHECKSUM_FLAG: u32 = 0x10000000; pub struct SecBuffer<'data> { pub buffer_type: u32, pub data: &'data mut [u8], } impl<'data> SecBuffer<'data> { pub fn new(buffer_type: u32, data: &'data mut [u8]) -> Self { Self { buffer_type, data } } }</code></pre> Now we need a Wrap Token implementation. I don't want to overengineer it. I wrote a simple Wrap Token header encoding/decoding. Again, we don't need anything more complex. pub struct WrapTokenHeader { pub flags: u8, pub ec: u16, pub rrc: u16, pub send_seq: u64, } impl WrapTokenHeader { pub fn encoded(&self) -> [u8; 16] { let mut header_data = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]; header_data[0..2].copy_from_slice(&[0x05, 0x04]); header_data[2] = self.flags; header_data[3] = 0xff; header_data[4..6].copy_from_slice(&self.ec.to_be_bytes()); header_data[6..8].copy_from_slice(&self.rrc.to_be_bytes()); header_data[8..].copy_from_slice(&self.send_seq.to_be_bytes()); header_data } pub fn from_bytes(mut src: impl Read) -> Self { if src.read_u16::<BigEndian>().unwrap() != 0x0504 { panic!("Invalid Wrap Token ID"); } let flags = src.read_u8().unwrap(); let filler = src.read_u8().unwrap(); if filler != 0xff { panic!("Invalid filler"); } let ec = src.read_u16::<BigEndian>().unwrap(); let rrc = src.read_u16::<BigEndian>().unwrap(); let send_seq = src.read_u64::<BigEndian>().unwrap(); Self { flags, ec, rrc, send_seq, } } }</code></pre> Great! Now the fun part: encryption implementation. First of all, we should define the constants we are going to use. // "Extra count" const EC: u16 = 16; // "Right Rotation Count" const RRC: u16 = 28; // https://www.rfc-editor.org/rfc/rfc4121.html#section-2 const CLIENT_ENCRYPTION_KEY_USAGE: i32 = 24; const CLIENT_DECRYPTION_KEY_USAGE: i32 = 22; const SERVER_ENCRYPTION_KEY_USAGE: i32 = CLIENT_DECRYPTION_KEY_USAGE; const SERVER_DECRYPTION_KEY_USAGE: i32 = CLIENT_ENCRYPTION_KEY_USAGE; const CB_SECURITY_TRAILER: usize = 16 /* wrap token header len / + 16 / confounder len / + EC as usize + 16 / wrap token header (encrypted) len / + 12 / checksum (hmac) len /;</code></pre> You may be surprised by key usage numbers. Don't worry. 7.5.1. Key Usage Numbers</a> The encryption and checksum specifications in RFC3961</a> require as input a "key usage number", to alter the encryption key used in any specific message to make certain types of cryptographic attack more difficult. </blockquote> In other words, a key usage number is a public known value used to derive the encryption key from a base key (session key). You should not care about it. We always use key usage numbers specified in the RFC/specification. Now we can start implementing RPC encryption. Finally! We start with the Wrap Token and filler generation. fn encrypt(key: &[u8], key_usage: i32, send_seq: u64, message: &mut [SecBuffer<'_>]) { let mut wrap_token_header = WrapTokenHeader { flags: 0x06, ec: EC, rrc: 0, send_seq, }; let encoded_wrap_token_header = wrap_token_header.encoded(); let filler = vec![0; usize::from(EC)]; todo!() }</code></pre> Then, we encrypt the data. Let's do it in two steps: Collect the data to encrypt in one buffer: DATA sec buffer + filler + wrap token header.</li> Encrypt it with the correct key usage number. There are not many Kerberos encryption algorithm implementations. The only actively used and well-maintained one is picky-krb</code>: docs.rs/picky-krb/crypto/aes/</a>.</li> </ol> fn encrypt(key: &[u8], key_usage: i32, send_seq: u64, message: &mut [SecBuffer<'_>]) { /// ... // Find the Enc buffer let mut data_to_encrypt = message.iter().fold(Vec::new(), |mut acc, sec_buffer| { if sec_buffer.buffer_type == DATA { acc.extend_from_slice(sec_buffer.data); } acc }); // + Filler data_to_encrypt.extend_from_slice(&filler); // + Wrap token header data_to_encrypt.extend_from_slice(&encoded_wrap_token_header); let cipher = Aes256CtsHmacSha196::new(); let EncryptWithoutChecksum { mut encrypted, confounder, ki: _, } = cipher.encrypt_no_checksum(key, key_usage, &data_to_encrypt).unwrap(); todo!() }</code></pre> I hope you didn't forget about the confounder. In the code above, it is autogenerated and returned from the encrypt_no_checksum</code> function. We need it to correctly calculate HMAC. What is next? Right, checksum calculation. fn encrypt(key: &[u8], key_usage: i32, send_seq: u64, message: &mut [SecBuffer<'_>]) { /// ... let mut data_to_sign = message.iter().fold(confounder, |mut acc, sec_buffer| { if sec_buffer.buffer_type == DATA | READONLY_WITH_CHECKSUM_FLAG { acc.extend_from_slice(sec_buffer.data); } else if sec_buffer.buffer_type == DATA { acc.extend_from_slice(sec_buffer.data); } acc }); // + Filler data_to_sign.extend_from_slice(&filler); // + Wrap token header data_to_sign.extend_from_slice(&encoded_wrap_token_header); let checksum = cipher.encryption_checksum(&key, key_usage, &data_to_sign).unwrap(); encrypted.extend_from_slice(&checksum); todo!() }</code></pre> As you can see, we calculate checksum separately from the encryption because we may have READONLY_WITH_CHECKSUM_FLAG</code> buffers which we don't need to encrypt. 😮‍💨 What's left? Right rotation, final Wrap Token construction, copying the data to the input message</code>. fn encrypt(key: &[u8], key_usage: i32, send_seq: u64, message: &mut [SecBuffer<'_>]) { /// ... encrypted.rotate_right(usize::from(RRC + EC)); wrap_token_header.rrc = RRC; // final Wrap Token let mut raw_wrap_token = wrap_token_header.encoded().to_vec(); raw_wrap_token.extend_from_slice(&encrypted); let (token, data) = raw_wrap_token.split_at(CB_SECURITY_TRAILER); let token_buffer = message .iter_mut() .find(|sec_buffer| sec_buffer.buffer_type == TOKEN) .expect("TOKEN buffer not found"); token_buffer.data.copy_from_slice(token); let data_buffer = message .iter_mut() .find(|sec_buffer| sec_buffer.buffer_type == DATA) .expect("DATA buffer not found"); data_buffer.data.copy_from_slice(data); }</code></pre> That's all! We just implemented RPC Kerberos encryption! 😎 🥳 Code: Decryptor</h1> Note. The full code is available here: RPC decryptor github/TheBestTvarynka/trash-code/rpc-decryptor</a>. I didn't explain the decryption process in detail, but it is completely the reverse of encryption. Look at the scheme below (you may need to zoom in or open it in a new tab): As you can see, the decryption process is a reversed encryption process (obviously, makes sense). Time to start implementing it. We start with the Wrap Token construction and split it into the header and encrypted parts. fn decrypt(key: &[u8], key_usage: i32, message: &mut [SecBuffer<'_>]) { let mut wrap_token = message .iter_mut() .find(|sec_buffer| sec_buffer.buffer_type == TOKEN) .expect("TOKEN buffer not found") .data .to_vec(); message.iter().for_each(|sec_buffer| { if sec_buffer.buffer_type == DATA { wrap_token.extend_from_slice(sec_buffer.data); } }); let (wrap_token_header, encrypted) = wrap_token.split_at_mut(16 / Wrap Token header length /); let wrap_token_header = WrapTokenHeader::from_bytes(wrap_token_header as &[u8]); encrypted.rotate_left(usize::from(wrap_token_header.rrc + wrap_token_header.ec)); todo!() }</code></pre> Then, we do the left rotation and data decryption: fn decrypt(key: &[u8], key_usage: i32, message: &mut [SecBuffer<'_>]) { /// ... encrypted.rotate_left(usize::from(wrap_token_header.rrc + wrap_token_header.ec)); let cipher = Aes256CtsHmacSha196::new(); let DecryptWithoutChecksum { plaintext, confounder, checksum, ki: _, } = cipher.decrypt_no_checksum(&key, key_usage, encrypted).unwrap(); let plaintext_len = plaintext.len() - usize::from(wrap_token_header.ec + 16 / Wrap Token header length /); // plaintext - decrypted data. // data - filler + wrap token. let (plaintext, data) = plaintext.split_at(plaintext_len); todo!() }</code></pre> Nice! And, of course, we need to calculate the checksum and compare it with the provided one. fn decrypt(key: &[u8], key_usage: i32, message: &mut [SecBuffer<'_>]) { /// ... let mut decrypted = plaintext; let mut data_to_sign = message.iter().fold(confounder, |mut acc, sec_buffer| { if sec_buffer.buffer_type == DATA | READONLY_WITH_CHECKSUM_FLAG { acc.extend_from_slice(sec_buffer.data); } else if sec_buffer.buffer_type == DATA { acc.extend_from_slice(&decrypted[0..sec_buffer.data.len()]); decrypted = &decrypted[sec_buffer.data.len()..]; } acc }); // + Filler + Wrap token header data_to_sign.extend_from_slice(data); let calculated_checksum = cipher.encryption_checksum(&key, key_usage, &data_to_sign).unwrap(); if calculated_checksum != checksum { panic!("Checksum mismatch: message is altered"); } todo!() }</code></pre> 😮‍💨 And the last one: copying the plaintext to the input message</code>. fn decrypt(key: &[u8], key_usage: i32, message: &mut [SecBuffer<'_>]) { /// ... let mut decrypted = plaintext; message .iter_mut() .filter(|sec_buffer| sec_buffer.buffer_type == DATA) .for_each(|sec_buffer| { sec_buffer.data.copy_from_slice(&decrypted[0..sec_buffer.data.len()]); decrypted = &decrypted[sec_buffer.data.len()..]; }); }</code></pre> 🎊 🥳 That's all! Now we have encryptor and decryptor implemented. Code: testing</h1> Let's add a simple test to ensure that it works at least against itself (encrypt-decrypt test). I wrote small wrappers for the encryption/decryption functions to make testing easier: pub struct KerberosClient { key: Vec<u8>, send_seq: u64, } impl KerberosClient { pub const TOKEN_LEN: usize = CB_SECURITY_TRAILER; pub fn new(key: Vec<u8>, send_seq: u64) -> Self { Self { key, send_seq } } pub fn encrypt_message(&self, message: &mut [SecBuffer<'_>]) { encrypt(&self.key, CLIENT_ENCRYPTION_KEY_USAGE, self.send_seq, message); } pub fn decrypt_message(&self, message: &mut [SecBuffer<'_>]) { decrypt(&self.key, CLIENT_DECRYPTION_KEY_USAGE, message); } } pub struct KerberosServer { key: Vec<u8>, send_seq: u64, } impl KerberosServer { pub const TOKEN_LEN: usize = CB_SECURITY_TRAILER; pub fn new(key: Vec<u8>, send_seq: u64) -> Self { Self { key, send_seq } } pub fn encrypt_message(&self, message: &mut [SecBuffer<'_>]) { encrypt(&self.key, SERVER_ENCRYPTION_KEY_USAGE, self.send_seq, message); } pub fn decrypt_message(&self, message: &mut [SecBuffer<'_>]) { decrypt(&self.key, SERVER_DECRYPTION_KEY_USAGE, message); } }</code></pre> Pretty straightforward and understandable. And most importantly, we don't need more. Here is a simple test: #[test] fn encrypt_decrypt() { let tbt = b"TheBestTvarynka"; let session_key = [ 91, 11, 188, 227, 10, 91, 180, 246, 64, 129, 251, 200, 118, 82, 109, 65, 241, 177, 109, 32, 124, 39, 127, 171, 222, 132, 199, 199, 126, 110, 3, 166, ]; let kerberos_client = KerberosClient::new(session_key.to_vec(), 681238048); let kerberos_server = KerberosServer::new(session_key.to_vec(), 681238048); let mut data_buf = tbt.to_vec(); let mut token_buf = vec![0; KerberosClient::TOKEN_LEN]; let mut message = vec![ SecBuffer::new(DATA, &mut data_buf), SecBuffer::new(TOKEN, &mut token_buf), ]; kerberos_client.encrypt_message(&mut message); kerberos_server.decrypt_message(&mut message); assert_eq!(message[0].data, tbt); }</code></pre> Good results so far! But do you remember our main goal? We wanted to decrypt the captured encrypted RPC request. Let's do it! Now nothing can stop us 😉 😆. #[test] fn decrypt_rpc_request() { let session_key = vec![ 19, 28, 59, 181, 9, 202, 41, 22, 25, 122, 144, 217, 9, 87, 170, 209, 72, 223, 145, 41, 12, 252, 9, 229, 45, 218, 206, 161, 199, 216, 243, 53, ]; let kerberos_server = KerberosServer::new(session_key, 41895117); // PDU header + PDU body header let mut header = vec![ 5, 0, 0, 3, 16, 0, 0, 0, 60, 1, 76, 0, 1, 0, 0, 0, 208, 0, 0, 0, 0, 0, 0, 0, ]; // PDU body: encrypted part let mut enc_data = vec![ 12, 0, 202, 158, 245, 73, 200, 196, 158, 187, 0, 135, 112, 187, 78, 253, 225, 60, 114, 70, 163, 77, 66, 75, 203, 192, 9, 184, 3, 167, 100, 222, 182, 217, 253, 203, 107, 132, 172, 128, 70, 81, 210, 183, 199, 117, 14, 7, 215, 161, 70, 44, 155, 123, 215, 95, 187, 68, 64, 145, 45, 106, 53, 92, 218, 167, 32, 113, 41, 179, 199, 93, 184, 19, 95, 44, 40, 91, 135, 207, 72, 105, 99, 217, 231, 6, 167, 98, 66, 15, 121, 183, 134, 103, 196, 161, 76, 117, 188, 160, 178, 41, 63, 253, 98, 127, 97, 67, 66, 175, 212, 255, 232, 212, 49, 154, 253, 229, 120, 25, 195, 157, 4, 167, 14, 203, 124, 241, 195, 102, 98, 45, 123, 75, 181, 38, 240, 116, 19, 47, 14, 200, 210, 153, 60, 194, 102, 179, 76, 65, 224, 161, 248, 169, 152, 208, 222, 177, 167, 43, 19, 130, 168, 105, 24, 173, 44, 97, 84, 13, 1, 191, 57, 214, 26, 41, 205, 106, 74, 58, 194, 114, 107, 110, 175, 16, 90, 183, 185, 21, 126, 100, 197, 127, 205, 114, 220, 100, 254, 60, 27, 170, 191, 139, 219, 3, 136, 73, ]; // PDU security trailer header let mut sec_trailer_header = vec![16, 6, 8, 0, 0, 0, 0, 0]; // PDU security trailer auth value let mut sec_trailer_auth_value = vec![ 5, 4, 6, 255, 0, 16, 0, 28, 0, 0, 0, 0, 2, 127, 68, 205, 187, 173, 74, 166, 160, 153, 40, 89, 22, 35, 47, 177, 219, 87, 47, 250, 226, 224, 221, 190, 242, 117, 26, 8, 246, 227, 66, 83, 51, 25, 251, 201, 147, 221, 150, 156, 38, 106, 201, 152, 57, 242, 145, 76, 4, 83, 52, 239, 115, 245, 174, 74, 173, 95, 157, 251, 98, 108, 134, 71, ]; let mut message = vec![ SecBuffer::new(DATA | READONLY_WITH_CHECKSUM_FLAG, &mut header), SecBuffer::new(DATA, &mut enc_data), SecBuffer::new(DATA | READONLY_WITH_CHECKSUM_FLAG, &mut sec_trailer_header), SecBuffer::new(TOKEN, &mut sec_trailer_auth_value), ]; kerberos_server.decrypt_message(&mut message); println!("Plaintext: {:?}", message[1]); }</code></pre> All these buffers were extracted from the RPC Request PDU, which you already saw at the start of this article. And yes, I already expect the question from you: Wait! Wait! Wait! Where the hell did you get the correct (I suppose correct) session key? 🤨 </blockquote> 😅 I know the user's password, so I can decrypt the previous Kerberos messages and extract the session key 🤭 🤫. Maybe I'll write an article about it someday. Ping me if you need to know how to do it 🙃. Enough talking. Let's run this test. We've been waiting for this for so long. Yeeeeeey. We have successfully decrypted the RPC request! It proves that the implementation is correct! The plaintext buffer on the screenshot is decrypted GetKey</code> RPC request. Hope you find this article informative and useful. Cheers! 😉 Unaddressed topics</h1> If you want to get any of the following topics explained, feel free to contact me. NTLM RPC encryption. But if you are really interested, you can read how it is implemented in sspi-rs</code>: github.com/Devolutions/sspi-rs/193f92a0fd73e43d672e3f9ac5775c894c02b200/src/ntlm/mod.rs#L419-L455</a>.</li> Kerberos auth process. Three-Leg DCE-Style Mutual Authentication</a>.</li> How I decrypted Kerberos traffic and extracted the session key.</li> </ul> Doc, references, code</h1> Code: RPC decryptor github/TheBestTvarynka/trash-code/rpc-decryptor</a>.</li> RPC Encryption - An Exercise in Frustration</a>.</li> [MS-KILE]: Kerberos Binding of GSS_WrapEx()</code></a>.</li> RPC 4121: The Kerberos Version 5 GSS-API</a>.</li> EncryptMessage</code> (Kerberos) function</a>.</li> SecBuffer structure (sspi.h</code>)</a>.</li> [MS-RPCE]</a>.</li> </ul> Tauri error handling recipes 2024-12-26T00:00:00+00:00 Intro</h1> I'm developing the desktop app to meet my own needs: Dataans</a>. Take notes in the form of markdown snippets grouped into spaces. </blockquote> I use Tauri</a> as the main framework and Leptos</a> as the frontend. I am satisfied with the developing experience and the result I got. During the initial implementation (PoC phase), I didn't implement any error handling and called .unwrap()</code> every time. However, after the first release, I needed to add proper error handling and make the app more fault-tolerant. The refactoring process wasn't easy and I spent more time than expected. I learned some important lessons and decided to write this article to summarize my knowledge. Disclaimer: I have experience using Tauri</code> only with Leptos</code> and other Rust-based frontend frameworks. Error handling approaches may be different when using JS-based frontend frameworks. Set up</h1> I set up a simple project (basically it is a cargo create-tauri-app</code> with minimal changes) to demonstrate the approach and the progress: github/TheBestTvarynka/trash-code/265f39bfde2aa9fb10055bfee5031714498f7b28/tauri-error-handling</a>. I left a corresponding commit link for each section in this article. So, you can follow this process and even reproduce it with me. So, what do we have? We have one simple Tauri command: // Backend: src-tauri/src/lib.rs #[tauri::command] fn greet(name: &str) -> String { format!("Hello, {}! You've been greeted from Rust!", name) } // Frontend: src/backend.rs pub async fn greet(name: &str) -> String { let args = to_value(&GreetArgs { name }).unwrap(); invoke("greet", args).await.as_string().unwrap() }</code></pre>Let's handle the error</h1> Usually, most of the commands may fail. We may want to return a Result</code> and handle the error on the frontend side. Let's try it. Suppose we need to validate the name. I came up with the following implementation (let's keep it simple): use thiserror::Error; #[derive(Debug, Error)] enum Error { #[error("invalid name: {0}")] InvalidName(String), } fn validate_name(_name: &str) -> Result<(), Error> { Err(Error::InvalidName("Tbt".into())) } #[tauri::command] fn greet(name: &str) -> Result<String, Error> { validate_name(name)?; Ok(format!("Hello, {}! You've been greeted from Rust!", name)) }</code></pre> Unfortunately, now we have a compilation error: error[E0599]: the method `blocking_kind` exists for reference `&Result<String, Error>`, but its trait bounds were not satisfied --> src-tauri/src/lib.rs:14:1 | 5 | enum Error { | ---------- doesn't satisfy `Error: Into<InvokeError>` ... 14 | #[tauri::command] | ^^^^^^^^^^^^^^^^^ method cannot be called on `&Result<String, Error>` due to unsatisfied trait bounds ... 25 | .invoke_handler(tauri::generate_handler![greet]) | ------------------------------- in this macro invocation | ::: /home/pavlo-myroniuk/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/result.rs:527:1 | 527 | pub enum Result<T, E> { | --------------------- doesn't satisfy `Result<std::string::String, Error>: IpcResponse` or `_: ResultKind` | = note: the following trait bounds were not satisfied: `Error: Into<InvokeError>` which is required by `Result<std::string::String, Error>: tauri::ipc::private::ResultKind` `Result<std::string::String, Error>: IpcResponse` which is required by `&Result<std::string::String, Error>: tauri::ipc::private::ResponseKind`</code></pre> 🤔. Our Result<String, Error></code> type must implement the IpcResponse</code> trait. Implementing serde::Serialize</code> should be enough (because of the impl<T: Serialize> IpcResponse for T</code> trait bound. docs</a>). #[derive(Debug, Error, Deserialize, Serialize)] enum Error { #[error("invalid name: {0}")] InvalidName(String), }</code></pre> Great. Compilation successful. Alternatively, you can read another explanation and example: The Tauri Documentation WIP/Inter-Process Communication#error-handling</a>. Lesson 1. All Error</code>s must implement the serde::Serialize</code> trait. What about frontend? How are we going to handle the error? Let's start with the simplest solution: Move the Error</code> type to a separate common</code> crate which can be used by frontend and backend.</li> Expect JsValue</code> to be parsed into Result<String, Error></code>.</li> </ol> pub async fn greet(name: &str) -> Result<String, Error> { let args = to_value(&GreetArgs { name }).expect("GreetArgs to JsValue should not fail"); let js_value = invoke("greet", args).await; from_value(js_value).expect("JsValue to Result should not fail") }</code></pre> Oops, we have a problem in runtime: But why 😭? According to the documentation (v2.tauri.app/calling-rust/#error-handling</a>), Tauri command error will be an exception on frontend 😕. So, we should not expect the Result<String, Error></code>, but handle a JS exception instead. Fortunately, we can ask wasm_bindgen</code> to handle this exception. Tauri docs say nothing about it, but if we open the wasm-bindgen documentation instead, we will find this treasure (wasm-bindgen/attributes/on-js-imports/catch</a>): The catch</code> attribute allows catching a JavaScript exception. This can be attached to any imported function or method, and the function must return a Result</code> where the Err</code> payload is a JsValue</code>: </blockquote> 😲. Let's use it. Now I'm going to change the binding generated by cargo create-tauri-app</code>. #[wasm_bindgen] extern "C" { #[wasm_bindgen(js_namespace = ["window", "TAURI", "core"], catch)] async fn invoke(cmd: &str, args: JsValue) -> Result<JsValue, JsValue>; }</code></pre> And now we can handle the exception as an error: pub async fn greet(name: &str) -> Result<String, Error> { let args = to_value(&GreetArgs { name }).expect("GreetArgs to JsValue should not fail"); match invoke("greet", args).await { Ok(msg) => Ok(from_value(msg).unwrap()), Err(err) => Err(from_value(err).unwrap()), } }</code></pre> And finally, it works 🎉. Here is the resulting code: github/TheBestTvarynka/trash-code/9f6437fe7799e87439430e483c92c0ed590b12f3/tauri-error-handling</a>. Lesson 2. Tauri command error is an exception on frontend. The exception can be caught by using the catch</code> attribute of the #[wasm_bindgen]</code> macro. Bonus: unexpected None</h1> As you can see from the invoke</code> function declaration, command input/output values are serialized into/deserialized from JsValue using the serde_wasm_bindgen</code> crate. But we are also aware that the JS type-system and Rust type-system are different. Thus, not all Rust objects can be represented in a JS type-system. Theoretically, we can find a Rust object, that: obj == serde_wasm_bindgen::from_value(serde_wasm_bindgen::to_value::(&obj).unwrap()).unwrap() // false</code></pre> I mean, objects before and after the serialization + deserialization process are not the same. It was theoretical until yesterday when I faced it in practice 🤪. Suppose we have the following Tauri command: #[tauri::command] fn greet(name: &str) -> Option<()> { // do the work here Some(()) }</code></pre> And the corresponding code on frontend side: pub async fn greet(name: &str) -> Option<()> { let args = to_value(&GreetArgs { name }).expect("GreetArgs to JsValue should not fail"); let js_value = invoke("greet", args).await.expect("should not fail"); let result: Option<()> = from_value(js_value).expect("deserialization should not fail"); info!("Result: {:?}", result); result }</code></pre> Let's try it. Oppps 😯. We have None</code> instead of Some(())</code>. Some(())</code> is serialized in undefined</code> and undefined</code> is deserialized in None</code>. You should be very careful with ()</code> and enum</code>s. You can reproduce it by yourself: github/TheBestTvarynka/trash-code/4a4f62d1d55687cc02538408ceb9eb6fee93187f/tauri-error-handling</a>. Lesson 3. Objects before and after the serialization + deserialization process may be different. Be careful when using types like ()</code>, enum</code>s, etc. If you say that no one uses the Option<()></code> type, then I will disagree. Try to search for Option<()></code> on GitHub and you will find many uses of this type: github/search?q=Option%3C%28%29%3E+lang%3ARust+&type=code</a>. Learned lessons</h1> All Error</code>s must implement the serde::Serialize</code> trait.</li> Tauri command error (Result::Err(E)</code>) is an exception on frontend. The exception can be caught by using the catch</code> attribute of the #[wasm_bindgen]</code> macro.</li> The JS type-system and Rust type-system are different. Objects before and after the serialization + deserialization process may be different.</li> </ol> Fun fact. This article looks nothing like the one I planned. I didn't know about the catch</code> attribute before writing the article. I discovered it accidentally 🤪. I even wrote my own Result</code> and Unit</code> types to implement proper error handling: github/TheBestTvarynka/Dataans/3f2b97e81cbe8b6c4cd0fffc6d0b44c0a8c4e748/dataans/common/src/error.rs</a> (but now I plan to delete them). > refactored error handling in my Tauri project > it turned out harder than I thought > decided to write a blog post about error handling in Tauri > while writing the blog post I found out that I could have done it much easier > now I am going to refactor the error handling again </blockquote> I like this side effect of writing articles. Most likely, you will do some research, investigation, and code examples to better explain your idea. Almost every time I write a blog post I discover something new for me. Writing blog posts is a good way to learn something more deeply 🤩.</li> </ol> Doc, references, code</h1> The Tauri Documentation WIP/Inter-Process Communication#error-handling</a>.</li> v2.tauri.app/calling-rust/#error-handling</a>.</li> wasm-bindgen/attributes/on-js-imports/catch</a>.</li> </ul> Announcing Dataans v.0.1.0 2024-10-19T00:00:00+00:00 Intro</h1> Around half a year ago, I started my new side project: Dataans</code></a>. It already has plenty of good features and I think I'm ready to present it to you. I'd say that it's not ready and missing a lot of functionality, but I remembered the following quote (Lessons learned in 35 years of making software</a>): When you deliver work you’re really proud of, you’ve almost certainly done too much and taken too long. </blockquote> I've been thinking about it for a while and decided to finally publish a first release. Otherwise, it might never happen. I always have something in mind to implement/fix/improve. It is an infinite process. Dataans</h1> The Dataans is a desktop app that allows you to take notes in the form of markdown snippets grouped into spaces. Yes, it's another note-taking app, but it has unique features that I miss in all other note-taking apps. Motivation</h2> I already have a post</a> about the motivation and features I lack in other existing note-taking apps. Please, read the Motivation</a> section of the article to understand why I decided to write my own app. Features</h2> Quake (drop-down) mode. The keybinding can be configured.</li> Cross-platform.</li> All notes are markdown snippets grouped into spaces. The following MD features are supported: Italic, bold, strike-through text.</li> Quotes.</li> Links.</li> Headers.</li> Tables.</li> In-line code and code blocks.</li> Pasting images from clipboard.</li> </ul> </li> Files can be attached to the note.</li> Common keybindings for text editing (like ctrl+k</code> for creating links).</li> The app can be configured using the config file. Color scheme also can be configured.</li> Simple note search.</li> Many different keybindings to control the application.</li> </ul> Read more detailed in the project Wiki: Dataans: Wiki</a>. How to try it</h2> Follow this instructions: github/TheBestTvarynka/Dataans/dataans/README.md</a> 😜 🤪 References & final note</h1> Dataans: Source code</a>.</li> Dataans: Contributing guide</a>.</li> Dataans: Technical decisions explained</a>.</li> Dataans: Wiki</a>.</li> Lessons learned in 35 years of making software</a>.</li> </ol> I want to add one thing: write programs for yourself in your favorite languages. I'm feeling happy every time I use my app for my needs. Even if your future app will do only one task, then still create it. No better software than software that does useful work. dataans 2024-10-19T00:00:00+00:00 Source code: github/TheBestTvarynka/Dataans</a>. The Dataans is a desktop app that allows you to take notes in the form of markdown snippets grouped into spaces. Yes, it's another note-taking app, but it has unique features that I miss in all other note-taking apps. Motivation</h1> I write notes almost every time I use my computer. Usually, it's small pieces of data/code/docs/links I found during the research or bug fixing. So, I need some place to store them and the ability to find needed pieces of information easily. I used to use a single text file and open it in Notepad</code> or VS Code</code>. After the research, I got the file with almost random symbols: Do I need to explain that it is inconvenient and hard to find anything? I needed a better solution. I tried other note-taking apps, but every one of them had flaws I didn't like a lot. What's wrong with existing note-taking apps?</h2> I relate a lot to this blog post (Why build a messenger app only for sending to yourself?</a>). I quote some key points from it but explain them in my own way and how they relate to me. Too many options. I don't need so much test editing functionality. I just want to take notes and have simple markup (styling) functionality. Something like markdown</a> or asciidoc</a>.</li> Unwanted features. This echoes the previous point, but I want to draw your attention specifically to the UNWANTED features. For example, I don't need AI to search for info in my notes. I only need a typical search engine. I don't want AI to write notes for me. I can write down my thoughts by myself.</li> Article-oriented mindset. People don't think in articles. Many note-taking apps look like you are going to write an article instead of just writing down the idea, thought, or a random piece of data. When I see a blank screen with the cursor, I feel I need to write a document with a defined structure, style, and line of thought. It throws off the thoughts I wanted to write at the beginning.</li> </ol> Missing features</h2> Despite the listed flaws above, I also have missing features I would like to have in my note-taking app. Some of them may feel weird and illogical to you, but I know what I want 😝. All note-taking apps I tried lack one or more features listed below. No one contains all of them at once (but if you find such an app, please, tell me about it). Desktop app. Yes, you heard it right. In the era of the web, I want a desktop app. Usually, the browser has dozens of opened tabs across multiple windows. It becomes hard to find the tab with notes (even when it's pinned).</li> Quake (drop-down) mode. I have used the Quake</a> terminal since 2019. I like it a lot and the most pleasant feature is a drop-down mode. I set a keybinding to the F1</code> key and always have my terminal with me. I would like to have the same ability for my note-taking app because I use it often.</li> Cross platform. It should behave the same on Windows and Linux. Other platforms may may be supported too, but it's not a requirement for me.</li> Markdown. It is simple, easy to learn, and looks good. It contains a perfect set of styling and markup functionality for me.</li> Grouping into channels/spaces/folders/or anything like that.</li> </ol> Solution</h1> These are the two closest apps to what I want: Telegram</a>. I have plenty of Telegram channels with one subscriber (me) where I save important, useful, and just interesting information. Despite rich Telegram features, I think you can understand why it isn't a solution.</li> Monoline</a>. It supports MD and has pretty UI, but I would like to organize my notes into groups (like messages are organized in channels in Telegram).</li> </ol> I decided to write my note-taking app after many tries and thoughts. The best part of being a programmer is the ability to do everything I want with the software. It's like a limitless power in the virtual world (unfortunately, everything has a limit, but we aren't about that). Tech stack</h2> We can debate a lot about the best technologies for such a project, but I already made my decision: Main programming language: Rust</code></a>.</li> App framework: Tauri</code></a>.</li> Frontend framework: Leptos</code></a>.</li> </ul> I wrote a comprehensive explanation of the chosen tech stack. You can read it here: tech_stack.md</code></a>. Features</h2> Quake (drop-down) mode. The keybinding can be configured.</li> Cross-platform.</li> All notes are markdown text. The following MD features are supported: Italic, bold, strike-through text.</li> Quotes.</li> Links.</li> Headers.</li> Tables.</li> In-line code and code blocks.</li> Pasting images from clipboard.</li> </ul> </li> Files can be attached to the note.</li> All notes are grouped into spaces. So, space is a collection of notes. It has a name and avatar picture.</li> Common keybindings for text editing (like ctrl+k</code> for creating links).</li> The app can be configured using the config file.</li> Color scheme also can be configured.</li> Simple note search.</li> Many different keybindings to control the application.</li> </ul> ...And more. This is a short list of main futures. The full features list and user manual can be found here: Dataans/wiki/User-manual</a>. Moving further</h1> I'm going to continue to improve Dataans</code> according to my needs. If someone wants some missing functionality, then create an issue</a> or a discussion</a>, and most likely I'll implement it. Or if you are interested in contributing to this project, then read the CONTRIBUTING.md</code></a> document. References & final note</h1> Dataans: Source code</a>.</li> Dataans: Contributing guide</a>.</li> Dataans: Technical decisions explained</a>.</li> Dataans: Wiki</a>.</li> Why build a messenger app only for sending to yourself?</a>.</li> </ol> I want to add only one thing: write programs for yourself in your favorite languages. I'm feeling happy every time I use my app for my needs. Even if your app will do only one task, then still create it. No better software than software that does useful work. crypto-helper 2024-08-24T00:00:00+00:00 Visit this tool at crypto.qkation.com</a>. The crypto-helper is an online app that helps to work with the different crypto algorithms. This app can hash/hmac, encrypt/decrypt, and sign/verify the data, debug JWT tokens, parse ASN1 structures, compute diffs, and more. All computations are performed on the client side. This tool never sends the data the any servers. Motivation</h2> Crypto helper</h3> During my work, I often need to work with bytes sequences that I need to encrypt or decrypt, hash or verify. For example, I might intercept the traffic, extract encrypted Kerberos cipher data, and try to decrypt it using the user's password. Or take payload + signature from the message and try to recalculate it. Such small tasks are very annoying and I decided to optimize this proccess. Basically, it just takes the input bytes in some format and applies the selected algorithm to it. The list of supported algorithms: Argon2</code></li> BCrypt</code></li> MD5</code></li> SHA1</code>/SHA256</code>/SHA384</code>/SHA512</code></li> Kerberos ciphers: AES128-CTS-HMAC-SHA1-96</code>/AES256-CTS-HMAC-SHA1-96</code></li> Kerberos HMAC: HMAC-SHA1-96-AES128</code>/HMAC-SHA1-96-AES256</code></li> RSA</li> Compression: ZLIB</code></li> </ul> JWT debugger</h3> Another interesting feature is the JWT debugger. I had some time working with Azure AD authorization and its JWT tokens. Why did I decide to implement my own if jwt.io</a> already exists? The answer is pretty obvious: the existing one is very inconvenient. I really don't like its big header. AzureAD tokens are pretty big. Such a massive header is just a waste of space.</li> Problems with scrolling. Did you ever try to scroll the header or payload section that contains more than five fields? If not then try.</li> Header and payload fields are not resizable.</li> Every header/payload/key change immediately overwrites the original JWT token. I often want to see the original one, compare it to a new one, or create a few different tokens based on the initial token.</li> Ads. Ads. Ads.</li> </ul> Here is my JWT debugger in action: Supported signature algorithms: none</code></li> HS256</code></li> HS384</code></li> HS512</code></li> RS256</code></li> RS384</code></li> RS512</code></li> ES256</code></li> ES384</code></li> ES512</code></li> </ul> ASN1 debugger</h3> Did you hear anything about ASN1 der</a>? If not, then this tool is not for you 😅 If yes and you work with asn1 structures encoding/decoding a lot, then you are at the right place 😉. The purpose of this tool is to parse input bytes as the asn1 der</a> structure and render the structural representation of it alongside the hex view of its fields and data. A list of supported asn1 types can be found here</a>. Long story short, here is a demo: I think you got it. Basically, it's a reimplementation of lapo.it/asn1js</a> but with my features and my design. There are some of them: Ability to copy asn1 node or node inner value.</li> Decode on ctrl + enter</code>.</li> Different hex view. I compare parsed structure with its hex representation very often, so I need more explicit and highlighted bytes rendering.</li> </ul> To implement it, the custom asn1 parser crate has been written: asn1-parser</a>. Yet another asn1</code> parser? (https://users.rust-lang.org/t/comparison-of-way-too-many-rust-asn-1-der-libraries</a>) </blockquote> It parses the asn1 types and remembers their position in the input data stream. It is a useful feature for pretty rendering. Diff checker</h3> The purpose of the diff-checker</a> is to just show the diff using selected diff algorithms. It lacks many modern diff-tool features. Most likely it will never have them implemented. The purpose of this tool is only to cover a few specific edge cases. I use it from time to time for raw data diff computation using different diff algorithms. Most likely you will never need this feature but at least it is needed by me and doesn't contain any ads. The journey</h2> Written in Rust</a> 🦀 using yew</a> ✨</li> </ul> It's a fun story because I rewrote this tool two times. Firstly, I planned to make it very simple and started implementing using plain html</code>/css</code>/js</code>. Then I realized that it's hard to improve, and maintain, and can cause a lot of stupid bugs. The second choice was React</code>. In the middle of the rewriting, I thought: why do not write it in Rust</code>? After some research, I decided to use the Yew</a> framework. Basically, it's like React</code> but for Rust</code>. From my experience, I can tell that yew is a great framework to write web applications (SPA</a>s) in Rust</code>. It works well, has great documentation, and is easy to work with. I have faced only one problem: styling. We don't have (so far) such convenient libraries for stlyling as we have for React</code>. I'm happy that I wrote this tool. I use it very often and learned a lot during the implementation. I have had to deal with the web frontend in Rust, web assembly, web workers, application designing and implementation, writing custom asn1 parser, deploying, and much more. How to use it</h2> Just follow the link</a> and paste your data. As you can see, the interface is pretty intuitive. Moving further</h2> At this point, this tool has the all needed functionality for me. I plan to improve it continuously according to my needs and goals. If you have any feature requests, then create an issue</a> with the description. It'll be a priority for me. Doc, references, code</h2> Visit this tool at crypto.qkation.com</a>.</li> Source code: TheBestTvarynka/crypto-helper</a>.</li> </ul> Writing unsafe Rust code? Please, consider Miri 2024-06-30T00:00:00+00:00 For ones who don't know (quote src</a>): Miri</a> is an interpreter for Rust's mid-level intermediate representation. Miri is an Undefined Behavior</a> detection tool for Rust. It can run binaries and test suites of cargo projects and detect unsafe code that fails to uphold its safety requirements. </blockquote> 👍 The description is pretty clear. So, why did you write an entire blog post about it? </blockquote> I have a few reasons to do it. Recently I decided to try it on my work project and was excited with the result.</li> I thought Miri</a> was complicated in usage/configuring like other similar checkers, but it turned out very easy and convenient (like all tools in the Rust ecosystem).</li> Show you some of the bugs Miri has found in the project.</li> Convince you to try it.</li> </ul> Getting started</h1> First of all, you need to install Miri: rustup +nightly component add miri</code></pre> Why nightly? The Miri can be run only with the nightly Rust. So, you should be able to compile your project with nightly Rust. Miri can run project binaries, tests, examples, and so on. cargo +nightly miri run cargo +nightly miri run --example <example name> cargo +nightly miri test cargo +nightly miri test <test_name> --features=<feature list></code></pre>Who should use Miri</h1> In my opinion, everyone who writes an unsafe Rust code should use Miri and test their code with this tool. It can be any kind of unsafe code: FFI (except fully generated ones).</li> Unsafe transmutations.</li> Box::from_raw/into_raw</code>.</li> slice::from_raw_parts/_mut</code>.</li> Interaction with OS API.</li> Magic tricks with references and raw pointers.</li> ...and many other cases.</li> </ul> If you are writing any unsafe Rust code, you should consider using Miri. "We have tested our code and it works well in production" </blockquote> You can be just lucky. Fortune and the compiler are on your side then. But one day you can change your code a little bit, just reorder some functions and boom 💥. Now you have an unexplainable magic error/behavior. This and other arguments are well explained in this talk: Unsafe Rust and Miri by Ralf Jung</a>. I highly recommend watching it! Let's find some UB</h1> The first run: Uuups, I can't compile the project with nightly Rust. I have warned you about it above. In my case, updating the Cargo.lock</code></a> is enough. But it's just a case and we can be forced to spend more time on it. The second run was more "successful": ...and more. But I'm not showing you this to ask you to fix my code. I want you to pay attention to the following: look at the second screenshot and tell me what Miri has printed. Comprehensive problem description: Undefined Behavior: memory access failed: alloc872852 has size 250, so pointer to 22 bytes starting at offset 236 is out-of-bounds.</li> The exact place in the code where the error happened: ffi/src/sspi/sec_pkg_info.rs:189:13</code>.</li> An approximate stack trace.</li> A place in the code where the memory was allocated: ffi/src/sspi/sec_pkg_info.rs:167:28</code>.</li> </ul> Isn't this cool? It's very cool! Now I have so much info to fix this UB. I can't ask for more info 😄. Miri has found more bugs than I expected. The UB was in such places I can't even think of. In this PR you can see what UB and memory leaks I have fixed thanks to Miri: Fix UB and memory leaks found by Miri</a>. Limitations</h1> Miri can execute only Rust code. It means you don't have access to all functions from libc</code> or any other foreign functions (system API, FFI). Miri can open the file or read the environment variable. But you can not establish a TCP connection, for example. It's another reason to write low-coupled code with high cohesion</a>.</li> Miri can not ensure that your program is sound. From Miri README.md</a>: Soundness</a> is the property of never causing undefined behavior when invoked from arbitrary safe code, even in combination with other sound code. In contrast, Miri can just tell you if a particular way of interacting with your code (e.g., a test suite) causes any undefined behavior. It is up to you to ensure sufficient coverage. </blockquote> </li> Miri can be slow on big/heavy tests. It's because Miri is an interpreter for Rust's mid-level intermediate representation. It means that the code is not compiled to ASM and is not executed directly on the CPU.</li> </ul> Configuration</h1> Miri can be configured by -Z</code> flags and environment variables. And yes, we don't have any .toml</code> file here. For example: MIRIFLAGS="-Zmiri-disable-isolation -Zmiri-ignore-leak" cargo +nightly miri test</code></pre> The full list of flags is listed in the Miri README.md</a>. Conclusions</h1> If you're writing unsafe</code> Rust code, don't forget to test it. If you're writing unsafe</code> Rust code, don't forget to run it with Miri</a>. That's all. Bye bye 👋. Doc, references, code</h1> Miri</a>.</li> Miri flags</a>.</li> Behavior considered undefined</a>.</li> Talk about Undefined Behavior, unsafe Rust, and Miri</a>.</li> </ul> Git recipes 2024-06-22T00:00:00+00:00 Recently I lectured in my company about git</code></a> and resolving specific situations with it. It inspired me to write this article. I know that there is tons of info about git</code> and related stuff. I wanted to make this one as a comprehensive set of instructions about resolving concrete git situations. Note. This page can be changed and improved in the future (I'll try to save the backward compatibility)! How to read this page</h1> You can read it in any order. If you are interested only in some particular scenario, then read only the corresponding section. All sections are independent from each other and you can start reading anywhere. The required git</code> knowledge level to read this article is pretty low. If you know what are commit, branch, and index (stage), then everything is good. Goals</h2> Explain how to resolve common git</code> tasks I face at work every time.</li> Improve your git</code> knowledge.</li> Dispel the fear of working with git</code> in uncommon cases (if you have one).</li> </ul> Non-goals</h2> Explain how git</code> works inside. There is a great section in the book</a> about git internals: Git Internals</a>.</li> Propagate to work with git</code> only from the terminal. If the GUI is enough for you to perform all needed tasks, then it's great! The less we interact with git</code>, the lower the probability of having problems with it 🙃.</li> Write zero to pro guide. It's just impossible 😄.</li> Replace the Oh Shit, Git!?!</a> page. The Oh-Shit-Git purpose is to quickly resolve common problems. But my purpose is to offer guides for common tasks.</li> </ul> Recipes</h1> I want to leave a few clarifications before I write recipes. HEAD</code>. How does Git know what branch you’re currently on? It keeps a special pointer called HEAD</code>. 3.1 Git Branching - Branches in a Nutshell</a>. What is HEAD in Git?</a></li> ~<number></code> and ^<number></code> is the same in many cases, but not always. Read this SO answer for more details: https://stackoverflow.com/a/2222920/9123725</a>.</li> </ul> Last commit to index</h2> So, you want to "undo" the last commit but save changes. The git</code> has a git reset</code></a> command for that: git reset --soft HEAD~1</code></pre> As a result, you'll not have the last commit and all changes will be in the stage. And yes, you can enter any number of commits instead of 1</code>. If you want changes to be untracked, then use the save command but with the --mixed</code> parameter: git reset --mixed HEAD~1</code></pre>Discard last commit</h2> I know that it's a bad idea, but if you really want, then this is how you can discard the last commit: git reset --hard HEAD~1</code></pre> Note. If you regret your decision, then you can restore lost commit with its hash: git cherry-pick <commit-hash></code></pre> If you don't know its hash, then try to find it with git reflog</code>. Edit last commit</h2> Actually, you only need the git commit --amend</code> to do it. git commit</code> has a lot of flags and possible use cases, but I list here only common ones so you can copy and use them. If you have any changes in the stage, then they will be added to commit after running git commit --amend</code>. For example: vim README.md git commit --amend --no-edit</code></pre> Now the last commit contains changes we made in README.md</code> file. The --no-edit</code> arg means do not edit the commit message. If you want to edit the commit message, then use the -m</code> arg and just nothing (git</code> will ask you for the message). It's also possible to alter the commit author, date, and so on: # Reset commit author saving commit date. This command is helpful when you've changed the `.gitconfig` file. git commit --amend git commit --amend -m "new commit message" git commit --amend --reset-author --no-edit --date="$(git log -n 1 --format=%aD)" git commit --amend --author=<author></code></pre> Run git commit --help</code> for more flags. Edit any commit in branch</h2> Lets say you want to edit 5th commit starting from the current one (5th parent). You can use the git rebase</code> with its interactive mode to do it: git rebase -i HEAD~5</code></pre> The git</code> will open a text file with a list of 5 commits (parent commits). Now you need to write edit</code> instead of pick</code> for commit you want to edit. Safe this file and clone the editor. The rebasing will stop in the specified commit. Now all you need is to edit selected commit and continue rebasing: Check this recipe to know how to edit the commit: Edit last commit</a>.</li> Continue rebasing:</li> </ul> git rebase --continue</code></pre>Example</h3> I want to edit the the parent commit: So, I type git rebase -i HEAD~2</code> and edit the opened text file: When I save the file and close the editor, I see (git status</code>) that rebasing has stopped in the needed commit (1</code> on the screenshot) and git even shows me helpful commands (2</code> on the screenshot): I edit files as I want, commit changes, and continue rebasing: That's all! Nothing complicated 😄. Split commit into two</h2> Oh, it is an easy one. Usually, I do it in two steps: "Undo" the last commit but save changes. Last commit to index</a>.</li> Create two different commits with git add</code> and git commit</code>.</li> </ol> # "Undo" last commit and save its changes. Changes will not be indexed after this command. git reset --mixed HEAD~1 # First commit: git add <files for first commit> git commit -m <first commit message> # Second commit git add <files for second commit> git commit -m <second commit message></code></pre> You can split one commit into multiple ones with this approach. If you need to split another commit in your branch (one of the parent commits), use the git rebase -i</code> and git reset</code> commands. More info: Edit any commit in branch</a>. Split branch into a few pull requests</h2> 😑 😟 😵‍💫 (post source</a>) The algorithm is simple: Decide how to split. How many branched you'll have and what commits you'll move to what branches.</li> If you need to split some commits in order to move them to separate branches, then read the Split commit into two</a> recipe, and go to the step 1. Otherwise, go to step 2.</li> Create needed branches in the base commit of the current branch.</li> Use git checkout</code> and git cherry-pick</code> commands to apply commits to corresponding branches.</li> </ol> Untrack file</h2> If you want git</code> to ignore some file, then just add it to the .gitignore</code> file. For example: echo "credentials.json" >> .gitignore</code></pre> But if this file was tracked by the git</code> before, then this is not enough. You also need to run: git rm --cached</code></pre> More info you can read in this SO answer: How to stop tracking and ignore changes to a file in Git?</a> Just my notes</h2> Commands below are my notes for quick access: # last commit with date git log --pretty=format:"%h%x09%an%x09%ad%x09%s" | head -n 1 # one commit changes git show ff10697de4f # move last 10 commits from HEAD to another_branch. # note. this command will not move the another_branch itself. just adds commits on top of it git rebase -i HEAD~10 --onto another_branch # author and commiter git cat-file commit HEAD # print any file in any commit. more info: https://juplo.de/cat-any-file-in-any-commit-with-git/ git show <hash>:<filepath></code></pre>Conclusion</h1> There are no any smart conclusions. But I have stupid ones 😁: Make small atomic commits. The smaller your commits are, the better. It's easier to work with such commits. Moreover, everything committed in git</code> always stays inside of it and can always be restored.</li> </ul> There are a lot of other git</code> commands. I just listed the most common situations for me. I just leave it here: HN: Nobody really understands git</a>. Doc, references, code</h1> The Pro GIT</a> book.</li> Oh Shit, Git!?!</a>.</li> </ul> Smart card container name 2024-04-22T00:00:00+00:00 Before I start</h1> Goals</h2> Explain how the Windows minidriver</a> treats the smart card container name.</li> Tell how a bad container name can break scard auth.</li> Fun 🥳.</li> </ul> Non-goals</h2> Explain every piece of smart card architecture in Windows.</li> Explain every smart card minidriver function.</li> </ul> Getting Started</h1> In my previous article</a>, I explained the different smart card cache items (files), their format, and purpose. The current one is some continuation of talking about PIV smart cards in Windows. The setup is the same: Windows smart card minidriver + PIV smartcard. But in this case, we'll use a fully emulated smart card 🔥. Note. The current article is full of technical things related to debugging (WinDbg) and reversing (IDA). Depending on your purpose, choose one of the following paths: If you want to reproduce each step with me, then perform all actions from the Debugging environment</a> section and continue reading the article.</li> If you don't want to set up the whole environment but want to reproduce each step on your machine, then download my TTD recoding and log files</a>. But I still recommend reading the Debugging environment</a> section (at least to know how I came up with it).</li> If you only want to read my findings and smart card container name structure, then jump to the What?</a> section.</li> else</code>: read it fully.</li> </ul> Happy reading 😊. Enjoy 😌. Debugging environment</h1> My goal is to simulate the data signing using an emulated smart card. To achieve this I need two key things: Some program signs the data using the smart card.</li> Hook somehow the winscard.dll</code> to use an emulated smart card.</li> </ul> A brief explanation of our debugging environment: Test VM + certificate with private key suitable for the data signing.</li> We'll have a small program written in C#. The only thing this program does is sign the hardcoded data using the smart card.</li> We'll have a bad.dll</code> that hooks the original winscard.dll</code> with our one.</li> And the last but not least thing is a "launcher": another program that creates a signing process, then loads the bad.dll</code> into it to hook the winscard.dll</code>, and then continues the execution.</li> </ol> Pheww, it starts looking like some kind of Frankenstein 🤪 </blockquote> Yes. You are right 🙃. Compile the data signer</h2> The code is very simple. It uses some high level C# API for the data signing. The full code can be found here</a>. Just clone the project and build it with Visual Studio</a>. Note: it required net6.0. The overall algorithm is pretty simple: we create a CSP that matches our smart card and then sign the data using the RSA crypto provider. The simplified code looks like this: // Create needed CSP. CspParameters csp = new CspParameters(1, "Microsoft Base Smart Card Crypto Provider", containerName // smart card container name ); csp.KeyPassword = pwd; // Sign the data. RSACryptoServiceProvider rsaCsp = new RSACryptoServiceProvider(csp); byte[] signature = rsaCsp.SignData(dataToSign, HashAlgorithmName.SHA1, RSASignaturePadding.Pkcs1);</code></pre> As a result, you should get SignDataTmp.exe</code> + SignDataTmp.dll</code>. Compile your own version of winscard.dll</code></h2> git clone https://github.com/Devolutions/sspi-rs.git cd sspi-rs/ffi git checkout 4cd21349311b8edad03cc0dc8edf56407ccc22b9 cargo build --all-features</code></pre> Of course, during my first debugging, I had a dev version of the sspi-rs</code>. But now all developed functionality has been merged</a>, so we need to pick a specific commit. After that you will have sspi.dll</code> in the target/debug</code> directory. You can rename it to winscard.dll</code> if you want (but it's not required). Compile bad.dll</code></h2> This bad.dll</code> represents how I do the hooking. There are many hooking techniques. My choice criteria were that it should work and not be hard to implement. So, I chose the MinHook</a> library. It is easy to use and does its job well. Project source code can be found here</a>. Just clone the project and build it with Visual Studio</a>. Note: do not forget to properly link it with MinHook</code>. But here is a surprise. When you try to read the code you'll find out that I hook some weird functions instead of the winscard.dll</code>. The reason for that is delayed loading</a>. We can not simply hook the winscard.dll</code> because the basecsp.dll</code> loads it via delayed loading 😕. But it is still possible to hook. Thanks to @awakecoding</a> I know how to do it. He had researched</a> and implemented it in the MsRdpEx</a>: fix winscard.dll delay-loading interception from basecsp.dll</a>. I reused some code and make it work for my debugging program 😅. So, as a result, you should get bad.dll</code>. Do not forget to set your own paths in the code! Compile the launcher</h2> The "launcher" is named as "HookLibrary" due to my own historical reasons 🤪. The full source code can be found here</a>. Important: do not forget to change hardcoded paths to your ones. It works as follows: Runs the data signer executable in a separate process, but does not run it.</li> Injects the bad.dll</code> into the process memory forcing it to load this dll. In turn, the bad.dll</code> will hook all needed methods.</li> Continues the execution.</li> </ol> So, as a result, you should get HookLoadLibrary.exe</code>. Run it</h2> Before trying to run this machinery we need to configure the emulated smart card. To do this set up the following environment variables: Name</th> Value</th> Example</th></tr></thead> SSPI_LOG_PATH</code></td> Path to the log file</td> D:\test_data\sspi.log</td></tr> SSPI_LOG_LEVEL</code></td> Log level</td> trace</td></tr> WINSCARD_PIN</code></td> Smart card PIN code</td> 214653</td></tr> WINSCARD_CERT_PATH</code></td> Path to the .cer file containing the smart card certificate</td> D:\test_data\user.cer</a></td></tr> WINSCARD_PK_PATH</code></td> Path to the .key file containing the smart card certificate</td> D:\test_data\user.key</a></td></tr> WINSCARD_CONTAINER</code></td> Container name</td> 1b22c362-46ba-4889-ad5c-01f7abcabcabedw</td></tr> WINSCARD_READER</code></td> Reader name</td> Microsoft Virtual Smart Card 2</code></td></tr> </tbody></table> And finally, run HookLoadLibrary.exe</code>. Debugging</h1> The first run</h2> Let's try to sign the data using the emulated smart card. Expectedly (otherwise, this article would not exist 🙂), we got an exception. Our goal for the rest of the article is to figure out the cause of the error and fix it. Error location</h2> At this point, I don't know what caused the error. So, let's use WinDbg</a> and make a TTD</a> recording. Finding a function that returns an error in this way will be easier. With TTD recording I can walk through the execution without rerunning the app. Note. Because the HookLoadLibrary.exe</code> runs the SignDataTpm.exe</code> in a separate process, you should attach WinDbg to this process instead of running the HookLoadLibrary.exe</code> in the WinDbg. Good. Now we can start debugging. At this point, I can assume only something: some function in msclmd.dll</code> or basecsp.dll</code> fails and breaks the signing. Let's see the last sspi-rs</code> FII calls in the logs to have at least some orientation on where to dig. If we catch a moment of the last winscard calls, we'll retrieve a call stack and then analyze those functions. These are the last meaningful records in the log file: We can see the SCardTransmit</code> function call. I'll search for them in the WinDbg. But because I use the custom winscard.dll</code>, all WinSCard-related functions have a Rust_</code> prefix. Example: I have omitted some manipulations so that this article is not too boring. In short, on the screenshot below we can see the last SCardTransmit</code> call and part of the call stack: As I expected, we see the msclmd.dll</code> and basecsp.dll</code> functions. The next algorithm is the following: I take the highest function from the call stack and check the resulting status code in the WinDbg. If it succeeds, I take a lower function from the call stack and check its status code. When I finally find a failing function, I'll use WinDbg and IDA to see the exact failing point. I expect something like return 0x80100004;</code> or any other status code. Hmmm 🤔. All functions up to the C#-related ones were succeeded. Perhaps the last calls to SCardTransmit</code> occurred after a signing error and Windows was trying to finalize the error/gather some information OR the signing process is not even started yet. I need another function to start with. Let's try SCardReadCacheW</code>. I had a "great" experience with smart card caches in the past</a>, so maybe it'll help me this time. Thanks to TTD I don't need to rerun it and can walk through the recording again and again. After some time I finally found something that looks like data signing 😌. Let's see the status codes of the highlighted functions: Cool! Now we know what the msclmd!I_PIVCardSignData</code> function returns the SCARD_E_INVALID_PARAMETER</code></a> status code. I need to find where in this function body the error has been thrown. Walking through every ASM instruction is boring and time-consuming, so I took an IDA and reversed this function as much as I could. After that, I marked some places (other function calls, if</code>s, and so on) that could fail and checked them with WinDbg. Okaaay, so, the key algorithm and id extraction have failed 🤔. Reversed pseudo-code: return_code = I_GetKeyAndAlgFromMapIndex( card_data, ((_BYTE )signing_info + 4),// bContainerIndex (unsigned __int8 )&key_buff, &key_alg, &key_size);</code></pre> I dug into it and found exact place where the fail happens. I skipped this part for you and show you the result: Here is the reversed part of the ASM code above. We can see that invalid cert file tag causes the SCARD_E_INVALID_PARAMETER</code> error: On the screenshot above the part of the msclmd!I_GetPIVKeyIDFromPIVCertFileTag</code> function is shown. The msclmd!I_IsValidPIVCertFileTag</code> function has returned 0</code> (false</code>). It means that the cert file tag is not valid. So, the cert_file_tag</code> must meet highlighted conditions to be valid. In our case it's equal to 0xaaaaaa</code> which is obviously not valid. To fix it we need to figure out how the cert_file_tag</code> is constructed and how we can change it to met one of the needed values. I did it for you, so take a look at the following screenshot: The screenshot above contains a part of the msclmd!I_GetKeyAndAlgFromMapIndex</code> function. This piece of code gives us answers to all our questions. First of all, the cert_file_tag</code> value is extracted from the cmap_record</code> using the swscanf_s</code></a> function. In simple words, it takes the cmap_record</code> buffer pointer, skips the first 30 characters (bytes), scans the following 6 characters (bytes), converts them from hex to decimal, and, finally, assigns it to the cert_file_tag</code> variable. This is our cert_file_tag</code> which is not valid. In turn, the cmap_record</code> is a pointer to the CONTAINER_MAP_RECORD</code></a> structure where the first field is a container name: #define MAX_CONTAINER_NAME_LEN 40 typedef struct _CONTAINER_MAP_RECORD { WCHAR wszGuid [MAX_CONTAINER_NAME_LEN]; BYTE bFlags; WORD wSigKeySizeBits; WORD wKeyExchangeKeySizeBits; } CONTAINER_MAP_RECORD, PCONTAINER_MAP_RECORD;</code></pre> According to the Microsoft's specification: The wszGuid</code> member consists of a UNICODE character string representation of an identifier that CAPI assigned to the container. This is usually, but not always, a GUID string. Identifier names cannot contain the special character “\”. When read-only cards are provisioned, the provisioning process must follow the same guidelines for identifier names. </blockquote> What?</h2> 😮‍💨 Let's summarize it all. The CONTAINER_MAP_RECORD</code> structure contains a smart card container name. The container map record value is extracted from the smart card cache</a>. This container name (wszGuid</code>) is not a random GUID value. This is a special value that must contain one of the allowed certificate file tags. The smart card driver (msclmd.dll</code>) decides how to sign the data based on this cer_file_tag</code>. All possible certificate file tags are defined and can be found in the PIV smart card specification</a> and open-source projects</a> that follow this spec. Soooo, if we just change the container name, then data signing should work? 👉👈 </blockquote> It seems like yes, it should. We have nothing to lose. Let's try it. The fix</h2> Change the WINSCARD_CONTAINER</code> environment variable. Name</th> Value</th> Example</th></tr></thead> WINSCARD_CONTAINER</code></td> Container name</td> 1b22c362-46ba-4889-ad5c-01f7aa5fc10awww</td></tr> </tbody></table> 🤞🤞🤞 🎉 It works! 🎉 Ignore the error at the bottom of the terminal. The only important thing is the resulting signature. It means, that the data signing using the emulated smart card works well 😌. Even in the sspi-rs</code> logs we can see SCardTransmit</code> calls with input padded digest and output signature: 😊😊😊 I don't know why this container name format is not documented anywhere (at least I didn't find any mention about it on the Internet). Conclusion</h1> There are a lot of ways of debugging programs. In this article, I told you about one of my cases where I need to be creative and patient. In the end, I could tell that Windows is bad and so on, but I'll say the following instead: Everything is possible. Any bug and problem is solvable. Be creative and have patience. </blockquote> Wish you more interesting bugs and fun debugging! 😝 Doc, references, code</h1> Smart Card Minidriver Overview</a>.</li> Emulated smart cards implementation</a>.</li> Source code of all used debugging programs</a>.</li> WinDbg</a>.</li> TTD</a>.</li> TTD recording and log files</a>.</li> </ul> Windows smart card cache 2024-04-04T00:00:00+00:00 Getting Started</h1> A few months ago I had a great opportunity to implement the smart card emulation</a>. The whole point was to emulate the smart card behavior without an actual smart card. I can talk a lot about such stuff, but in this article, I'm going to share my experience in smart card cache exploration and implementation. To make all my decisions clear for you, I provide you with some additional context. Imagine the reimplemented winscard.dll</code> without actual calls to the security device but with the smart card emulation under the hood. Boom 💥! We have scard auth without a physical device. You can have questions like "How can we replace the original dll", "What is the point of such implementation", "What are the limitations of it", and so on. Unfortunately, it's out of the scope of this article. Here I focused only on debugging and smart card cache (I'll explain the reason for it further in the article). Goals</h2> A brief overview of the smart card architecture in Windows.</li> Explain some smart card cache items for PIV</a> smart cards in Windows.</li> Fun 🥳.</li> </ul> Who should read this article: Ones who are interested in how PIV smart cards work on Windows.</li> Ones who just need PIV scard cache items format (Hi 👋, msclmd.dll</code>).</li> </ul> Non-goals</h2> Explain every piece of smart card architecture in Windows.</li> </ul> Overview</h1> In Windows, WinSCard</a> is the lowest accessible API for communicating with smart cards. You operate only using handles (SCARDCONTEXT</code>/SCARDHANDLE</code>/etc), raw APDU</a>s (SCardTransmit</a> function), and a bunch of other low-level functions. If you look at the CSP and KSP-based architecture diagram</a>, you'll see WinScard almost at the bottom of the picture: </a> Let's analyze this diagram above. Usually, when you deal with any crypto operations in Windows, you use some high-level API - CryptoAPI</a>. But the CryptoAPI is just a high-level API and every CSP</a> implements it. In the case of smart cards, the corresponding CSP (basecsp.dll</code>, in our case) can not contain the CryptoAPI implementation because all crypto operations should be performed on the security device. This is why BaseCSP uses (depends on) the smart card minidriver. Such a minidriver translates high-level functions into WinSCard API calls. Nothing more. Usually, every smart card vendor ships its smart card driver (you can see it on the diagram). The Windows itself contains two smart card drivers: One driver for Windows proprietary smart cards with closed specifications and documentation.</li> And another driver for PIV-compatible smart cards. It has open specifications (Smart Card Minidriver Specification</a> + NIST SP 800-73-4</a>) but closed documentation.</li> </ul> "...two smart card drivers:" </blockquote> It was a small lie 😅. I mean, yes, it's true, but those two drivers are implemented inside one dll: msclmd.dll</code>. For the rest of this article, we'll talk only about msclmd.dll</code> and PIV-compatible smart cards. Got it. But why is card cache the main topic of this article? </blockquote> I have a few reasons for it: No matter how good our WinSCard implementation is, authentication will fail without a working cache.</li> Only a few cache items are described in the Window's minidriver specification. Many of them are undocumented and unknown (or known but the final file structure is not clear).</li> It was the hardest part to debug and implement 😝. It took me a few months.</li> </ul> Firstly, I thought I can found cache items format in the specification. But I was wrong. Secondly, I was astonished that I didn't found any information about them in the Internet. In the result, it took me more then month to properly reverse/debug msclmd.dll</code> and extract all needed information about scard cache items. And I decided to write an article you are reading now. Note. I didn't reverse the whole smart card driver. I explored only needed functions. So, this article has gaps in some places filled with my assumptions. If you are more experienced and know more about such places, please, leave a comment and I'll edit the article to be more complete. Thanks😌 </blockquote> Debugging</h1> It's easier to work if we have a concrete destination point. Let's take the Remote Desktop Client (mstsc.exe</code>), replace the wiscard.dll</code>, and try to connect to the remote machine using the emulated smart card. The established RDP connection is our goal. To make my life easier, I took the MsRdpEx</a> and hooked the winscard.dll</code> using the MSRDPEX_WINSCARD_DLL</code></a> environment variable. Pretty simple and usable. For the further work, only two instruments will be used: IDA.</li> WinDbg</a>.</li> </ul> Time Travel Debugging</a>. I encourage you to try it! This technique saved me a lot of time. The possibility of re-playing and moving backward gives us almost unlimited power 💪. I do not recommend debugging smart cards with API Monitor</a> because it can miss some WinSCard API function calls. Let's start the journey</h1> I suppose the whole debugging process is boring for you, so I'll show relevant reversed parts of the msclmd.dll</code> with small descriptions. If you need only resulting cache items format (structure), then you should skip next sections and jump right to the implementation</a>. At some point in time, the msclmd.dll</code> calls the SCardReadCacheW</code></a> function to extract some information from the smart card cache. What data should we return? What format does the data (cache file) have? The next chapters contain answers to those questions. You can use the following material in two cases: You are developing your own winscard.dll</code> replacement.</li> You are using the WinSCard API (maybe developing your minidriver) under Windows and want to know what the SCardReadCacheW</code> function returns.</li> </ul> Otherwise, it's just a fun reading for you. Enjoy 😚 Cached_CardmodFile\Cached_Pin_Freshness</code></h2> Usually, when the driver asks you for the Cached_CardmodFile\Cached_Pin_Freshness</code> cache item, it wants to form the CARD_CACHE_FILE_FORMAT</code></a> structure and the PIN freshness counter is the first field in the structure: typedef struct _CARD_CACHE_FILE_FORMAT { BYTE bVersion; BYTE bPinsFreshness; WORD wContainersFreshness; WORD wFilesFreshness; } CARD_CACHE_FILE_FORMAT, PCARD_CACHE_FILE_FORMAT;</code></pre> The actual cache file is just one-byte number that represents a PIN freshness counter. If you want, it can be two-byte value, but the value will be casted to one-byte. Implementation example</a>: const PIN_FRESHNESS: [u8; 2] = [0x00, 0x00]; cache.insert( "Cached_CardmodFile\\Cached_Pin_Freshness".into(), PIN_FRESHNESS.to_vec(), );</code></pre> All three freshness counters are extracted from the card cache using the msclmd!I_GetPIVCachedFreshnessCounter</code> function. Every freshness counter has an ID: 1</code> - PIN freshness.</li> 2</code> - file freshness.</li> 3</code> - container freshness.</li> </ul> What about bVersion</code> field? </blockquote> Currently, it's always equal to 1</code> (this value is hardcoded in the DLL. you'll see it on the next screenshots). But the CARD_CACHE_FILE_FORMAT</code> structure creation is not just cache items copying to the structure fields. The driver has an additional algorithm. Okay, let's move on. Cached_CardmodFile\Cached_File_Freshness</code></h2> We are already familiar with freshness counters, so I skip this cache item introduction and jump to the implementation</a>: const FILE_FRESHNESS: [u8; 2] = [0x0b, 0x00]; cache.insert( "Cached_CardmodFile\\Cached_File_Freshness".into(), FILE_FRESHNESS.to_vec(), );</code></pre> Note. The [0x0b, 0x00]</code> is almost random value. The freshness counter is just a counter. More importantly, this counter matches the next cache items (files) in this article. I'll explain later what I mean. Cached_CardmodFile\Cached_Container_Freshness</code></h2> And finally, the container freshness counter. Code</a>: const CONTAINER_FRESHNESS: [u8; 2] = [0x01, 0x00]; cache.insert( "Cached_CardmodFile\\Cached_Container_Freshness".into(), CONTAINER_FRESHNESS.to_vec(), );</code></pre> So the driver can finally create a CARD_CACHE_FILE_FORMAT</code> structure? </blockquote> Well yes but actually no. Let's see a small reversed part of the msclmd!I_GetPIVCache</code> function: What is an activity_count</code>? </blockquote> It's a result value of the msclmd!I_GetCardActivityCount</code> function. Here is how it calculated: The activity counter is calculated based on the dwEventState</code> value returned from the SCardGetStatusChangeW</code></a> function. Furthermore, this value also affects the CARD_CACHE_FILE_FORMAT</code> structure creation. I pay attention to it on purpose because it's important to know the structure fields' values. Cached_CardProperty_Read Only Mode_0</code></h2> To make further explanations easier, I introduce the cache value header: typedef struct _CARD_FILE_HEADER { CARD_CACHE_FILE_FORMAT file_format; uint16 padding1; uint16 padding2; uint16 padding3; uint32 value_len; } CARD_FILE_HEADER;</code></pre> value_len</code> bytes are placed right after this header. I can assume that padding bytes also have a meaning but in my practice, they are always equal to zero and I didn't find any application for them. This is a general layout for cache items related to the PIV smart card. Now we can finally see, how freshness and activity counters affect other cache files. In the case of the current cache item, the value_len</code> is 4, and value bytes represent the BOOL</code> flag. Example implementation</a>. The meaning of this flag hides in the specification. It represents the CP_CARD_READ_ONLY</code> card property: If True, all write operations are blocked at the Base CSP layer. This flag also affects the data cache. If the card indicates that it is read-only, the Base CSP/KSP does not write to the cardcf file. </blockquote> Cached_CardProperty_Cache Mode_0</code></h2> The structure of this cache item is the same as in the previous one</a>. The value_len</code> is 4, and value bytes represent the DWORD</code> number. Example implementation</a>. It corresponds to the CP_CARD_CACHE_MODE</code> card property and can have one of the possible values: #define CP_CACHE_MODE_GLOBAL_CACHE 1 #define CP_CACHE_MODE_SESSION_ONLY 2 #define CP_CACHE_MODE_NO_CACHE 3</code></pre>Cached_CardProperty_Supports Windows x.509 Enrollment_0</code></h2> The value_len</code> is equal to 4 in the CARD_FILE_HEADER</code> structure and those 4 bytes represent the BOOL</code> value. Example implementation</a>. The meaning of this flag is the following: Indicates whether Windows PKI should be allowed to write or renew certificates on the card. This should be used to avoid unexpected results because of a lack of support for multiple PINs in Windows PKI enrollment client. </blockquote> Cached_GeneralFile/mscp/cmapfile</code></h2> It is worth noting that smart cards can have a few key containers</a>. And here every card container are represented in the CONTAINER_MAP_RECORD</code></a> structure: #define MAX_CONTAINER_NAME_LEN 40 typedef struct _CONTAINER_MAP_RECORD { WCHAR wszGuid [MAX_CONTAINER_NAME_LEN]; BYTE bFlags; WORD wSigKeySizeBits; WORD wKeyExchangeKeySizeBits; } CONTAINER_MAP_RECORD, PCONTAINER_MAP_RECORD;</code></pre> The current cache item describes one or more such container records. The value_len</code> is equal to 0x56</code> (size of the CONTAINER_MAP_RECORD</code>) amount of container records structures. Example implementation</a>. Cached_CardmodFile\Cached_CMAPFile</code></h2> This one is almost the same as the previous one</a> but without the CARD_FILE_HEADER</code> structure at the start of the cache file. You can just remove the first 16 bytes from the Cached_GeneralFile/mscp/cmapfile</code> cache file and you'll get this one. Example implementation</a>. The driver (msclmd.dll</code>) usually uses this cache file to find the needed card container specified in the credentials. For example, basecsp!I_ContainerMapFind</code>, or basecsp!ContainerMapFindContainer</code>, or basecsp!FindCard</code> functions. Cached_ContainerProperty_PIN Identifier_0</code></h2> The value_len</code> is equal to 4 in the CARD_FILE_HEADER</code> structure and those 4 bytes represent the DWORD</code> value. Example implementation</a>. As I understand from the specification, the PIN identifier is associated with the role. typedef DWORD PIN_ID, PPIN_ID; #define ROLE_EVERYONE 0 #define ROLE_USER 1 #define ROLE_ADMIN 2</code></pre>Cached_ContainerInfo_00</code></h2> This cache file describes the key container for more information about which keys are present. The value_len</code> of the CARD_FILE_HEADER</code> structure depends on the key inside of this container. You'll see it further. Here is the example implementation</a>. It's easier to follow the explanation when you have the code. After the CARD_FILE_HEADER</code> structure we have a 16-byte container info header. The purpose of the first 12 bytes is unknown to me, but the last 4 is the length of the rest of the data (which also depends on the container key). The actual data has the following format: // This structure is assembled by me typedef struct _CONTAINER_INFO_DATA { PUBLICKEYSTRUC public_key_info; RSAPUBKEY rsa_pub_key; BYTE[] public_key_modulus; } CONTAINER_INFO_DATA; // https://learn.microsoft.com/en-us/windows/win32/api/wincrypt/ns-wincrypt-publickeystruc typedef struct _PUBLICKEYSTRUC { BYTE bType; BYTE bVersion; WORD reserved; ALG_ID aiKeyAlg; } BLOBHEADER, PUBLICKEYSTRUC; // https://learn.microsoft.com/en-us/windows/win32/api/wincrypt/ns-wincrypt-rsapubkey typedef struct _RSAPUBKEY { DWORD magic; DWORD bitlen; DWORD pubexp; } RSAPUBKEY;</code></pre> Note. If the public_key_modulus</code> is smaller than the specified key length (for example, by one byte), then it should be padded with zeroes to match the key length. But you wrote above the smart card can have more than one key container. Which one should we use here? </blockquote> It's a trick question. I don't know 😞. In my experience, I was working with smart cards that have only one key container. Cached_GeneralFile/mscp/kxc00</code></h2> In short, this cache file contains a compressed DER-encoded certificate usinf ZLIB compression. Example implementation</a>. Compression</a>. Of course, the value_len</code> of the CARD_FILE_HEADER</code> structure depends on the compressed data length. So, move on. The actual data has the following format: // This structure is assembled by me typedef struct _CONTAINER_INFO_DATA { int16 flags; int16 uncompressed_cert_len; BYTE[] compressed_cert_len; } CONTAINER_INFO_DATA; </code></pre> I suppose the first two bytes should indicate if the certificate is compressed or not. But I didn't find any evidence about it and just used extracted values from the real smart card. Again. What certificate should we use? </blockquote> This time I know 😝. In the specification, kxc00</code> is key exchange cert 0. Cached_CardProperty_Capabilities_0</code></h2> This cache file describes the card and card-specific minidriver combination for the functionality that is provided at this level, such as a certificate or file compression. The value_len</code> is equal to 12 in the CARD_FILE_HEADER</code> structure and those 12 bytes represent the CARD_CAPABILITIES</code> structure</a>: #define CARD_CAPABILITIES_CURRENT_VERSION 1 typedef struct _CARD_CAPABILITIES { // The version of the structure that is being used. DWORD dwVersion; // Set TRUE to indicate that the card minidriver implements its own compression of certificates. BOOL fCertificateCompression; // Set TRUE to indicate that the card can generate keys. BOOL fKeyGen; } CARD_CAPABILITIES, PCARD_CAPABILITIES;</code></pre> Example implementation</a>. (Just a reminder. In Windows, the BOOL</code> type is 4-byte long 😕) Cached_CardProperty_Key Sizes_1</code></h2> This cache file describes the different key length values that are available. The value_len</code> is equal to 20 in the CARD_FILE_HEADER</code> structure and those 20 bytes represent the CARD_KEY_SIZES</code> structure</a>: #define CARD_KEY_SIZES_CURRENT_VERSION 1 typedef struct _CARD_KEY_SIZES { DWORD dwVersion; DWORD dwMinimumBitlen; DWORD dwDefaultBitlen; DWORD dwMaximumBitlen; DWORD dwIncrementalBitlen; } CARD_KEY_SIZES, PCARD_KEY_SIZES;</code></pre> Example implementation</a>. Sometimes the driver can ask for the Cached_CardProperty_Key Sizes_2</code> cache item. I'm not sure what this number means. Maybe it depends on the key container, maybe on some algorithm id. IDK 😓 Conclusion</h1> I'm not sure what to conclude. The only thing that stuck in my head is that anything related to smart cards is complex, hard to implement/understand, or find documentation. Anyway, we have no other choice than to live with what we have and try to improve it 😊 I hope if one day someone is stuck with a similar problem as me, then they'll find this article and use it as a reference. I just don't know who else might need such a detailed description of smart card cache files 😅 Doc, references, code</h1> Smart Card Architecture</a>.</li> winscard.h</code></a>.</li> Smart Card Minidrivers</a>.</li> Smart Card Minidriver Specification</a>.</li> Implemented smart card caches: scard_context.rs</code></a>.</li> Time Travel Debugging</a>.</li> </ol> Debugging with API Monitor 2023-07-28T00:00:00+00:00 The API Monitor</a> speaks for itself. It's a great tool for the system API calls debugging, monitoring, information extraction, exploring. Especially in cases when you don't know how exactly API works. Disclaimer. (If you don't get it yet). Mentioned API Monitor is a tool for monitoring the system API calls. So, I you are back/front-end dude which is not interested in such stuff, this article will not be useful for you. Goals</h1> Goals: Explain how to deal with cases when existing API definitions are not enough.</li> Show on the example how we can debug custom DLLs and how to write definitions for them.</li> </ul> Non-goals: Write the "ultimate" guide to the API Monitor</code>.</li> Describe how the API Monitor</code> works under the hood.</li> </ul> If you are not familiar with the API Monitor, then I propose you reading the following articles: Monitoring your first application</a>.</li> API Monitor (youtube video)</a>.</li> </ul> The current article does not include API Monitor basics. Let's debug WinSCard API</h1> So, what problem or edge cases do you want to talk about? </blockquote> I prefer example-based explanations, so we start from the API monitoring. And today's target is WinSCard API</a>. In a short, it's a Microsoft's API for the communication with smart cards. I have a small program that uses smart card for some operations. Let's assume I don't know what exactly it's doing or know what but don't know how. Out task for now is to try to explore the API calls. Start as usual: Select only needed API.</li> </ol> Run the executable in the API Monitor. Here is what I got:</li> </ol> Cool. We see a lot of functions calls, their before and after parameters, flags, and much more. But here is one problem that blocks us from the further investigation: we don't have recorded in- and outbound APDU messages. Note. The APDU message - is the communication unit between a smart card reader and a smart card. More info: wikipedia</a>, yubikey-reference/apdu</a>. The ScardTransmit</code></a> function has in- and outbound APDU messages and they defined in the function signature as follows 👇: LONG SCardTransmit( [in] SCARDHANDLE hCard, [in] LPCSCARD_IO_REQUEST pioSendPci, [in] LPCBYTE pbSendBuffer, // inbound APDU message buffer [in] DWORD cbSendLength, [in, out, optional] LPSCARD_IO_REQUEST pioRecvPci, [out] LPBYTE pbRecvBuffer, // outbound APDU message buffer [in, out] LPDWORD pcbRecvLength );</code></pre> From the function definition we can see that buffers length is defined in the cbSendLength</code> and pcbRecvLength</code> parameters. So, it's logical to assume that API Monitor will capture those buffers during the recording because lengths are known. But unfortunately, we have only pointer value recorded. Why it's happening? The API Monitor captures buffers with defined (known) length as I remember from the documentation. </blockquote> The answer is in the next section. Time to fix XML definitions</h1> To answer the previous question, lets ask a new one: how the API Monitor even parse and recognize the API functions, parameters, etc? Of course, here is no any magic, but XML definitions 😔 . Basically, the API Monitor has XML file for every supported library with defined API in it. So, maybe we can just edit existing XML for the WinSCard API and record the buffers? </blockquote> Exactly. Someone who wrote the XML definition for the WinSCard didn't put enough attention and wrote them somehow. Initially, I read about fixing those XML definitions in this</a> article. I was surprised that only very little devs know about it and how to use it. Okay, enough talking, time to fix the code:   <Variable Name="[SCARD_DISPOSITION]" Type="Alias" Base="LONG"> <Display Name="LONG" /> <Enum> <Set Name="SCARD_LEAVE_CARD" Value="0" /> <Set Name="SCARD_RESET_CARD" Value="1" /> <Set Name="SCARD_UNPOWER_CARD" Value="2" /> <Set Name="SCARD_EJECT_CARD" Value="3" /> </Enum> </Variable></code></pre> <Api Name="SCardDisconnect"> <Param Type="SCARDHANDLE" Name="hCard" /> <Param Type="[SCARD_DISPOSITION]" Name="dwDisposition" /> <Return Type="[SCARD_ERROR]" /> </Api> <Api Name="SCardEndTransaction"> <Param Type="SCARDHANDLE" Name="hCard" /> <Param Type="[SCARD_DISPOSITION]" Name="dwDisposition" /> <Return Type="[SCARD_ERROR]" /> </Api> <Api Name="SCardTransmit"> <Param Type="SCARDHANDLE" Name="hCard" /> <Param Type="LPCSCARD_IO_REQUEST" Name="pioSendPci"/> <Param Type="LPCBYTE" Name="pbSendBuffer" Count="cbSendLength" /> <Param Type="DWORD" Name="cbSendLength" /> <Param Type="LPSCARD_IO_REQUEST" Name="pioRecvPci" /> <Param Type="LPBYTE" Name="pbRecvBuffer" PostCount="pcbRecvLength"/> <Param Type="LPDWORD" Name="pcbRecvLength" /> <Return Type="[SCARD_ERROR]" /> </Api></code></pre> Note. You will have other paths if you installed the API Monitor in the non-default location. You can compare old and new XML and see the difference. Now all should work. Let's try again to record API calls. Here is my result: Congratulations 🌺 . Now it works well and we can observe input and output buffers. What if we have a custom DLL to monitor?</h1> Firstly, I was not planned to write this section. But if I touched fixing XML definitions then it's obviously fun to write definitions for our custom library and try to debug it. But before actual debugging, let's write the program and dll itself. I suggest a simple task: Look at this site: https://imgur.com</a>. Imgur is an American online image-sharing and image-hosting service. It has an API</a>. Let's write a library that encapsulates API calls and provides us with a simple interface for communication with Imgur. I plan to implement only two functions: ImgurInitClient</code>: initializes the Imgur client (or context).</li> ImgurGetComment</code>: retrieves the comment information based on its id.</li> </ul> And in order to make a more "real" example, I'll pack Rust code into the dll and call its functions from the C++ code. flowchart TD A[C++ code] --> dll[dll] subgraph dll C[Rust FFI bindings] --> D[Rust Imgur API client] end </div>Write a program and dll</h2> Note: this section contains a lot of code. There is no point to read it very carefully. You should just have a rough idea of what it does. Good. We have a purpose but don't have any obstacles. Firstly, we need to write a pure Rust implementation. There is no point to explain it in detail, so I just paste the code below: /// The Imgur API client pub struct ImgurApi { /// Imgur application client ID client_id: String, /// Imgur application client secret _client_secret: String, } impl ImgurApi { /// Initializes the Imgur API client pub fn init<I, S>(client_id: I, _client_secret: S) -> Self where String: From<I>, String: From<S>, { Self { client_id: client_id.into(), _client_secret: _client_secret.into(), } } /// Retrieves the comment information based on its id pub fn comment(&self, comment_id: u64) -> ImgurResult<Comment> { let raw_comment = Client::new() .get(format!("https://api.imgur.com/3/comment/{}", comment_id)) .header(AUTH_HEADER, format!("Client-ID {}", self.client_id)) .send()? .bytes()?; let comment: Comment = serde_json::from_slice(&raw_comment)?; Ok(comment) } }</code></pre> The full imgur-api-client</code> code: @TheBestTvarynka/trash-code/@c42f4d80/debugging-with-api-monitor/imgur-api-client</a>. After that, the next step is to write a FFI bindings</a>. It's not hard because we have two small functions to export: #[no_mangle] /// Initializes the Imgur API client /// We return the `mut c_void` pointer because we don't want to expose the internal context structure to users pub unsafe extern "C" fn ImgurInitClient(client_id: const c_char, client_secret: const c_char) -> mut c_void { let client_id = CStr::from_ptr(client_id); let client_secret = CStr::from_ptr(client_secret); let context = ImgurApi::init(client_id.to_str().unwrap(), client_secret.to_str().unwrap()); Box::into_raw(Box::new(context)) as mut _ } /// Retrieves the comment information based on its id #[no_mangle] pub unsafe extern "C" fn ImgurGetComment(context: mut c_void, comment_id: c_ulonglong, comment: mut mut FiiComment) -> u32 { let context: Box<ImgurApi> = Box::from_raw(context as mut _ ); if let Ok(c) = context.comment(comment_id) { comment = Box::into_raw(Box::new(c.into())); 0 } else { 1 } }</code></pre> And of course, the full imgur-api-dll</code> code you can find here: @TheBestTvarynka/trash-code/@c42f4d80/debugging-with-api-monitor/imgur-api-dll</a>. When you build this crate, you'll find the imgur_api.dll</code> file in the target/debug/</code> directory. Good. And the last part is a C++ code that loads our dll and calls its functions. Before it, we need to generate the .h</code> file with our structures and functions. Sure, with such a small example, we can do it manually. But I wanna show you an easier way: I'll use the cbindgen</code></a>. Run the following command in the terminal from the imgur-api-dll</code> crate location: cbindgen --crate imgur-api-dll --output imgur_api.h</code></pre> As a result, you'll get the imgur_api.h</code> file ready for use in the C++ project. I'll need the types of functions, so I add them manually: typedef void (ImgurInitClientFn)(const char, const char); typedef uint32_t (ImgurGetCommentFn)(void*, unsigned long long, FiiComment**);</code></pre> The full imgur_api.h</code> file: @TheBestTvarynka/trash-code/@a333b128/debugging-with-api-monitor/imgur-api-dll/imgur_api.h</a>. Finally, we can write our C++ program. I'm not much of a C++ dev, so don't judge me, please. // code listed below is simplified and some lines are omitted // follow the link under this snippet to read the full src code HMODULE imgur = LoadLibraryA("imgur_api.dll"); FARPROC ImgurInitClientAddress = GetProcAddress(imgur, "ImgurInitClient"); FARPROC ImgurGetCommentAddress = GetProcAddress(imgur, "ImgurGetComment"); const char client_id = "3d8012c2f66acfb"; const char client_secret = "708d779959d043dd6da2d158abaa022931f708a8"; void context = ((ImgurInitClientFn)ImgurInitClientAddress)(client_id, client_secret); unsigned long long comment_id = 1911999579; FiiComment comment = nullptr; uint32_t status = ((ImgurGetCommentFn)ImgurGetCommentAddress)(context, comment_id, &comment); if (!status) { cout << "Success! Comment data:\n"; cout << "id: " << comment->data.id << endl; // some lines are omitted } else { cout << "Error: " << status << endl; }</code></pre> Note: don't worry about credentials in the code. I've already deleted this application from my Imgur account. So everything is fine. Read the full src code here: @TheBestTvarynka/trash-code/@a333b128/debugging-with-api-monitor/TestImgurDll</a>. Basically, our program has three main parts: Initialization: here we obtain module handle and pointers of functions.</li> Comment information retrieving.</li> Print the result of the execution.</li> </ul> Pretty simple, I think. If you run this program, you should get smth like this: Now we have a working program that uses custom external DLL. Perfect. The most interesting and fun part begins in the next section. Writing XML definitions</h2> There are no official guides on how to write such XML definitions. I just explored existing XMLs in the C:\Program Files\rohitab.com\API Monitor\API</code> directory and wrote my own for the imgur_api.dll</code>. Here is my shitty XML: <ApiMonitor> <Include Filename="Headers\windows.h.xml" /> <Module Name="imgur_api.dll" CallingConvention="STDCALL" Category="Imgur"> <Variable Name="FfiCommentData" Type="Struct"> <Field Type="UINT64" Name="id" /> <Field Type="const char" Name="image_id" /> <Field Type="const char" Name="comment" /> <Field Type="const char" Name="author" /> <Field Type="UINT64" Name="author_id" /> <Field Type="char" Name="on_album" /> <Field Type="const char" Name="album_cover" /> <Field Type="UINT32" Name="ups" /> <Field Type="UINT32" Name="downs" /> <Field Type="UINT32" Name="points" /> <Field Type="UINT32" Name="datetime" /> <Field Type="UINT64" Name="parent_id" /> <Field Type="char" Name="deleted" /> <Field Type="char" Name="is_voted" /> <Field Type="char" Name="vote" /> <Field Type="const char" Name="platform" /> <Field Type="char" Name="has_admin_badge" /> <Field Type="UINT64" Name="children" /> <Field Type="UINT32" Name="children_len" /> </Variable> <Variable Name="FfiComment" Type="Struct"> <Field Type="FfiCommentData" Name="data" /> <Field Type="UINT32" Name="status" /> <Field Type="char" Name="success" /> </Variable> <Variable Name="FfiComment*" Type="Pointer" Base="FfiComment" /> <Variable Name="FfiComment**" Type="Pointer" Base="FfiComment" /> <Api Name="ImgurInitClient"> <Param Type="const char" Name="client_id" /> <Param Type="const char" Name="client_secret" /> <Return Type="void" /> </Api> <Api Name="ImgurGetComment"> <Param Type="void*" Name="context" /> <Param Type="unsigned long" Name="comment_id" /> <Param Type="FfiComment**" Name="comment" /> <Return Type="UINT32" /> </Api> </Module> </ApiMonitor></code></pre> Why "shitty"? </blockquote> I don't like XML at all. So, for me, every XML is shitty. The file with the src code: @TheBestTvarynka/trash-code/@a333b128/debugging-with-api-monitor/imgur_api.xml</a>. The code above looks pretty simple and easy to understand, but I'll give you some advice on how not to face problems: If the loaded definitions don't have all functions or don't have any at all, then they are probably invalid and you need to fix the XML. The API Monitor will not show you any message about what does wrong. For example, it'll not tell you that the function uses an unknown param type. It'll just ignore this function.</li> You can split types and variables definitions into .h.xml</code> and .xml</code> files. The idea is obvious: you can include .h.xml</code> files in other API definitions. In such a way you can reduce the code duplication.</li> If you have the defined MyStruct</code> structure, that does not mean that you automatically have the MyStruct</code> and MyStruct</code> pointer types. You should also define pointer types as I did in the code above for the FfiComment</code>.</li> Why did I use the char</code> instead of BOOL</code>? The defined BOOL</code> type has a 4-byte len but I need only one byte:</li> </ul>  <Variable Name="BOOL" Type="Integer" Size="4"> <Enum DefaultName="TRUE"> <Set Name="TRUE" Value="1" /> <Set Name="FALSE" Value="0" /> </Enum> <Success Return="NotEqual" Value="0" /> </Variable></code></pre> So I decided just to use char</code>. It's enough for us. Debugging</h2> How to tell the API Monitor about our new API? </blockquote> Just place the file into the API</code> directory. On the next API Monitor start it'll load all API definitions again, including our new one. For example: C:\Program Files\rohitab.com\API Monitor\API\Imgur\imgur_api.xml</code></pre> If you did everything right, then you should get smth like this: Now let's debug the test application we wrote before. Start it as a usual process monitoring. Here is my result: Now we can fully observe what has been passed into and returned from our functions. Also, we compare the API Monitor values with those printed in the terminal to ensure that we did nothing wrong in the XML definitions. On the screenshot above you can see secrets passed to the init function. Just like I saw passwords and emails during debugging the Windows SSPI 😜. Cool 😎. The values are the same as in the terminal. References & final note</h1> Trace APDU on Windows</a>.</li> Process memory editor, Unions and Arrays</a>.</li> Rustonomicon - Foreign Function Interface</a>.</li> The (unofficial) Rust FFI Guide</a>.</li> </ol> Sometimes our tools are capable of much more than we think. Maybe something you're searching for is just around the corner. Rust Clap recipes 2023-06-15T00:00:00+00:00 I bet you know, but I still wanna recall it. Clap</code></a> is a full-featured, fast Command Line Argument Parser for Rust. docs.rs</a>. crates.io</a>. Goals</h1> Goals: Show non-standard examples of the cli arguments configuration.</li> Tell you about alternate ways to parse cli args with the clap</code>.</li> </ul> Non-goals: Write the "ultimate" guide to the clap</code> library.</li> Create an introduction (or for newbies) article about the clap.</li> </ul> The current article is basically a recipe. It means that here we have described concrete problems, ways to solve them, and examples of their execution. Recipes</h1> Collecting multiple values</h2> Imagine the following situation: you are writing some server. It can be the server-side of some protocol, a proxy server, or something like that. At some point, you decided to add encryption. Now you need to add the ability for the user to pass allowed encryption algorithms (ciphers) for server sessions. Let's try to do it. The first idea that comes to mind is just to use a vector of strings: /// Server config structure #[derive(Parser, Debug)] struct Config { /// Allowed encryption algorithms list #[arg(long, value_name = "ENCRYPTION ALGORITHMS")] pub enc_algs: Vec<String>, }</code></pre> Note 1: personally, I prefer the derive</a> approach to configure clap instead of a builder</a> approach. Note 2: For all examples in this article, I use the same main</code> function: /// Just prints the server configuration. /// `Config` is a previously defined structure (like in the example above). fn main() { println!("{:?}", Config::parse()); }</code></pre> It'll work. Here is an example: ./cool-server --enc-algs aes256 --enc-algs des3 --enc-algs thebesttvarynka # Config { enc_algs: ["aes256", "des3", "thebesttvarynka"] }</code></pre> Good, but here we have a big problem: values are not validated, and the user can specify unsupported or just wrong algorithms. Let's delegate algorithm validation to clap</code>. First, we need to implement the Cipher</code> enum. Most likely, you'll have such an enum already implemented in the project, but here we need to write it: #[derive(Debug, Clone, clap::ValueEnum)] pub enum Cipher { Aes128, Aes256, Des3, } // The next traits implementations are omitted // and left for the reader as a homework 😝 // // impl AsRef<str> for Cipher { ... } // impl Display for Cipher { ... } // impl TryFrom<&str> for Cipher { ... }</code></pre> Pay attention that we also added the clap::ValueEnum</code> derive. It'll generate the ValueEnum</code></a> trait implementation, and clap</code> will be able to parse the raw string into a concrete enum value. Now time to test it: ./cool-server --enc-algs aes256 --enc-algs des3 # Config { enc_algs: [Aes256, Des3] } ./cool-server --enc-algs aes256 --enc-algs des3 --enc-algs thebesttvarynka # error: invalid value 'thebesttvarynka' for '--enc-algs <ENCRYPTION ALGORITHMS>' # [possible values: aes128, aes256, des3] # # For more information, try '--help'.</code></pre> Very cool 🔥. Clap validates the input for us and even tells the user possible values if smth goes wrong. You can read all code of the example above here</a>. In general, we can finish this recipe at this point, but I used to specify multiple values using a comma-separated string. The perfect arg looks to me like this: ./cool-server --enc-algs aes256,des3 # or even ./cool-server --enc-algs aes256:des3</code></pre> "This move will cost us" some additional code writing 🥲. We need to create a wrapper over the Vec<Cipher></code> and implement parsing for it: #[derive(Debug, Clone)] struct EncAlgorithms(pub Vec<Cipher>); // Implementation of this trait is left for the reader. // impl fmt::Display for EncAlgorithms { ... } fn parse_encryption_algorithms(raw_enc_algorithms: &str) -> Result<EncAlgorithms, Error> { let mut parsed_enc_algs = Vec::new(); for raw_enc_alg in raw_enc_algorithms.split(':').filter(|e| !e.is_empty()) { parsed_enc_algs.push(raw_enc_alg.try_into()?); } if parsed_enc_algs.is_empty() { return Err(Error::new(ErrorKind::InvalidValue)); } Ok(EncAlgorithms(parsed_enc_algs)) }</code></pre> And only after this, we can use it in our configuration: /// Server config structure #[derive(Parser, Debug)] struct Config { /// Allowed encryption algorithms list (separated by ':'). #[arg( long, value_name = "ENCRYPTION ALGORITHMS LIST", // https://docs.rs/clap/latest/clap/struct.Arg.html#method.value_parser value_parser = ValueParser::new(parse_encryption_algorithms), default_value_t = EncAlgorithms(vec![Cipher::Aes256, Cipher::Aes128]), )] pub enc_algs: EncAlgorithms, }</code></pre> Why should we create the wrapper? Why is custom parser + Vec<Cipher></code> not enough? </blockquote> Yes, you can write just Vec<Cipher></code> with custom value_parser</code></a> and it'll compile. But it'll fail at runtime with the following error: thread 'main' panicked at 'Mismatch between definition and access of `enc_algs`. Could not downcast to cool_server::cipher::Cipher, need to downcast to alloc::vec::Vec<cool_server::cipher::Cipher> ', src/main.rs:37:19 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace</code></pre> Understandable. </blockquote> Let's try to run the implementation with a wrapper: ./cool-server --enc-algs aes256:des3 # Config { enc_algs: EncAlgorithms([Aes256, Des3]) } ./cool-server --enc-algs aes256:des3:thebesttvarynka # error: invalid value 'aes256:des3:thebesttvarynka' for '--enc-algs <ENCRYPTION ALGORITHMS LIST>': error: invalid value 'Invalid algorithm name: thebesttvarynka' for 'enc-algs' # # For more information, try '--help'. ./cool-server --help # Server config structure # # Usage: cool-server [OPTIONS] # # Options: # --enc-algs <ENCRYPTION ALGORITHMS LIST> # Allowed encryption algorithms list (separated by ':') [default: aes256:aes128] # -h, --help # Print help</code></pre> The full src</a> code of the example above. Okay, but are you sure that we can't do better? I am still thinking about a better solution. </blockquote> IDK 😕. Let's explore the docs more thoroughly... Oh, it turns out it's called the value_delimiter</a>: /// Server config structure #[derive(Parser, Debug)] struct Config { /// Allowed encryption algorithms list (separated by ':' or space). #[arg( long, value_name = "CIPHERS", default_values_t = vec![Cipher::Aes256], value_delimiter = ':', num_args = 1.., // at least one allowed cipher type )] pub enc_algs: Vec<Cipher>, }</code></pre> Good job. It's way better now. </blockquote> Yep. With this approach, we remove all custom parsing and formatting stuff. Moreover, the help message and error reporting became better: ./cool-server --help # Server config structure # # Usage: cool-server [OPTIONS] # # Options: # --enc-algs <CIPHERS>... Allowed encryption algorithms list (separated by ':' or space) [default: aes256] [possible values: aes128, aes256, des3] # -h, --help Print help ./cool-server --enc-algs aes256:des3 # Config { enc_algs: [Aes256, Des3] } ./cool-server --enc-algs aes256:des4 # error: invalid value 'des4' for '--enc-algs <CIPHERS>...' # [possible values: aes128, aes256, des3] # # tip: a similar value exists: 'des3' # # For more information, try '--help'.</code></pre> The full src</a> code of the example above. Commands and arg groups</h2> Before we start this recipe, I wanna recall one thing: the difference between commands and args. First of all, commands don't contain any dashes or slashes in their names. They are just words. Commands specify what to do, whereas args specify how to do it. Example: gcloud auth login --cred-file creds.json # What to do? `login`. # How to do it? By taking a file with creds named `creds.json`.</code></pre> In this recipe, we'll work with commands and args. You'll see a more complex example of the clap</code> configuration. Interesting. What do we need to configure? 😄 </blockquote> We all know the Imgur</a> site (if not, then just visit). It has an API</a>. Let's imagine we decided to write a CLI tool to help us work with the Imgur site using its API. So, now we need to design the tool interface. We do not plan to cover the whole API. Just downloading and uploading. A quick draft configuration: #[derive(Debug, Clone, Subcommand)] enum Command { Upload, Download, } /// Img tool config structure #[derive(Parser, Debug)] struct Config { /// command to execute #[command(subcommand)] pub command: Command, /// Path to the api key file #[arg(long, env = "API-KEY")] pub api_key: PathBuff, }</code></pre> We immediately have a few interesting moments: the Subcommand</a> trait derive and the PathBuff</code> type in the api_key</code> field. We are not forced to use only simple types for args like String</code>s, numbers, etc. If you have a concrete type that describes your value (PathBuff</code></a> for file path, url::Url</code></a> for URLs, or even custom ones), then use this type in the configuration. It'll handle more errors during parsing and make further work easier. Now we add params for downloading the command: #[derive(Debug, Clone, Subcommand)] enum Command { Upload, Download { /// Source image link #[arg(long)] link: Url, // <--- Pay attention to the type. We use the `Url` here and not the `String`. /// Path for the image #[arg(long)] dest_file: PathBuf, } }</code></pre> To download the image, we need only two things: the source image link and the destination file path. This is how it works: ./img-tool download --help # Usage: img-tool --api-key <API_KEY> download --link <LINK> --dest-file <DEST_FILE> # # Options: # --link <LINK> Source image link # --dest-file <DEST_FILE> Path for the image # -h, --help Print help ./img-tool --api-key key.json download --link https://imgur.com/gallery/vNOUshX --dest-file ferris.png # Config { command: Download { link: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("imgur.com")), port: None, path: "/gallery/vNOUshX", query: None, fragment: None }, dest_file: "ferris.png" }, api_key: "key.json" } ./img-tool --api-key key.json download --link :imgur.com/gallery/vNOUshX --dest-file ferris.png # error: invalid value ':imgur.com/gallery/vNOUshX' for '--link <LINK>': relative URL without a base # # For more information, try '--help'.</code></pre> Cool and pretty simple. But the upload is a little bit more complex. We wanna have two options to take the photo to upload: file image on the device or any public URL on the Internet. Here is the configuration for the upload command: #[derive(Debug, Clone, Args)] #[group(required = true, args = ["file", "link"])] /// Possible types of the file source struct FileSource { /// Path to the image on the device #[arg(long)] file: Option<PathBuf>, /// Url to the image on the Internet #[arg(long)] link: Option<Url>, } #[derive(Debug, Clone, Subcommand)] enum Command { Upload { /// File to upload #[command(flatten)] file_source: FileSource, /// Folder for the image on the site #[arg(long)] folder: String, }, Download { / omitted / }, }</code></pre> Okay, we have two file sources: a file or a link. And we use ArgGroup</a> to require the user to specify only one of them: either file or link. And here is the demo: ./img-tool upload --help # Possible types of the file source # # Usage: img-tool --api-key <API_KEY> upload --folder <FOLDER> <--file <FILE>|--link <LINK>> # # Options: # --file <FILE> Path to the image on the device # --link <LINK> Url to the image on the Internet # --folder <FOLDER> Folder for the image on the site # -h, --help Print help ./img-tool --api-key key.json upload --folder ferris --file crab_ferris.png # Config { command: Upload { file_source: FileSource { file: Some("crab_ferris.png"), link: None }, folder: "ferris" }, api_key: "key.json" } ./img-tool --api-key key.json upload --folder ferris --link https://i.imgflip.com/7gq1em.jpg --file crab_ferris.png # error: the argument '--file <FILE>' cannot be used with '--link <LINK>' # # Usage: img-tool --api-key <API_KEY> upload --folder <FOLDER> <--file <FILE>|--link <LINK>> # # For more information, try '--help'. ./img-tool --api-key key.json upload --folder ferris --link https://i.imgflip.com/7gq1em.jpg # Config { command: Upload { file_source: FileSource { file: None, link: Some(Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("i.imgflip.com")), port: None, path: "/7gq1em.jpg", query: None, fragment: None }) }, folder: "ferris" }, api_key: "key.json" }</code></pre> Another interesting thing: we can see the separated --file</code> and --link</code> args in the help message by the <></code> triangle brackets. It shows the user that only one of them is needed. Cool, right? 😎 The full src</a> code of the example above. Parsing args into a custom structure</h2> This recipe will be smaller than the previous ones and similar to the second one. But I just want to show that we can do such tricks. We are going to improve the previous example by adding a more flexible way to pass the API key. Assume that for authentication we need two tokens: app id and app secret. And we want to operate them as one structure. Usually, in such cases, developers treat them as two separate strings and then create a single structure based on those strings. But we are smarter and know how to use commands: #[derive(Debug, Clone, Args)] struct ApiKey { /// app id #[arg(long)] pub api_app_id: String, /// app secret #[arg(long)] pub api_app_secret: String, } /// Img tool config structure #[derive(Parser, Debug)] struct Config { /// command to execute #[command(subcommand)] pub command: Command, /// API key data #[command(flatten)] pub api_key: ApiKey, }</code></pre> Let's test it and see the trick: ./img-tool --help # Img tool config structure # # Usage: img-tool --api-app-id <API_APP_ID> --api-app-secret <API_APP_SECRET> <COMMAND> # # Commands: # upload Upload image to the Imgur # download # help Print this message or the help of the given subcommand(s) # # Options: # --api-app-id <API_APP_ID> app id # --api-app-secret <API_APP_SECRET> app secret # -h, --help Print help</code></pre> We have two separate args api-app-is</code> and api-app-secret</code>, but they will be parsed and placed in the one ApiKey</code> structure. It's very convenient and we can continue to work with the auth data as one structure without any additional actions. The full src</a> code of the example above. Unsolvable problem</h2> This section is not an actual recipe. I'll tell you about a problem that doesn't have a perfect solution yet (or I just cannot find one). Let's take the previous recipe and make the task more difficult. Assume that the user should specify the app id and secret OR API key file. In other words, the user should provide the one --api-key-file</code> arg or --api-app-id</code> and --api-app-secret</code> args. It means one OR two arguments. I found a few similar questions on the Internet: Clap either -a OR (-b AND -c) arguments</a></li> Using clap-derive with two groups of arguments</a></li> </ul> But neither of them had any useful answers. So far, the following code is the best way we can solve this problem: #[derive(Debug, Clone, Args)] struct ApiKeyData { /// app id #[arg(long, requires = "api_app_secret")] pub api_app_id: Option<String>, /// app secret #[arg(long, requires = "api_app_id")] pub api_app_secret: Option<String>, } #[derive(Debug, Clone, Args)] #[group(required = true, args = ["api_key_file", "api_app_id", "api_app_secret"])] /// Possible types of the api key source struct ApiKey { /// Path to the json file with API key #[arg(long)] api_key_file: Option<PathBuf>, /// Specify API key data in args #[command(flatten)] api_key_data: ApiKeyData, } // The `Config` structure remains unchanged since the last recipe</code></pre> The main idea is to create an ArgGroup</code></a> and require additional args in the ApiKeyData</code> structure if one of the needed args is not specified. This approach has a lot of big inconveniences: Fields in the ApiKeyData</code> structure are optional. Yes, during parsing, they are validated, and 100% have values, but for further work, we are forced to unwrap them and create another structure.</li> The help message for the user is not fully informative. It shows that among those three arguments, only one is required. That is a lie because we need a key file OR app id + secret.</li> </ul> Enough talking. Let's see it in action: ./img-tool --help # Img tool config structure # # Usage: img-tool <--api-key-file <API_KEY_FILE>|--api-app-id <API_APP_ID>|--api-app-secret <API_APP_SECRET>> <COMMAND> # # Commands: # upload Upload image to the Imgur # download # help Print this message or the help of the given subcommand(s) # # Options: # --api-key-file <API_KEY_FILE> Path to the json file with API key # --api-app-id <API_APP_ID> app id # --api-app-secret <API_APP_SECRET> app secret # -h, --help Print help ./img-tool --api-key-file key.json download --link https://imgur.com/gallery/vNOUshX --dest-file ferris.png # Config { command: Download { link: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("imgur.com")), port: None, path: "/gallery/vNOUshX", query: None, fragment: None }, dest_file: "ferris.png" }, api_key: ApiKey { api_key_file: Some("key.json"), api_key_data: ApiKeyData { api_app_id: None, api_app_secret: None } } } ./img-tool --api-app-id tbt --api-app-secret secret download --link https://imgur.com/gallery/vNOUshX --dest-file ferris.png # Config { command: Download { link: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("imgur.com")), port: None, path: "/gallery/vNOUshX", query: None, fragment: None }, dest_file: "ferris.png" }, api_key: ApiKey { api_key_file: None, api_key_data: ApiKeyData { api_app_id: Some("tbt"), api_app_secret: Some("secret") } } } ./img-tool --api-app-id tbt download --link https://imgur.com/gallery/vNOUshX --dest-file ferris.png # error: the following required arguments were not provided: # --api-app-secret <API_APP_SECRET> # # Usage: img-tool <--api-key-file <API_KEY_FILE>|--api-app-id <API_APP_ID>|--api-app-secret <API_APP_SECRET>> <COMMAND> # # For more information, try '--help'.</code></pre> The full src</a> code of the example above. Conclusion: this problem is solvable, but in a very inconvenient way. References & final note</h1> Official docs: derive</a> and builder</a> references.</li> </ol> The end. The official reference has all you need. As you can see, the clap</code> gives us a lot of opportunities to configure args parsing and validation. Sometimes it's really worth it to read the docs or references. Advice for the future: try to move as much work as you can to the clap</code>. It's a very powerful tool, so you shouldn't be bothered by manual parsing or validation of the args. SSPI introduction 2023-04-27T00:00:00+00:00 Goals</h1> The main purpose of this article is to tell you about Windows SSPI</a>. If you know nothing about it and want to start working with it, this article will benefit you. If you are familiar with SSPI and have worked with it, then this article most likely will not be useful. Goals: Provide a comprehensive SSPI description.</li> Introduce and explain basic SSPI terms.</li> Understand core SSPI functions, invocation order, and the purpose of each of them.</li> </ul> Non-goals: Explain how concrete SSP packages work (like NTLM</code>, Kerberos</code>, etc).</li> Teach how to write a custom SSP package. _{I mean, after reading this article you will have a brief idea of what you need to write but it's obviously not enough.</li> </ul> Does this article replace reading the documentation? Of course not. It's just a good place to start. Happy reading! 😃}What is SSPI</h1> First of all, let's start with the official SSPI definition</a>: SSPI is a software interface. Distributed programming libraries such as RPC can use it for authenticated communications. One or more software modules provide the actual authentication capabilities... </blockquote> So, now we know that it's an interface. It can be used for authentication. The general idea is to provide a unified interface for the authentication and if any program will need to authenticate some user, then it "just" can use SSPI. Moreover, the authentication process (function call order, etc) doesn't depend on the underlying authentication protocol. But buffer sizes and types can be different. We'll see how we can handle such cases. Basic concepts and definitions</h1> SSP - Security Support Provider</a> - a dynamic library (.dll</code> file) that has one or more SSPI implementations. Security package - one SSPI implementation. Usually, it reflects some authorization protocol. Good, now let's think a little bit. What do we mean under "successful authentication"? Usually, it's something like "username + password is correct, we can do our work further" or "we've got some token that proves our identity/permissions". In SSPI, the process of authenticating is called context establishing. The result of authenticating is the (established) security context with the session key. The session key can be used for further encryption/decryption (you'll see it later), checksum calculation/verification, and so on. It's valid only for this session. After another context establishing the security context will have another session key. The Microsoft documentation gives us a good session context definition</a>: A security context is the set of security attributes and rules in effect during a communication session. </blockquote> A communication session. After successful authentication, the client and the server can continue to send messages to each other. They can use the session key to secure this communication. It means you can still need SSPI even after the authentication process. Okay, sounds pretty simple and reasonable. Time to see the actual SSPI interface. All mandatory functions are listed in one structure: the SSPI function table. Here it is</a>: typedef struct _SECURITY_FUNCTION_TABLE_W { unsigned long dwVersion; ENUMERATE_SECURITY_PACKAGES_FN_W EnumerateSecurityPackagesW; QUERY_CREDENTIALS_ATTRIBUTES_FN_W QueryCredentialsAttributesW; ACQUIRE_CREDENTIALS_HANDLE_FN_W AcquireCredentialsHandleW; FREE_CREDENTIALS_HANDLE_FN FreeCredentialsHandle; void Reserved2; INITIALIZE_SECURITY_CONTEXT_FN_W InitializeSecurityContextW; ACCEPT_SECURITY_CONTEXT_FN AcceptSecurityContext; COMPLETE_AUTH_TOKEN_FN CompleteAuthToken; DELETE_SECURITY_CONTEXT_FN DeleteSecurityContext; APPLY_CONTROL_TOKEN_FN ApplyControlToken; QUERY_CONTEXT_ATTRIBUTES_FN_W QueryContextAttributesW; IMPERSONATE_SECURITY_CONTEXT_FN ImpersonateSecurityContext; REVERT_SECURITY_CONTEXT_FN RevertSecurityContext; MAKE_SIGNATURE_FN MakeSignature; VERIFY_SIGNATURE_FN VerifySignature; FREE_CONTEXT_BUFFER_FN FreeContextBuffer; QUERY_SECURITY_PACKAGE_INFO_FN_W QuerySecurityPackageInfoW; void Reserved3; void Reserved4; EXPORT_SECURITY_CONTEXT_FN ExportSecurityContext; IMPORT_SECURITY_CONTEXT_FN_W ImportSecurityContextW; ADD_CREDENTIALS_FN_W AddCredentialsW; void Reserved8; QUERY_SECURITY_CONTEXT_TOKEN_FN QuerySecurityContextToken; ENCRYPT_MESSAGE_FN EncryptMessage; DECRYPT_MESSAGE_FN DecryptMessage; SET_CONTEXT_ATTRIBUTES_FN_W SetContextAttributesW; SET_CREDENTIALS_ATTRIBUTES_FN_W SetCredentialsAttributesW; CHANGE_PASSWORD_FN_W ChangeAccountPasswordW; void Reserved9; QUERY_CONTEXT_ATTRIBUTES_EX_FN_W QueryContextAttributesExW; QUERY_CREDENTIALS_ATTRIBUTES_EX_FN_W QueryCredentialsAttributesExW; } SecurityFunctionTableW, PSecurityFunctionTableW;</code></pre> Along with table W, also a similar table A</a> exists. What's the difference? This is not a part of this story but here is the answer on SO</a>. Today we'll work only with the W table. But remember that all things that apply to the W table, are also applicable to the A table. SSPI</h1> The SSPI functions table has a lot of _{(?terrible?) functions. We'll talk about them, and how and when you should use them.}Overview</h2> To bring at least some order, we will split SSPI functions into four categories. The official documentation</a> has a good description of it so we follow it: Package management. SSPI package management functions initiate a security package, enumerate available packages, and query the attributes of a security package.</li> </ol> EnumerateSecurityPackagesW</code>: list all available security packages in the SSP.</li> InitSecurityInterfaceW</code>: initialize security table.</li> QuerySecurityPackageInfoW</code>: query some information about the security package.</li> </ul> Credential management. Functions used to obtain credentials handle for the security context, set credentials attributes, and query information about credentials.</li> </ol> AcquireCredentialsHandleW</code></li> ExportSecurityContext</code></li> FreeCredentialsHandle</code></li> ImportSecurityContextW</code></li> QueryCredentialsAttributesW</code></li> </ul> Context management. Functions used to establish security context, query and set its attributes, etc.</li> </ol> AcceptSecurityContext</code></li> ApplyControlToken</code></li> CompleteAuthToken</code></li> DeleteSecurityContext</code></li> FreeContextBuffer</code></li> ImpersonateSecurityContext</code></li> InitializeSecurityContextW</code></li> QuerySecurityContextToken</code></li> QueryContextAttributesW</code></li> SetContextAttributesW</code></li> RevertSecurityContext</code></li> </ul> Message support. Those functions usually are called when the security context is established. You can use them to secure messages (encryption, hashing, etc).</li> </ol> EncryptMessage</code>.</li> DecryptMessage</code>.</li> MakeSignature</code>.</li> MakeSignature</code>.</li> </ul> Enough talking. Let's see SSPI in action. From the example below you will understand functions call order, and their purpose. SSPI in action</h2> Two things are needed to demonstrate SSPI in action: Client.</li> Server.</li> </ol> Both of them should use the same security package to successfully authenticate. As the client, I took the NTLM security package</a> from the Microsoft-provided SSP</a>. To keep my code example clearer, as the server, I took this sspi implementation in Rust</a>. All my code is written in Rust. To call Windows API functions I use crates with bindings like winapi</a>, windows-sys</a>. You should focus only on the client code and not on the server code. Why NTLM? Because it's simple. It's a good protocol to use as an example (but bad for the real world [1]</a> [2]</a>. If you can avoid it then avoid it 🐺). Here</a> you can find the source code. Initialization</h3> Good so far. Let the journey begin! What do we need to start using SSPI? Correct! SSPI function table. How can we initialize it? By calling the InitSecurityInterfaceW</code> function. This function returns a structure (see it above</a>) with function pointers. If we want to use Windows SSP then we can just use Win API. But if we decide to use some custom SSP, then we need to load the corresponding dll, find the InitSecurityInterfaceW</code> function and call it. Example: // load our SSP let sspi_handle = LoadLibraryW("my_cool_ssp.dll"); // get the pointer to the needed function let init_security_interface_fn = GetProcAddress(sspi_handle, "InitSecurityInterfaceW"); // transmute (cast) the pointer to the needed function type let init_security_interface_w_fn: INIT_SECURITY_INTERFACE_W = unsafe { std::mem::transmute(init_security_interface_w_fn) }; let sspi_function_table = init_security_interface_w_fn(); // now we have initialized function table // ...further authentication</code></pre> But in the scope of this article, I use Windows native SSP. So I don't need to initialize the function table. It already is initialized inside Windows. Time to see what Windows can offer us: let mut number_of_packages = 0; let mut packages = null_mut(); // Query information about the available Windows security packages // https://learn.microsoft.com/en-us/windows/win32/api/sspi/nf-sspi-enumeratesecuritypackagesw let result = EnumerateSecurityPackagesW(&mut number_of_packages, &mut packages);</code></pre> When I printed it on the screen (src</a>), I got such the output: Number of packages: 13 ------------------------- fCapabilities: 8928179 wVersion: 1 wRPCID: 9 cbMaxToken: 48256 Name: Negotiate Comment: Microsoft Package Negotiator ------------------------- ...</code></pre> We can see a lot of security packages. But as I said above, today we are working only with NTLM. Let's gather more information about the NTLM security package (src</a>): let mut raw_package_name = str_to_win_wstring("NTLM"); let mut package_info = null_mut(); let status = QuerySecurityPackageInfoW(raw_package_name.as_mut_ptr(), &mut package_info); println!("{} package info:", package_name); println!("fCapabilities: {}", (package_info).fCapabilities); println!("wVersion: {}", (package_info).wVersion); println!("wRPCID: {}", (package_info).wRPCID); println!("cbMaxToken: {}", (package_info).cbMaxToken); println!("Name: {}", c_wide_string_to_rs_string((package_info).Name)); println!( "Comment: {}", c_wide_string_to_rs_string((package_info).Comment) );</code></pre> Output: NTLM package info: fCapabilities: 42478391 wVersion: 1 wRPCID: 10 cbMaxToken: 2888 Name: NTLM Comment: NTLM Security Package</code></pre> In general, different security packages support different attributes. So refer to the documentation for more concrete information. Credentials</h3> Before starting the actual authentication we need to prepare credentials. In other words, to acquire credentials handle. Basically, the credentials handle is a pointer to some structure that contains prepared credentials for use during the authentication, maybe some flags, and other credentials-related information. What type of this structure? We don't know and we don't need to know. The security provider creates and works with this object. It'll be good if you also read the official credentials handle definition</a>. Enough talking. Now time back to the code (src</a>). let mut credentials_handle = CredHandle::default(); let mut expiry = TimeStamp::default(); let mut package_name = str_to_win_wstring("NTLM"); let mut domain = str_to_win_wstring(""); let mut user = str_to_win_wstring("testuser"); let mut password = str_to_win_wstring("test"); let mut identity = SEC_WINNT_AUTH_IDENTITY_W { User: user.as_mut_ptr(), UserLength: 8, Domain: domain.as_mut_ptr(), DomainLength: 0, Password: password.as_mut_ptr(), PasswordLength: 4, Flags: 0x2, }; let status = AcquireCredentialsHandleW( null_mut(), package_name.as_mut_ptr(), // SECPKG_CRED_OUTBOUND: https://learn.microsoft.com/en-us/windows/win32/secauthn/acquirecredentialshandle--ntlm 2, null_mut(), &mut identity as mut SEC_WINNT_AUTH_IDENTITY_W as mut _, None, null_mut(), &mut credentials_handle as mut _, &mut expiry as mut _, );</code></pre> After the AcquireCredentialsHandleW</code> function call, the credentials_handle</code> variable will contain the credentials handle. Note. In SSPI all handles (context, credentials, etc) are instances of the SecHandle</code></a> structure. Each such structure has two numbers (pointers): lower and upper. When you are working with SSPI you don't need to know anything more about them. If you are writing your own SSP then you shouldn't expose any information about them. However, to satisfy your curiosity, I will say what there may be (but don't tell anyone 😜): One of them can contain a pointer to some object (for example, a security context object), and another one can contain a pointer to the security package name. Because SSP can contain a lot of security packages, we need to store the security package name in some way that is currently used. This function has a lot of parameters and they are well explained in the official documentation</a>. Just don't forget that the type for the pauthdata</code> pointer is package-specific. It means that different security packages usually require different types of pauthdata</code>. Also, the SSPI has a place for customization. Security packages implement the QueryCredentialsAttributesW</code> and SetCredentialsAttributesW</code> functions. With them, we can add and get additional information (attributes) about concrete credentials handle. Small example (src</a>): let mut credentials_name = SecPkgCredentials_NamesW::default(); let status = QueryCredentialsAttributesW( client_credentials_handle, SECPKG_CRED_ATTR_NAMES, &mut credentials_name as mut SecPkgCredentials_NamesW as mut _, ); println!( "Credentials name: {:?}", c_wide_string_to_rs_string(credentials_name.sUserName) );</code></pre> Output: Credentials name: "testdomain\\testuser"</code></pre>Authentication</h3> Perfect. We have prepared credentials (credentials handle). Time to start actual authentication. The authentication process consists of sequential InitializeSecurityContext</code> function calls. When we should stop? </blockquote> When this function returns SEC_E_OK</code>, SEC_I_COMPLETE_AND_CONTINUE</code>, or SEC_I_COMPLETE_NEEDED</code> status code. i. e. we call the InitializeSecurityContext</code> function until it returns the appropriate status code. The generalized authentication flow is shown in the diagram below: flowchart TD A(Begin auth) --> B(InitializeSecurityContext) B --> C{status code} C -->|Error| F(Auth failed) C -->|SEC_E_OK,...| E(Auth succeeded) C -->|SEC_I_CONTINUE_NEEDED| D(Send buffers to the server) D -->|TCP| G(Receive buffers from server) G --> B </div> Great, pretty simple. Now let's implement it in the code (src</a>): let client_status = InitializeSecurityContextW( &mut client_credentials_handle, unwrap_sec_handle(&mut client_security_context), target_name.as_mut_ptr(), // MUTUAL_AUTH and ALLOCATE_MEMORY 0x2 | 0x100, 0, // Native data representation: 0x10, &mut client_input_buffers, 0, &mut new_client_security_context, &mut client_output_buffers, &mut context_attributes as mut i32 as mut _, &mut expiry as mut _, );</code></pre> Oh my gosh. So many parameters. We definitely need an explanation. </blockquote> Agreed. But before I explain each of them, I would like to insert small note about the source code you will find in the repo: I didn't create any real servers or TCP/TLS connections. I just convert and pass buffers from one function to another. You will see it in the code comments. Hardest part described below: phcredential</code>: a pointer to the credentials handle. We have it after the AcquireCredentialsHandleW</code> function call.</li> phcontext</code>: a pointer to the security context handle.</li> </ul> But wait. We don't have a such one. </blockquote> Yes. On the first function call we pass the NULL</code> here. The security context handle will be created by the function itself during the first invocation and written in the phnewcontext</code> parameter. psztargetname</code>: "a pointer to a null-terminated string that indicates the service principal name (SPN)" (docs</a>).</li> fcontextreq</code>: a context requirements flags. Basically, this flags tell the security package how to do authentication. Like, what session key to use, what additional things to require, etc. Those flags differ in the different security packages.</li> reserved1</code>, reserved2</code> are reserved and not used.</li> targetdatarep</code>: "The data representation, such as byte ordering, on the target." (docs</a>)</li> pinput</code>: input buffers for this function. On the first function call, we don't have any input buffers so we should pass empty SecBufferDesc</code>.</li> </ul> What types of security buffers should I choose, how many of them do I need, and in what sequence? </blockquote> Docs. Search for it in the documentation of the corresponding security package. If you can't find it there, then search in the open-source projects. And remember one important thing: never change buffers' order or type. phnewcontext</code>: a new security context handle will be written here. You should use this handle for the next function call.</li> poutput</code>: same as pinput</code> but output buffers instead of input.</li> pfcontextattr</code>: flags that describe established security context. We use fcontextreq</code> to specify some options for authentication. Ans use pfcontextattr</code> to see what options have been established (set).</li> ptsexpiry</code>: the expiration time of the context.</li> </ul> The security context also has functions for setting and getting different attributes: QueryContextAttributesW</code> and SetContextAttributesW</code>. They have similar behavior to the credentials-related functions. For example, in the NTLM security package, we have the possibility to extract the established session key (src</a>): let mut session_key = SecPkgContext_SessionKey::default(); let status = QueryContextAttributesW( client_security_context, SECPKG_ATTR_SESSION_KEY, &mut session_key as mut SecPkgContext_SessionKey as mut _, ); println!( "Established session key: {:?}", from_raw_parts( session_key.SessionKey as const u8, session_key.SessionKeyLength as usize ) );</code></pre> Output: Established session key: [96, 1, 13, 58, 82, 191, 222, 134, 149, 184, 3, 75, 254, 126, 225, 74]</code></pre> You can run the code a few times. Each time you will have another session key. Phew, the hardest part is gone. Now we (finally) have the established security context. What's next? Now we can do everything we want, but more importantly, we can safely transfer any messages to the server and receive server messages securely. Communication</h3> Imagine the situation: we want to send a very secret message to the server. In such case we should use the EncryptMessage</code> function: let status = EncryptMessage(client_security_context, 0, &mut message, sequence_number); // send `message` to the server</code></pre> I think you already guessed it, but still: encryption/decryption, and signature generation/verification are always in-place. This is why we don't have the output buffer parameter in this function. If we want to read encrypted messages (e. g. received from the server) then: let status = DecryptMessage(client_security_context, 0, &mut message, sequence_number);</code></pre> The same thing with the MakeSignature</code> and VerifySignature</code> functions. Clean up</h3> Yes, memory leaks are memory safe, but it's better to not forget to clean up everything. SSPI has three functions for that: FreeContextBuffer</code>. (doc</a>). Only one rule: if the buffer was allocated by the security package, then you should free it using this function. If the buffer was allocated by yourself, then you should free it in the usual way.</li> FreeCredentialsHandle</code>. (doc</a>).</li> DeleteSecurityContext</code>. (doc</a>).</li> </ul> // ... let status = FreeCredentialsHandle(client_cred_handle); let status = DeleteSecurityContext(client_context_handle);</code></pre>Conclusion</h1> Finally, this story goes to the end. If you want to use SSPI on the server side then just replace the InitializeSecurityContextW</code> function with AcceptSecurityContext</code>. Wait! You haven't described other SSPI functions! </blockquote> Don't panic, I know it. The other functions are rarely used and do not take a direct part in the authorization process. I hope you enjoy reading it and it'll help you in some way. If you want to share some feedback, ask a question, just task to me, then use this page</a> to contact me. Doc, references, code</h1> Security Support Provider Interface (SSPI).</a></li> SSPI.</a></li> sspi.h</code>.</a></li> Authentication Functions.</a></li> Using SSPI.</a></li> Example source code.</a></li> </ol> yew-notifications 2023-04-17T00:00:00+00:00 Notifications components library for Yew</a>. It's like react-toastify</a> but for Yew</a> and more simpler (so far 😏). TL;DR: documentation</a>. Motivation</h3> I was writing my personal project crypto-helper</a> some time ago. I was forced to write awful code ([1]</a>, [2]</a>) to add some notifications functionality to my web app. So, I decided to write this library that allows the easy showing of notifications to users. Inspired by yew-toastrack</code></a>. Core concepts</h3> First of all, simplicity is one of the main purposes. I would like to have a general provider and hook that will spawn notifications. Something like that: <Provider> // inner components </Provider></code></pre> ... and spawn notifications in any inner component using one simple hook: let notifications = use_notifications(); notifications.spawn(/ /);</code></pre> I managed to implement it as I wanted using the yew ContextProvider</a>. Basically, all notifications are saved in this context provider. But we need to add new notifications, remove old ones, and so on. So the simple Vec</code> inside of the context provider is a bad idea. Actually, it holds the UseReducerDispatcher</code> wrapper into the NotificationsManager</code>. This NotificationsManager</code> can spawn new notifications using the inner UseReducerDispatcher</code> and the context will handle them. The second goal is customization. I don't want to force users to use only built-in notification components. This is a reason why the NotificationsProvider</code> component and use_notifications</code> hook have generic parameters. For the notifications provider component, we need to specify two generic parameters: Notifications type. This is a struct that described notification data like title, text, lifetime, etc. You can create your own such struct and use it here. It must implement the Notifiable</code></a> trait.</li> Notification Factory type. The purpose of this type is actual notification component creation. It must implement the NotifiableComponentFactory</code></a> trait. When the library will try to render the user's notification, it'll use this factory to create notification yew component from the Notification instance.</li> </ol> <NotificationsProvider<Notification, NotificationFactory> component_creator={/ /}> // some inner components </NotificationsProvider<Notification, NotificationFactory>></code></pre> For the use_notifications</code> hook, we need to specify only the notification type: let notifications = use_notification::<Notification>(); notifications.spawn(Notification::new(/ /));</code></pre> Pay attention: if you specify the different notification types in the provider and hook then the code will compile but fail in the runtime. But you can use different notification types if they provider components do not overlap. For example, with fully customized notifications, we can achieve the following result: src</a>. demo site</a>. How to use it</h3> Decide which notification components to use. yew-notifications</code> already has implemented standard notifications but you can write your own. See basic</code></a> and custom</code></a> examples for more information.</li> </ol> # Cargo.toml # if you want to use standard notification components yew-notifications = { git = "https://github.com/TheBestTvarynka/yew-notifications.git", features = ["standard-notification"] } # if you decide to write and use custom notification components yew-notifications = { git = "https://github.com/TheBestTvarynka/yew-notifications.git" }</code></pre> Include yew-notification</code> styles into your project:</li> </ol> <link data-trunk rel="sass" href="https://raw.githubusercontent.com/TheBestTvarynka/yew-notifications/main/static/notifications_provider.scss" /></code></pre> This one below is needed only when you decide to use components from the yew-notifications</code>: <link data-trunk rel="sass" href="https://raw.githubusercontent.com/TheBestTvarynka/yew-notifications/main/static/notification.scss" /></code></pre> Or you can copy scss file into your project and specify the path to it instead of the link. At this point, the scss file is one possible way to import styles. Wrap needed components into NotificationProvider</code>:</li> </ol> // Notification, NotificationFactory - your notification types. // They can be imported from this library or written by yourself (see `custom` example). // component_creator is an instance of the NotificationFactory <NotificationsProvider<Notification, NotificationFactory> component_creator={/ */}> // some inner components </NotificationsProvider<Notification, NotificationFactory>></code></pre> Spawn notifications:</li> </ol> use yew_notifications::use_notification; // Notification - your notification type. // It can be imported from this library or written by yourself (see `custom` example). let notifications_manager = use_notification::<Notification>(); notifications_manager.spawn(Notification::new(...));</code></pre> See the examples</code></a> directory for the code examples. Moving forward</h3> At this point, this library has minimal functionality implemented. I plan to improve it continuously according to my needs and goals. If you have any feature requests, then create an issue</a> with the description. It'll be a priority for me. bytes-formatter 2023-04-09T00:00:00+00:00 Visit this tool at bf.qkation.com</a>. Motivation</h3> During my work, I often need to convert bytes between different representations (format) like from hex to decimal and vice versa. From hex/base64 to ASCII. And so on. Before I just have one small rust project on my laptop with a few dependencies and every time I inserted my new data, run this project, and copy the output. After some time it became inconvenient: I was forced to keep this project on every machine, compile it after every code change, etc. After some suffering, I wrote this user-friendly tool in pure html/css/js. The code is clear-cut and perfectly does the job. Features: supported formats: decimal</code>, hex</code>, base64</code> (both: usual and url-encoded), ascii</code>, binary</code>, utf-8</code>, and utf-16</code>.</li> ability to copy only selected range of the bytes. It automatically counts the amount of the selected bytes.</li> share by the link.</li> integrated asn1 parser</a> (available for hex</code> and base64</code> output types).</li> </ul> How to use it</h3> Just follow the link</a> and paste your data. The interface is pretty intuitive. Anyway, it also has examples of the different inputs. Click the Show input examples</code> button to see them. You'll see that you can insert even unfiltered data and this tool will do basic filtering and formatting. Moving forward</h3> I already implemented the needed functionality for me. If you found the bug or have the feature request, then create a new issue</a> with the proper description.

TheBestTvarynka

Drawing Genealogy Graphs. Part 1: Tree Drawing Using Reingold-Tilford Algorithm

First walk</h2> As I wrote above, during the first walk, we determine preX</code>, mod</code>, and shift</code> values for each node in the tree. Let me explain what these parameters mean.</p>

Announcing crypto-helper v.0.16.0

ASN1 major features</h1> ASN1 tree editing</h2> </p>

ASN1 tree editing</h2> </p>

How RDP smart card logon works

Announcing Dataans v.0.3.0

Demo</h1> Before explaining how it works and how I implemented it, I want to show you the demo.</p> Better to see something once than hear about it a thousand times.</em></p> </blockquote>

How it works</h1> Now let's talk about interesting things: how it works 😊. Below are detailed technical explanations. You can jump right to the interesting part. There is no need to read it in order.</p>

Infrastructure</h2> This section is boring, because I don't like to describe infrastructure in detail. So, if you are not interested in such things, then skip it 😉.</p>

Safety Comments Matter

Implementing Kerberos RPC encryption over SSPI

Intro</h1> Today's world has many unknown and magical things. Proprietary protocols and libraries are among them 🤪. In this blog post, I will explain how RPC encryption works under the hood and how to implement it over the SSPI interface.</p>

Tauri error handling recipes

Let's handle the error</h1> Usually, most of the commands may fail. We may want to return a Result</code> and handle the error on the frontend side. Let's try it. Suppose we need to validate the name. I came up with the following implementation (let's keep it simple):</p>

Learned lessons</h1> All Error</code>s must implement the serde::Serialize</code> trait.</li> Tauri command error (Result::Err(E)</code>) is an exception on frontend. The exception can be caught by using the catch</code> attribute of the #[wasm_bindgen]</code> macro.</li>

Announcing Dataans v.0.1.0

Dataans</h1> The Dataans is a desktop app that allows you to take notes in the form of markdown snippets grouped into spaces. Yes, it's another note-taking app, but it has unique features that I miss in all other note-taking apps.</p>

Features</h2> </p> Quake (drop-down) mode. The keybinding can be configured.</li> Cross-platform.</li>

dataans

Solution</h1> These are the two closest apps to what I want:</p>

Features</h2> </p> Quake (drop-down) mode. The keybinding can be configured.</li> Cross-platform.</li>

crypto-helper

Writing unsafe Rust code? Please, consider Miri

Getting started</h1> First of all, you need to install Miri:</p>

Who should use Miri</h1> In my opinion, everyone who writes an unsafe</em> Rust code should use Miri and test their code with this tool. It can be any kind of unsafe</em> code:</p>

Configuration</h1> Miri can be configured by -Z</code> flags and environment variables. And yes, we don't have any .toml</code> file here. For example:</p>

Git recipes

Goals</h2> Explain how to resolve common git</code> tasks I face at work every time.</li> Improve your git</code> knowledge.</li>

Recipes</h1> I want to leave a few clarifications before I write recipes.</p>

Discard last commit</h2> I know that it's a bad idea, but if you really want, then this is how you can discard</strong> the last commit:</p>

Untrack file</h2> If you want git</code> to ignore some file, then just add it to the .gitignore</code> file. For example:</p>

Conclusion</h1> There are no any smart</em> conclusions. But I have stupid ones 😁:</p>

Smart card container name

Before I start</h1>

The first run</h2> Let's try to sign the data using the emulated smart card.</p> </p> Expectedly (otherwise, this article would not exist 🙂), we got an exception. Our goal for the rest of the article is to figure out the cause of the error and fix it.</p>

How it works</h1>
Now let's talk about interesting things: how it works 😊. Below are detailed technical explanations. You can jump right to the interesting part. There is no need to read it in order.</p>

Intro</h1>
Today's world has many unknown and magical things. Proprietary protocols and libraries are among them 🤪. In this blog post, I will explain how RPC encryption works under the hood and how to implement it over the SSPI interface.</p>

Dataans</h1>
The Dataans is a desktop app that allows you to take notes in the form of markdown snippets grouped into spaces. Yes, it's another note-taking app, but it has unique features that I miss in all other note-taking apps.</p>

The first run</h2>
Let's try to sign the data using the emulated smart card.</p>
</p>
Expectedly (otherwise, this article would not exist 🙂), we got an exception. Our goal for the rest of the article is to figure out the cause of the error and fix it.</p>