| ↑↑ Home | ↑ Net & Web |
As other operations based on public-key systems, S/MIME does not directly encrypt the message with the public key of the recipient. Rather, the message is encrypted with a symmetric encryption method, and the randomly generated message encryption key is encrypted with the public key of the recipient and included in the mail. This has two main advantages: Public key operations are computationally expensive, so having to encrypt and decrypt only the message key with the public key is more efficient. And the message may have multiple recipients and should also be readable to the sender from the Fcc folder, so encrypting only the message key with the required public keys saves space compared to multiple encrypted versions of the whole message which would otherwise be necessary.
When the private key needed for decryption is located on a smart card, it gets more complicated still. The smart card may contain multiple public-private key pairs that share a key ID. But the mail message contains only the serial number of the public key. So a two-stage search is required: First the keys on the card are searched for a match of one of the public keys from the mail message (remember: multiple keys for sender + recipient(s)). Then the ID of the matching public key is read, and the card is searched for the private key with the same ID. To my knowledge there is no command-line program that performs this double search, so decrypting such a message on the command line requires several manual steps and/or some custom coding.
The following software packages will be required or at least helpful:
will be needed for several purposes will usually be required to access your smart card at all contains the
command-line program pkcs11-tool and the card access tracing library
pkcs11-spy.so contains the eponymous
command-line program (and depends on its library package libp11-kit) contains the command-line program
p11tool; it depends on the p11-kit library
Save the S/MIME encrypted mail to a file, msg.mail. It should have the content type (possibly with an additional name= attribute):
Content-Type: application/pkcs7-mime; smime-type=enveloped-data
Otherwise it may not be an encrypted mail, or differ in other ways from the one I have investigated, and the following may not apply to it. (The procedure for analysing it may still be of use.)
First we list the structure of the "enveloped" message. This requires the preprocessing steps:
openssl smime -in msg.mail -pk7out -out msg.pem openssl pkcs7 -inform pem -outform der -in msg.pem -out msg.der
The abstract structure of an S/MIME mail is defined by
ASN.1 (the X.580 standard).
The first of the openssl commands above generates a file in PEM format,
which is a base64-encoded version of the binary DER format, which we obtain
with the second openssl command.
Simply base64-decoding the mail body also results in a DER-format file, but in
my experience it can have some peculiarities that our subsequent commands do
not like. The openssl smime command tolerantly decodes the S/MIME contents
and combines sections of the encrypted message if they are stored in several
blocks, which will be useful later.
Now we can list the internal structure of the DER file. The more widely available way of doing that is with openssl:
openssl asn1parse -inform der -in msg.der
The output looks somewhat like this:
0:d=0 hl=4 l=3626 cons: SEQUENCE
4:d=1 hl=2 l= 9 prim: OBJECT :pkcs7-envelopedData
15:d=1 hl=4 l=3611 cons: cont [ 0 ]
19:d=2 hl=4 l=3607 cons: SEQUENCE
23:d=3 hl=2 l= 1 prim: INTEGER :00
26:d=3 hl=4 l=1326 cons: SET
30:d=4 hl=4 l= 438 cons: SEQUENCE
34:d=5 hl=2 l= 1 prim: INTEGER :00
37:d=5 hl=3 l= 157 cons: SEQUENCE
40:d=6 hl=3 l= 140 cons: SEQUENCE
43:d=7 hl=2 l= 11 cons: SET
45:d=8 hl=2 l= 9 cons: SEQUENCE
47:d=9 hl=2 l= 3 prim: OBJECT :countryName
52:d=9 hl=2 l= 2 prim: PRINTABLESTRING :DE
56:d=7 hl=2 l= 15 cons: SET
58:d=8 hl=2 l= 13 cons: SEQUENCE
60:d=9 hl=2 l= 3 prim: OBJECT :stateOrProvinceName
[... localityName, organizationName, organizationalUnitName, commonName ...]
183:d=6 hl=2 l= 12 prim: INTEGER :B7A699164CABAFE0826D9B31
197:d=5 hl=2 l= 13 cons: SEQUENCE
199:d=6 hl=2 l= 9 prim: OBJECT :rsaEncryption
210:d=6 hl=2 l= 0 prim: NULL
212:d=5 hl=4 l= 256 prim: OCTET STRING [HEX DUMP]:...
472:d=4 hl=4 l= 438 cons: SEQUENCE
[...]
1356:d=3 hl=4 l=2270 cons: SEQUENCE
1360:d=4 hl=2 l= 9 prim: OBJECT :pkcs7-data
1371:d=4 hl=2 l= 29 cons: SEQUENCE
1373:d=5 hl=2 l= 9 prim: OBJECT :aes-256-cbc
1384:d=5 hl=2 l= 16 prim: OCTET STRING [HEX DUMP]:CEEB7B64723F5366A1629C91402BC488
1402:d=4 hl=4 l=2224 prim: cont [ 0 ]
The first number is the byte offset in the DER file. The three variables are the structure nesting level d, the header length hl and the content length l. Adding the offset, the header length and the content length together gets you the offset of the following item, skipping all deeper nesting levels. On the right after the colon is the type or value of an item (if any).
A nicer output with indentation is provided by the
dumpasn1 program,
though it omits the header lengths and prints some cryptic ASN.1 number tuples.
It should be run from the same directory as its configuration file. I learned
about it
from
this answer, which also deals with S/MIME, but not with smart cards.
Going back to the listing, the SEQUENCE at offset 30 contains the symmetric message key encrypted with a recipient's public key and associated information. The sub-SEQUENCE at offset 37 contains the issuer and serial number of the certificate key used for encrypting the message key. In this context public keys come with a certificate because an authority certifies that the public key belongs to a certain person (unlike in PGP where there is no such authority). The certificate issuer is given by the strings labeled with "countryName", "stateOrProvinceName", "localityName", "organizationName", "organizationalUnitName" and "commonName" and wrapped into the SEQUENCE at offset 40. The certificate serial number is the INTEGER at offset 183, B7A69916....
After the certificate issuer and serial number, at offset 197 the encrypted message key follows. The label "rsaEncryption" indicates the encryption method, and the data at offset 212 are the ciphertext (i.e. the public-key encrypted message key, not the symmetrically encrypted message). Actually, the ciphertext to decrypt is the content of the OCTET STRING item not including the header, which starts at offset 216 and is 256 bytes long.
After the end of the SEQUENCE from offset 30 that contains all the above, there is another SEQUENCE of the same length. I have not included it in the example listing, but it has exactly the same structure. It contains the certificate issuer and serial number and the encrypted message key for a different recipient of the mail message (or for the sender, so they can read their own Fcc). A recipient has to search these blocks for a key to which they have the corresponding private key. If a smart card wasn't involved, the recipient could use openssl pkeyutl (formerly rsautl) to decrypt the message key with their private key. How to decrypt it with a smart card is described below.
After the encrypted message key blocks, at offset 1356 the message ciphertext and associated information follows. The label aes-256-cbc gives the encryption method used, and the octet string following it is the initial value that this encryption method needs. When you have the symmetric message key, you can decrypt the message with:
openssl enc -d -aes-256-cbc -K <symmetric key> -iv <initial value> -in <message ciphertext> -out <output file>
<symmetric key> and <initial value> are given as strings of hexadecimal digits; <message ciphertext> and <output file> are file names.
To process data from the DER file further, you will have to extract parts of it. This can be done with one of the following equivalent commands:
tail -c +<offset+1> < msg.der | head -c <length> > <filename> dd if=msg.der bs=1 skip=<offset> count=<length> of=<filename>
It is worth noting that openssl asn1parse also works on PEM files (which are base64-encoded DER files). But the offsets in its output always refer to the DER file, and to extract the data you need the DER format in any case.
The details in this section, and smart card stuff in general, may depend on your type of card and card reader. I will describe what worked for me.
First, the pcscd daemon has to be running to access a cryptographic smart card. Install pcsclite and start the daemon. It is not necessary for a pcscd process to run permanently. It is usually started on demand when a program tries to access a cryptographic token via the library libpcsclite.so.
Libraries distributed with commercial smart cards usually link to some version of libpcsclite.so. If you have such a library, you have to give its path on the following command lines. I will call it vendor.so.
One tool for accessing smart cards is pkcs11-tool from the opensc package. The most basic command to try is listing the available card slots:
pkcs11-tool -L
If this fails, your card reader is probably not supported. If a slot is found, but your smart card requires a vendor library, it will output "token not recognized" after the card slot. With the vendor library, information about the smart card is printed, and you can use -T instead of -L to omit output about unoccupied slots:
pkcs11-tool --module /path/to/vendor.so -T
Next we can list the keys on the smart card with one of the following commands:
pkcs11-tool --module /path/to/vendor.so -O pkcs11-tool --module /path/to/vendor.so -O -l
The -l option logs in to the smart card after prompting for your password. How much information is displayed with or without login will depend on your card. My card allows listing only the public keys and certificates without a login, but with -l the private keys are also displayed.
If these commands work as expected, you should be good regarding smart card access.
As an alternative to pkcs11-tool, p11tool from gnutls or p11-kit from the eponymous package can be used to test card access. Both depend on p11-kit and require configuration of your vendor library in a file in ~/.config/pkcs11/modules/ or /etc/pkcs11/pkcs11.conf. The format is described in the manual page pkcs11.conf (5), but this single line should suffice:
module: /path/to/vendor.so
Then your vendor library and smart card should show up in the list of modules (and tokens):
p11-kit list-modules
With p11tool you can list the tokens with:
p11tool --list-tokens
Using the URL from the list of tokens, you can list the public and private keys with:
p11tool --list-all <URL> p11tool --login --list-all-privkeys <URL>
As explained above, we have to find the key ID matching the certificate serial number. As there seems to be no standard tool that helps with that, I have written this small program. It doesn't do much, but it lists the serial numbers and IDs of all objects stored on the card. Its only mandatory argument is the path to the vendor library:
sclist /path/to/vendor.so sclist -l /path/to/vendor.so
The -l option lets you log in to the card and may display a few more objects. Search for a certificate object with one of the serial numbers with which the message key was encrypted. (grep -C 1 is a great help.) Note the ID of the matching certificate, as this is also the ID of the private key required to decrypt the message key.
To decrypt the symmetric message key, you first have to extract it to a file, say msgkey.cipher, as described above. Obviously you have to choose the encrypted message key from the same ASN.1 structure that contained your certificate serial number. Then you can decrypt it with your card using pkcs11-tool:
pkcs11-tool --module /path/to/vendor.so -l --decrypt --id <keyID> -i msgkey.cipher -o msgkey.clear
The <keyID> has to be given as a hexadecimal string without separators (which should be documented but isn't). The parsing function for such arguments is apparently meant to support colon separators but doesn't due to a bug.
With a hexdump (again without separators) of the binary message key from msgkey.clear you can then decrypt the message itself as described above.
When using some tools (and also when coding your own) it can be useful to know which version of the PKCS11 standard your vendor library supports. As the different versions have a slightly different list of API functions, this can be determined by listing the symbols defining them, or counting them:
nm -D /path/to/vendor.so | grep " T C_" nm -D /path/to/vendor.so | grep " T C_" | wc -l
nm lists symbols,
i.e. labels, in binary executables and libraries. The -D options lists
the symbols in a shared library. The letter T in the output indicates symbols
in the text section of the object file, where executable code is located. And
including "C_" in the grep pattern catches all PKCS11 functions, which all
start with that prefix.
For comparison, download the header files pkcs11f.h from the include file directories of different versions of the PKCS11 standard (links below). Then you can get a short list of the API functions with:
grep CK_PKCS11_FUNCTION_INFO pkcs11f.h
PKCS11 V3.0 has 92 API functions, older versions have around 70 or so (I have encountered 68 and 72 functions, possibly accounted for by different 2.x versions). If you want to compare the lists one on one, piping into | sort | nl may help.
If you have several versions of your vendor library, it may be that they report the properties of the smart card slightly differently. I have encountered one library giving the model of the same card as a number, and another as a name. This may affect you if you use p11tool or another program using p11-kit and you have to specify the card.
The opensc package contains the library pkcs11-spy.so that acts as a man in the middle and detailed function call logger for another PKCS11 library. It is very useful if you have a program that can access your smart card with your vendor library but you cannot get it to work with other software. The application in question must allow choosing an arbitrary PKCS11 library for accessing the card.
You generate a log as follows: First it is advisable to run the
script program
to save the copious output to a log file. Then set the PKCS11SPY
environment variable to the vendor library path. Then start the application
that can access your card from the same terminal and tell it to use
/usr/lib/pkcs11-spy.so (or the correct path on your system) as the card
access library. As you use the application, plenty of output should be
generated about the PKCS11 functions called and their arguments and return
values. After closing the application and exiting from the shell, the log file
will be closed by script.
Caution: If you log in to the card, the dump will contain your password among the parameters to the C_Login function! Sanitise the dump if you want to pass it on to ask for advice!
A less important caveat is that pkcs11-spy.so seems to work only with libraries that support the same PKCS11 standard version as your opensc package. When I tried to use it with a mismatching vendor library, smart card operations simply did not work, probably because pkcs11-spy.so could not pass through function calls that it (or the vendor lib) was not aware of. A simple way to find out which PKCS11 version your pkcs11-spy.so supports is to count the PKCS11 API function symbols as described above for the vendor library.
Except for the PKCS11 standard documents, I did not find them overly helpful and have hardly read them to debug my problem. But if you need them, here are direct links to the relevant standards I could find. Not all of them are on the open web any more, so some point to the Internet Archive Wayback Machine.
= PKCS #7 for smart card access and
header
file directory for smart card access and
header
file directory linking to the standards documents
and related resources