Moving funds, signing messages, and interacting with cryptocurrency networks is possible only after securely generating and storing cryptographic secret keys. Reference cryptocurrency software commonly stores keys in a recognizably encoded string like spesk1XU..., while traditional cryptographic hardware canonically uses the much older PEM encoding format to export key material. In this post we share how and why we built our key encoder to derive usable Tezos Public (and Secret) Keys from traditional Hardware Security Modules.

Key Generation

Behind every operation in the tezos network is a signature made by an elliptic curve secret key. These keys can be easily generated with the reference tezos-client through the gen keys command, which can then be nicely output for an end user like so:

Generating Keys

In this case, we’ve chosen the hardened Secp256k1 elliptic curve used in Bitcoin, randomly generated a new secret key on our local machine and encoded three types of keys with the recognizable base58 check encoding prefixes defined for tezos here. We can now easily use these local keys with the tezos baker, endorser, accuser and client to interact with our blockchain. Success!

Our key is now stored and available1 on a single machine, but being a proof-of-stake network where a large amount of value might be required to be stored online and “staked”, can we do better and keep the secret key fully isolated from this machine to keep our staking operations out of the blockchain graveyard?

Enter hardware security modules or HSMs. HSM’s are special purpose hardware built for the sole purpose of generating, storing, signing and controlling access to valuable cryptographic keys. These traditional cryptographic devices are however unfamiliar with tezos (and vice versa) and expect traditionally formatted PEM keys like so:

-----BEGIN PUBLIC KEY-----
...
-----END PUBLIC KEY-----

To bridge the gap between these exported PEM keys and reference tezos (or other cryptocurrency) software, we must derive the above recognizable tezos formats. Here’s how that’s done.

PEM Key Files

All HSM’s support the extraction of public key material but only some support exporting secret key material2. Both are typically exported in a PEM file. During a secure initialization ceremony we extract both and durably backup our secret material in (ice) cold storage. To obtain our tz... formatted keys, we’ll need to parse the contents of these files.

Note: Once this ceremony is complete, our keys are never again extractable through internet connected devices, thereby maximally reducing the risk that any online risk can ever gain access to our secret keys.

Once a key is exported, if it is human readable you’ll find yourself looking at one of two kinds of PEM formatted files, the more common PKCS#8 format [RFC5208] beginning with BEGIN PRIVATE KEY or the EC standard [RFC5915] beginning with BEGIN EC PRIVATE KEY. Both contain the same contents. Within these markers you’ll find the Base64 encoded contents of a DER encoded data structure.

openssl ecparam -genkey -name secp256k1 -outform pem

-----BEGIN EC PARAMETERS-----
BgUrgQQACg==
-----END EC PARAMETERS-----
-----BEGIN EC PRIVATE KEY-----
MHQCAQEEIClsdFnu77xDixNb5tXj1M/poZbFKEfxnpISZTNvk67toAcGBSuBBAAK
oUQDQgAEwzrAZMqby7rOYC5epIAq4G3sduPERNwD374Wo4Ask4cnBGMRmxMwxbzc
RKZg9gdokat+vwJrF+NJ5kuG5YVfrw==
-----END EC PRIVATE KEY-----

EC Encoded PEM File

openssl ecparam -genkey -name secp256k1 -outform pem |  openssl pkcs8 -topk8 -nocrypt

-----BEGIN PRIVATE KEY-----
MIGEAgEAMBAGByqGSM49AgEGBSuBBAAKBG0wawIBAQQgKWx0We7vvEOLE1vm1ePU
z+mhlsUoR/GekhJlM2+Tru2hRANCAATDOsBkypvLus5gLl6kgCrgbex248RE3APf
vhajgCyThycEYxGbEzDFvNxEpmD2B2iRq36/AmsX40nmS4blhV+v
-----END PRIVATE KEY-----

PKCS#8 Encoded PEM File

If your HSM instead exports a binary file, you likely have the raw DER encoded data which you can convert back to an ASCII formatted PEM file using openssl again like so:

cat raw_file | openssl pkcs8 -topk8 -nocrypt -inform der -outform pem
Converting from DER to PEM Formats

Converting from DER to PEM Formats

The DER encoded bytes in these files contain a data structure with a few fields that contain the values we’re looking for: our raw, unencoded private and public keys along with their curve identifiers. In order to pack these parameters together in one file, the Abstract Syntax Notation (ASN.1) structure is used. Depending on the format we’ve detected of the key in this file (PKCS#8 or EC), we can apply one of two ASN.1 data structures and extract our goods.

Key Extraction

In the older PKCS#8 format, our EC key contents will be packed into the PrivateKey field which we can first extract through the PKCS8 ASN.1 structure. We can then apply the appropriate EC ASN.1 structure directly to these bytes and retrieve our curve parameters as done here. We now have our raw EC key params.

// ecPrivateKey is an ASN.1 encoded EC key
type ecPrivateKey struct {
    Version       int
    PrivateKey    []byte
    NamedCurveOID asn1.ObjectIdentifier `asn1:"optional,explicit,tag:0"`
    PublicKey     asn1.BitString        `asn1:"optional,explicit,tag:1"`
}

// publicKey is an ASN.1 encoded Subject Public Key Info
type subjectPublicKeyInfo struct {
    Algorithm pkix.AlgorithmIdentifier
    PublicKey asn1.BitString
}

Public and Private Key Extraction [source]

Key Encoding

Now that we’ve extracted the raw elliptic curve keys and parameters, we can encode these keys into a format that the reference tezos utilities expect. This is only strictly necessary for our public keys, but exporting properly encoded secret keys provides an extra layer of protection if keys must later be used while HSM’s are offline or unavailable.

Tezos, like many newer cryptocurrencies reuses Bitcoin’s Base58 Check Encoding for turning binary key material into human readable strings. The rationale behind this is summed up best in Bitcoin’s source code:

Why base-58 instead of standard base-64 encoding?

  • Don’t want 0OIl characters that look the same in some fonts and could be used to create visually identical looking data.
  • A string with non-alphanumeric characters is not as easily accepted as input.
  • E-mail usually won’t line-break if there’s no punctuation to break at.
  • Double-clicking selects the whole string as one word if it’s all alphanumeric.

For readability each type of key (Secret, Public and Public Key Hash) has its own familiar prefix like tz1, tz2 or tz3 to distinguish the key’s contents. These prefix bytes are prepended to the binary key material before being base58 check encoded. To provide some anonymity to keys, Public Keys are additionally one-way hashed into a Public Key Hash using the Blake2b algorithm, a former contestant to become the new SHA-3 standard.

/* Public Key Hashes */
tzEd25519PublicKeyHash   = "06a19f" // tz1
tzSecp256k1PublicKeyhash = "06a1a1" // tz2
tzP256PublicKeyHash      = "06a1a4" // tz3

Tezos Prefix Bytes

The results of our encoding are output with our key-encoder as follows. We can see from the Public Key Hash prefix characters that this was encoded as expected as a Secp256k1 public key.

# Install 
go install gitlab.com/unit410/key-encoder

# Encode Keys
key-encoder keys/secp256k1.pem
> Curve:  Secp256k1: 1.3.132.0.10
> Tezos Secret Key:  spsk2rBBj5a6ahir2xZwbNkdeBuyZTxQZC9Pr6UvSAM4GNPeXfM3ix
> Tezos Public Key:  sppk7c9QAGWCJEvFWp6vGBs3VuxFax7GDwWQiPXR2rGSYPN7NMQN9rP
> Tezos Public Key Hash:  tz2PH72CdqfsBJRsDcGfUu3UvuXwEyzqzs3s

Putting Keys To Work

Now that we’ve derived a Public Key and Public Key Hash from a securely generated HSM PEM file, we can easily interact with these public keys using the reference tezos client software. Sending funds to them or monitoring on the blockchain. All the while our secret key material remains safe inside our HSM, limiting the maximum plausible exposure of our funds.

Messages can then be signed by these keys using our Tezos HSM Signer which we’ll discuss in a later post.

Looking forward, we’re excited both to continue bridging the gap between traditional cryptographic tools and cryptocurrency and lowering the barriers to securely interacting with online key material. Maturing multiparty generation/signature algorithms and the inclusion of modern curves in traditional cryptographic tooling will continue to advance the state of the art.

Footnotes

  1. The key is encrypted at rest so a disk snapshot would not be able to easily recover the key, but an attacker or insider (best assumed indistinguishable) on the box could export the raw key from memory. ↩

  2. Disasters happen. If your HSM doesn’t support some limited set of secret key extraction, think long and hard about whether or not you’re comfortable depositing funds on the device. ↩