Ahhhh, it’s already included right? Due to the wifi version of pico? I’ve seen it during cmake but didn’t make the connection. I’ll start there and put together a basic command to decode from the internal storage.
1 Like
Do you actually need the X.509 and certificate handling in the bus pirate firmware itself?
Yes, it would be nice if the firmware could tell “yes, that signature is genuine”. But I wouldn’t say that this is really required and justifies longer development work and complicating the build dependencies and their management long-time.
The firmware could instead dump all the relevant data in some given text format, like PEM, JSON, XML whatever. You then copy this output from the commandline into some web form on check.buspirate.com and get back a yes or no.
I would say doing such a cert check on a full webserver environment with all the libraries readily available and so on is trivial compared to doing it on an embedded system.
Also provisioning the certificate during production doesn’t need full X.509 handling in the firmware. You just need code to read out the serial of the RP2350 and hand that out to the programming jig control software running on a PC. From there the certificate request is either locally handled or, better, handed off to some server that has the private key. The signed X.509 is then given back to the PC software and from there programmed byte-by-byte into OTP. So the BP firmware wouldn’t need to understand what it is writing into OTP.
Or is there some use case that I don’t see yet where you would need each BP to have it’s own private key stored in OTP that would be used to sign some messages or similar? Since it would have to be in OTP all firmwares could read out such a key. So I’m a bit in doubt about the usefulness of such a feature.
3 Likes
I agree it shouldn’t need to encode the cert, at least as I understand the process.
The ultimate goal for any part of the bus pirate is to be educational, with sufficient docs to understand why and how decisions were made and how to apply that to your own work. In this case it would be a demo of the tool chain, and docs about how we put it together plus working code base.
I’m really clueless on this, but am super interested.
2 Likes
No, we don’t need anything. To be fair, x509 validation using a public key in firmware like the BP is pretty trivial. There are libraries for this (as mentioned earlier). I’ve done it in bootloaders as part of secure boot chain of trust. For a certificate like this, there’s no need to go “outside” for verification, as long as you’ve hardcoded the public key(s), it’s just a basic call to the crypto library.
Totally agree - you just need to generate a CSR with the appropriate data. That can be done by a manufacturing computer.
That was my intention; I didn’t spell it out well enough.
You’re exactly right - there’s no point in putting a private key in OTP; it’s not private. A very common use of x509 is to share the public half of a keypair for secure comms as you said, but that’s not the only use. We don’t even need to generate a public/private pair for each BP.
The certificate in question here is just used to store serial numbers (for both BP and RP2350 chip), maybe hardware revision, and to identify Dangerous Prototypes in the subject. Signing that certificate with a private key that DP holds ensures that the information in the cert is authenticated as coming from DP and hasn’t been tampered with.
Maybe the question should be: “why did I start out with x509 certs right away?”
- Using a cert in this way is a pretty standard solution to have a “birth certificate” for a device that can be cryptographically validated
- x509 is a well established standard, there are mature libraries for handling it
- Homebrew roll-your-own solutions usually have some hole in them
- The PKI (public key infrastructure) needed for signing is also an established thing, and the problem has been solved in many ways. It just depends on if you’re willing to put in the effort and costs to do so.
And yes; this is coming from someone who most recently worked at an ICS vendor where this kind of thing was critically important. Does that mean I’m overly complicating things? One could make that argument, I’m sure. What I can say is that this method has been examined, tested, beaten up, and certified as secure.
Again, is it necessary for the BP? That was never my call; I think it would be cool to have in an open source product, but it doesn’t mean I wouldn’t use it without that feature.
My intention was to spur good, interesting conversation about this, and that’s exactly what we’re having 
The feedback and questions are important and good stuff.
3 Likes
In my experience bigger external libraries are a pain to deal with. Not short term, you integrate it once and it works. But the pain comes when you maintain your software over several years and the external library changes it’s functions, the old version doesn’t compile anymore with a new version of your compiler, there are name clashes with some other library you need and so on.
I don’t say you should not use libraries and instead write all your code yourself, this would be impractical. But what I want to get across is that adding an additional library comes with a hidden cost. And this cost has to be paid not by the one sending the pull request, but by the one that is maintaining the software long term.
So I think it should be carefully judged if such a validation mechanism in firmware justifies an additional dependency, when this extra feature is not critical to the main purpose of the bus pirate and could instead be easily implemented in some web form.
The feature would be fully working, it would be a nice showcase of how to do such things and there would be no hindrance to implement an in-firmware check at a later time.
I didn’t ask this question because I guessed your reasoning. Using established standards and tools has it’s merits, even if you don’t need all of their features. So I fully concur. Thanks for spelling it out explicitly.
2 Likes
I agree. I’ve been caught in this, and have had to spend many, many hours maintaining and fixing things I didn’t break (but broke because of a library change). But in this case… it looks like mbedTLS is already included in the pico_sdk. I didn’t realize that until @ian noted it above. That pushes a lot of that work upstream to rPI.
No worries - I just realized I hadn’t been specific about it, and that not everyone would understand my thinking
2 Likes
I’m not so sure if that really lessens the pain of maintaining. RasPi will make sure it compiles with their stuff, sure. But the functions for X.509 validation could still change from one version to the other. Since they intend the mbedTLS to be primarily used indirectly through their Wifi stack, they could even replace it alltogether with some other TLS library that they then integrate with their Wifi stack. Then you are stuck on one version of the pico sdk because of this.
Maybe I’m crying wolf with this, but I’ve been bitten by dependencies often enough. So I’d say it is @ian s call if he wants to integrate the in-firmware signature check now or not.
3 Likes
One more thing that’s critical:
Defining a starting location in the OTP where we define a list of the “other stuff” that was also programmed into the OTP.
NOTE: END OF LIST IS DEFINED AS AN ALL-ZERO EntryType. Preferably, an entire OTP_DIRECTORY_ITEM would all be zero.
Variable-length records … So something such as:
typedef struct _OTP_DIRECTORY_ITEM {
uint16_t EntryType; // with 0x0000 defined as "end of list"
uint16_t StartRow; // row where that entry is stored
uint16_t RowCount; // count of consecutive rows
uint16_t CRC16; // Validates the prior three entries.
} OTP_DIRECTORY_ITEM;
Importantly, this will allow for later changes to what is programmed in, while allowing the firmware to search for (and use) an entry, or detect that the entry does not appear to have been programmed.
It also prevents issues with partially-programmed entries, if the directory is written only after the entry is successfully written.
I’m reserving EntryType of 0x0000 for end of list (functionally required to be this value, to allow appending items later), and reserving 0x0001 for usb_whitelabel usage.
3 Likes
Good idea.
How about adding a CRC32 to this?
This would allow the firmware to check if one _OTP_DIRECTORY_ITEM is valid as defined by this structure. Some other firmware or the user could have manually added some values that could confuse the firmware. With a checksum this becomes quite unlikely. The checksum would be programmed last and make the entry valid.
2 Likes
Your point about maintainability is absolutely valid. As far as I can tell, MBEDlts is part of the PICO SDK the same way that TinyUSB is. I’m looking into it now.
As a cert isn’t at all critical to the system, more an educational/tutorial exercise, if the library causes issues in the future we can just eliminate the dependency by removing the command.
I read up on x509, SSL, certs, libraries, etc last night for fun, and think it would be cool to know a bit more and take a stab at implementing something.
1 Like
I really like the idea of a directory for OTP; that helps in a lot of ways to make things more reliable. The only question is about the end of list - since this is OTP, we wouldn’t be able to modify that later if something new was added after end of list was marked.
One other thing might be the first “Bus Pirate” row to be a version identifier. It’s likely that things could change later; even with the OTP_DIRECTORY_ENTRY and the firmware needs to know that before processing any data.
Absolutely! This is an excellent point - this is educational/tutorial. A developer could learn about all of this from the BP and apply it to their own product if they needed.
1 Like
Pushed a cert command to a new branch that includes mbedtls, parses a PEM cert/key and verifies it. It compiles, it runs, however you may notice a glaring issue:
const char *public_key_pem = "-----BEGIN PRIVATE KEY-----\n\
MIIJQgIBADANBgkqhkiG9w0BAQEFAASCCSwwggkoAgEAAoICAQCmpTSnVyvhIyEJ\n\
EhUPmjxdOewyT3cmZE1X7LcnEqGYoFa2KUtj9b6z+h2zfgsqbIt14Zv/eDK3/SW9\n\
mmp820kRx5cE2bUzNXVz343nBia16eO/qn1DEmfGFKTkt48yhWvzFM4CScWOlLDv\n\
I generated a cert (yeah!) and a private key (oops), but no idea how to get the public key. Rough learning curve on this for me, and so much is focused on HTTPS/SSL that I get kind of lost.
openssl req -x509 -newkey rsa:4096 -nodes \
-out ./nginx/config/cert.pem \
-keyout ./nginx/config/key.pem -days 365
Generated the pair with openssl on WSL. I’m sure it’s dead obvious but I need a break
Some notable terms I (think I) have learned:
- x509 - A format for storing info in a certificate. Default values are name. location, organization, etc
- PEM - A base64 encoding scheme for keys and certs. Begins with
-----BEGIN CERTIFICATE-----\n, has explicit new line characters, and ends with-----END CERTIFICATE-----. Substitute CERTIFICATE for PUBLIC KEY or PRIVATE KEY - DER - Binary encoding scheme for keys and certs (what we’re after)
openssl rsa -in key.pem -pubout > mykey.pub
The public key is generated from the private key.
Now we can verify Getting the data out is for another day.
1 Like
OpenSSL can be such a beast to use…
I’ll do a quick example of how I would create a keypair, create a certificate signing request, create the self-signed cert, and convert that cert to a .der. (there are many ways to skin this cat, this is just one example). The assumption for simplicity is that there are no intermediate signings.
Create the keypair
$ openssl genrsa -out ./private.pem 2048
This generates the public and private keypair, but outputs just the private key, in this case private.pem:
$ cat private.pem
-----BEGIN PRIVATE KEY-----
MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDOOgFKWGsvwYii
<< Many many lines of Base64 >>
6d4hBl0JoCKdCJX31nhoLmgs
-----END PRIVATE KEY-----
If you want the public key, you can get it with:
$ openssl rsa -in ./private.pem -pubout > ./public.pem
writing RSA key
$ cat ./public.pem
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAzjoBSlhrL8GIopcOfB1L
3AjrIFjPo0KXFZW9eFuz8jZj8Af4DxmYecxhvQH2tJdFfIjkCvd1O+6W2Ykniqvp
qJvG+X32nuOanRQyJ5foRqwKUL6+anNRMdNYygZbLZ6+LKpsr62c7mwPvngOICnw
+Ad1WrcTSt2gXqRuJ2ZkuF1APcISaVKc8WWUHacQtP/8HXw8TeMh/uIFT0qVvPrF
L2JTlsz+DRHbXLHr2505jVytERpaAEEf7XHR19jIY1rfJV+AggZ8uT2ZPcxMPUiV
ojQlcARPuvmlye2tbaXngQJpnTxID07SF4pZbPfOuca1cx6bM3HHwCQqiKdKojS1
tQIDAQAB
-----END PUBLIC KEY-----
You can always redirect that output to a file.
That’s the public key that would be in the firmware and used to validate/authenticate the certificate.
Generate the CSR
The certificate signing request will have all of the info in the certificate, and can be created with OpenSSL:
$ openssl req -new -key ./private.pem -out ./sample.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:US
State or Province Name (full name) [Some-State]:WI
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Mattydyne Heavy Industries
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:
Email Address []:test@example.com
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
Signing the certificate from the CSR and private key
$ openssl x509 -req -days 3650 -in ./sample.csr -signkey ./private.pem -out sample.pem
Certificate request self-signature ok
subject=C = US, ST = WI, O = Mattydyne Heavy Industries, emailAddress = test@example.com
Now we have a keypair, csr, and certificate!
$ ls -l
total 20
-rw------- 1 matty matty 1704 Jan 29 09:30 private.pem
-rw-rw-r-- 1 matty matty 451 Jan 29 09:30 public.pem
-rw-rw-r-- 1 matty matty 993 Jan 29 09:35 sample.csr
-rw-rw-r-- 1 matty matty 1196 Jan 29 09:36 sample.pem
Peek at that cert
We can take a quick look at the certificate with OpenSSL:
$ openssl x509 -in sample.pem --inform PEM --text
Certificate:
Data:
Version: 1 (0x0)
Serial Number:
0c:cb:02:34:b5:af:89:b4:42:25:0f:e8:38:ab:83:c4:58:a1:ae:d4
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = US, ST = WI, O = Mattydyne Heavy Industries, emailAddress = test@example.com
Validity
Not Before: Jan 29 15:36:52 2025 GMT
Not After : Jan 27 15:36:52 2035 GMT
Subject: C = US, ST = WI, O = Mattydyne Heavy Industries, emailAddress = test@example.com
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:ce:3a:01:4a:58:6b:2f:c1:88:a2:97:0e:7c:1d:
<< MANY LINES OF HEX >>
b5:73:1e:9b:33:71:c7:c0:24:2a:88:a7:4a:a2:34:
b5:b5
Exponent: 65537 (0x10001)
Signature Algorithm: sha256WithRSAEncryption
Signature Value:
97:e2:92:42:bc:16:d8:36:7b:5e:4c:c4:3c:11:c1:ec:2f:2d:
<< MANY LINES OF HEX >>
c4:b1:63:f6
-----BEGIN CERTIFICATE-----
MIIDRzCCAi8CFAzLAjS1r4m0QiUP6Dirg8RYoa7UMA0GCSqGSIb3DQEBCwUAMGAx
<< MANY LINES OF BASE64 >>
Y1oCPkf92elzmndAsh0pwmF+vhF3NuLEsWP2
-----END CERTIFICATE-----
We can see the stuff I entered in the CSR
Converting that cert from PEM to DER
Once again, OpenSSL:
$ openssl x509 -in ./sample.pem --inform PEM --outform DER --out ./sample.der
All of the files:
$ ll
total 20
-rw------- 1 matty matty 1704 Jan 29 09:30 private.pem
-rw-rw-r-- 1 matty matty 451 Jan 29 09:50 public.pem
-rw-rw-r-- 1 matty matty 993 Jan 29 09:35 sample.csr
-rw-rw-r-- 1 matty matty 843 Jan 29 09:54 sample.der
-rw-rw-r-- 1 matty matty 1196 Jan 29 09:36 sample.pem
That’s the real basic version, but it’ll get you what you need for coding and testing
Darn it @ian , you’re finding it all before I can even respond 
1 Like
I kept referring to x509 extensions in my earlier posts. TL;DR version is that everything in the cert has an “OID”, and extensions are just things with specific OIDs.
Custom OIDs for “extensions” are usually under 1.3.6.1.4.1.___
Kind of like IP port numbers ,there’s a registry; but in practice just use some big number. To add them to the CSR, create a config file (I did in vim, it’s called bp.cnf):
$ cat bp.cnf
[req]
req_extensions = v3_req
[v3_req]
1.3.6.1.4.1.555555=ASN1:UTF8String:"BP serial 1234567"
1.3.6.1.4.1.555556=ASN1:UTF8String:"rPI serial a1b2c3d4999"
Then, the command to generate the cert has some added stuff:
$ openssl x509 -req -days 3650 -in ./sample.csr -signkey ./private.pem -out ./sample.pem -extfile bp.cnf -extensions v3_req
Certificate request self-signature ok
subject=C = US, ST = WI, O = Mattydyne Heavy Industries, emailAddress = test@example.com
When you look at the cert now:
$ openssl x509 -in ./sample.pem --text
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
68:a4:a1:76:af:71:5d:b0:68:7f:17:93:f2:6a:63:53:8c:45:42
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = US, ST = WI, O = Mattydyne Heavy Industries, emailAddress = test@example.com
Validity
Not Before: Jan 29 16:14:11 2025 GMT
Not After : Jan 27 16:14:11 2035 GMT
Subject: C = US, ST = WI, O = Mattydyne Heavy Industries, emailAddress = test@example.com
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:ce:3a:01:4a:58:6b:2f:c1:88:a2:97:0e:7c:1d:
<< CLIPPED OUT STUFF >>
b5:b5
Exponent: 65537 (0x10001)
X509v3 extensions:
1.3.6.1.4.1.555555:
..BP serial 1234567
1.3.6.1.4.1.555556:
..rPI serial a1b2c3d4999
X509v3 Subject Key Identifier:
FE:7A:8A:86:95:27:0E:AF:36:9E:5E:25:19:8A:0E:7C:24:65:AF:71
Signature Algorithm: sha256WithRSAEncryption
Signature Value:
If you look down, you see X509v3 extensions:, and there’s our custom data.
Check the library, there should be methods for extracting arbitrary OIDs, or by extension.
1 Like
Thank you for both of those posts, that solidifies a lot of little bits I picked up today. It is very helpful.
1 Like
You’re very welcome. It’s funny that you’re moving ahead faster than I can document, haha
1 Like
You are right; It is important to be able to add to the list. I clarified what the “end of list” record looks like. It’s simply having four rows of OTP that are all-zero (ECC encoded) … meaning that they haven’t been written … meaning that you can add another entry.
[[EDIT: See also error-handling in next post … end-of-directory also defined better there.]]
2 Likes
For where to store the OTP directory:
I recommend starting at the last non-reserved OTP rows (0xF7C … 0xF7F), and walking backwards. Why? Simple: It avoids having to guess how much space to reserve at the start for a growing list. Plus, there’s no extra data access costs in storing the data in reverse.
Note: The above functionality will require a few items:
- Helper function – An iterator that returns the next record, modeled after file system directory enumeration APIs.
void ResetOtpIterator(); // explicitly start overbool GetNextOtpEntry(OTP_DIRECTORY_ITEM* out_direntry); // true when there's a next valid entry- Note: could define an
OTP_HANDLEto allow multiple sources to iterate. However, likely just keep distinct state for each core … keeps the API simpler.
- Helper function – Data retrieval from record. Returns the OTP data into the caller-supplied buffer. Handles all the edge cases / buffer overflow checks / etc. for the caller.
bool GetOtpItemData(const OTP_DIRECTORY_ITEM* direntry, void* buffer, size_t buffer_len); // retrieve the data represented by the OTP_DIRECTORY_ITEM into user-provided buffer- Note: User could also setup this structure manually, to have the API handle the edge cases and such to read non-listed data.
- Error handling – If ECC error for a row … can we ignore that directory entry? Yes.
- Error handling – If no ECC errors, but CRC16 does not match, can we ignore that directory entry? No. Maybe show a warning, but writing the wrong data is fatal for OTP. Thus, any incorrect CRC16 (without ECC error in that record) is the end of the list.
- Reserve
EntryTypevalues for predefined (by RPi) OTP areas. Do not actually store those in the OTP directory, but include them as results from thebool GetNextOtpEntry()API. (akameta-entry) - Define
OTP_DIR_TYPEso value encodes if stored with ECC or as raw data (use one bit). - Define
OTP_DIR_TYPEso value encodes if entry is a meta-entry (which are read-only, and not actually in the list of entries). - Helper function - get next entry of a requested type.
bool find_otp_entry(OTP_DIR_TYPE type, OTP_DIRECTORY_ITEM* out_direntry);
2 Likes