Proper Bus Pirate side interface, receiving requests and issuing responses.
Python client sending status request, getting list of modes.
A very rough half-implementation of the Rust I2C transaction interface is done. buspirate-hal/src/lib.rs at main · robjwells/buspirate-hal · GitHub
No error handling yet, and the actual transfer method is stubbed out, but the logic for I2C transactions is there. (And really it looks more complicated than it really is due to the requirement on the Rust side for same-operation type coalescing.)
It was pretty straightforward once I got my head round working with flatbuffers. I’m looking forward to working more on this!
1 Like
A successful query for modes, and a second query to change into a mode.
root_type?
@robjwells and I have been working a bit in the live chat and ran into an issue we don’t know how to resolve properly.
union RequestPacketContents { StatusRequest, DataRequest}
table RequestPacket {
version_major:uint8=0;
version_minor:uint8=1;
contents:RequestPacketContents;
}
union ResponsePacketContents { StatusResponse, DataResponse}
table ResponsePacket{
version_major:uint8=0;
version_minor:uint8=1;
contents:ResponsePacketContents;
}
root_type ResponsePacket;
I split the packet into a response and request packet, but now we don’t know how to define the root packet. It apparently has consequences for the rust tooling, so I assume other consequences as well.
Option 1 is to go back to a single Packet that can carry both the Request and Response in the same union. This leaves gaps in the array of functions I use to handle requests on the Bus Pirate side. On the other hand we can have a handler for all the Response packets that returns an error (host should send request, not response).
union RequestPacketContents { StatusRequest, DataRequest}
union ResponsePacketContents { StatusResponse, DataResponse}
table Packet{
version_major:uint8=0;
version_minor:uint8=1;
request:RequestPacketContents;
response:ResponsePacketContents;
}
root_type Packet;
Another option is a single packet with individual unions for request and response. Unpopulated fields are not sent.
Explicitly define the type
**StatusRequest.AddConfigurationType(builder, ModeConfiguration.ModeConfiguration.I2CConfig)**
StatusRequest.AddConfiguration(builder, i2c_config)
I note that (using python tooling) if I don’t explicitly add the (configuration) type, then the type is always 0. Not sure if this is normal, or if I’m doing something wrong.
Status Tables
It’s getting to the point where we need some sort of pattern for using the tables.
For the status table I envision:
Host Status Requests
- Get list of modes
- Get/Set and configure current mode
- Get hardware firmware versions
- Get all voltage readings
- Get all pin states (and set aux pins?, maybe belongs in data request)
- Get/Set pull-resistor state
- Set LED colors
- Get/Set PSU voltage and current
Currently the Bus Pirate response is to include everything. This presents everything to the user, but results in very large packets. For example you probably only need the list of
enum StatusRequestOptions {GetModes, SetMode, GetADC, GetVersions, GetPullup, SetPullup}
table StatusRequest{
//vector of request options
options:[StatusRequestOptions]
//below are the options for a specific SET action (set mode + config)
name:string; // Mode enter, none for mode query.
configuration:ModeConfiguration;
}
So is something like this reasonable? options is a vector of enums (not sure this is possible, may need to be ubyte), which specify what status information to GET and SET, and the configuration info for the SET options?
This feels a bit shaky to me, but making individual tables for every possible action also seems overkill and not the way to go either.
ETA:
Probably the status table is only for requesting status info, and we should have a separate config table for configuring the hardware? Then each piece of hardware has a configuration table of its own? This seems cleaner to me.
In the FlatBuffers tutorial (only, as far as I can tell) the language is that the union type field “must” be filled, presumably the implementation is not actually enforcing that? I’ll see what the Rust generated codes does later. Presumably on the receiving side if this is missing the request can be rejected as malformed?
Edit: tutorial language:
When serializing a union, you must explicitly set this type field, along with providing the union value.
1 Like
This is long so I’ll wrap it in expandovision:
StatusRequests
enum StatusRequestTypes:byte{All, Version, Mode, Pullup, PSU, ADC, IO, Disk};
table StatusRequest{
query:[StatusRequestTypes]; // List of status queries to perform.
}
Status request query is a vector (array) of data queries, this feels a lot like webQL.
// returns the status queries requested in StatusRequest
// if query is empty, then all queries are performed
table StatusResponse {
hardware_version_major:uint8; //HW version
hardware_version_minor:uint8; //HW revision
firmware_version_major:uint8;//FW version
firmware_version_minor:uint8; //FW revision
firmware_git_hash:string; //Git hash of the firmware.
firmware_date:string; //Date of the firmware build.
modes_available:[string]; // List of modes available on the device.
mode_current:string; // Current mode name.
pullup_enabled:bool; // Pull-up resistors enabled.
psu_enabled:bool; // Power supply enabled.
psu_set_voltage_mv:uint32; // Power supply voltage in millivolts.
psu_set_current_ma:uint32; // Power supply current in milliamps.
psu_measured_mv:uint32; // Measured power supply voltage in millivolts.
psu_measured_ma:uint32; // Maximum power supply current in milliamps.
adc_mv:[uint32]; // ADC values in millivolts.
io_direction:[bool]; // IO pin directions (true for output, false for input).
io_value:[bool]; // IO pin values (true for high, false for low).
disk_size_mb:uint32; // Size of the disk in megabytes.
disk_free_mb:uint32; // Free space on the disk in megabytes.
error:string; // Error message if any.
}
The response contains all or some of these fields, depending on the status query.
ConfigurationRequests
table I2CConfig {
speed_khz:uint32; // Speed in kHz.
}
union ModeConfiguration { I2CConfig,}
table ModeConfig {
name:string; // Name of the mode.
configuration:ModeConfiguration;
}
table PullupConfig {
enabled:bool; // Enable or disable pull-up resistors.
}
table PSUConfig {
enabled:bool; // Enable or disable power supply.
set_voltage_mv:uint32; // Set voltage in millivolts.
set_current_ma:uint32; // Set current in milliamps.
}
table IOConfig {
direction:[bool]; // IO pin directions (true for output, false for input).
value:[bool]; // IO pin values (true for high, false for low).
}
table LEDconfig {
color:[uint32]; // LED colors in RGB format (0xRRGGBB).
}
union ConfigurationRequestContents {ModeConfig, PullupConfig, PSUConfig, IOConfig, LEDconfig}
table ConfigurationRequest {
config:[ConfigurationRequestContents]; // List of configuration requests.
}
Configuration Requests actually update things on the Bus Pirate. This was my first stab: each item is a table, and we send a vector of tables that config different things. This is going to lead to a huge amount of code bloat and general complexity though.
table I2CConfig {
speed_khz:uint32; // Speed in kHz.
}
union ModeConfiguration { I2CConfig,}
table AlternateConfigurationRequest {
mode:string; // Name of the mode to configure.
mode_configuration:ModeConfiguration; // Configuration for the mode.
pullup_enabled:bool; // Enable or disable pull-up resistors.
psu_enabled:bool; // Enable or disable power supply.
psu_set_voltage_mv:uint32; // Set voltage in millivolts.
psu_set_current_ma:uint32; // Set current in milliamps.
io_direction:[bool]; // IO pin directions (true for output, false for input).
io_value:[bool]; // IO pin values (true for high, false for low).
led_color:[uint32]; // LED colors in RGB format (0xRRGGBB).
}
In this version everything is in one flat table, except the mode config which will all be different and need their own tables. This generates a lot less code (both flatfile .h code, and userland shuffling things around), and we can just test for the presence of a field and act accordingly.
table ConfigurationResponse{
error:string; // Error message if any.
}
The response is simply an error code, if any.
DataRequests
table DataRequest {
dstart:bool=true; // Start condition.
dstart_alt:bool=false; // Alternate start condition.
daddr:ubyte; // Device address (Bus Pirate automatically will set read/write bit)
ddata:[ubyte]; // Data to write
dreadbytes:uint32; // Number of bytes to read.
dstop:bool=true; // Stop condition.
dstop_alt:bool=false; // Alternate stop condition.
}
table DataResponse {
ddata:[ubyte]; // Data read from device
derror_message:string; // Error message if any.
}
I think we can get by with a single datarequest packet. It follows the syntax available in all modes (2 starts, 2 stops, data out, data in). The i2c address is there for convenience, but won’t be used for other modes.
The d prefix is because start and stop are reserved else were in flat buffers.
Single unified wrapper packet
union RequestPacketContents { StatusRequest, ConfigurationRequest, DataRequest}
union ResponsePacketContents { StatusResponse, ConfigurationResponse, DataResponse}
table Packet{
version_major:uint8=0;
version_minor:uint8=1;
request:RequestPacketContents;
response:ResponsePacketContents;
}
root_type Packet;
Finally, a single unified wrapper packet for requests and responses.
interesting! Thank you for finding that. It’s weird because the C tooling seems to do it automatically, which python is happy to consume and can identify the type no problem, but packets the other way are always type 0 causing the Bus Pirate to reject them.
table ModeConfig {
speed_khz:uint32; // Speed in kHz.
baud:uint32;
stop_bits:uint8;
cpol:bool;
etc etc etc
}
This might also be over kill. Maybe we only need one long mode config table with all the options for every mode. Just add new ones to the end of the table as need. Less tidy, but way less tooling and work to configure the packet.
I don’t know what the “best” option would be, but I can suggest a possible option: separate root_type for each direction. Also split up table definitions into separate files for better maintainability (and the ability to set up root_types).
// FILE: status_request.fbs
table StatusRequest {
// ...
}
root_type StatusRequest;
// FILE: data_request.fbs defined similarly
// FILE: status_response.fbs defined similarly
// FILE: data_response.fbs defined similarly
// FILE: request_packet.fbs
// This is used for packets sent from the host to the Bus Pirate
include "status_request.fbs"
include "data_request.fbs"
union RequestPacketContents { StatusRequest, DataRequest}
table RequestPacket {
version_major:uint8=0;
version_minor:uint8=1;
contents:RequestPacketContents;
}
root_type RequestPacket;
// FILE: response_packet.fbs
// This is used for packets sent from the Bus Pirate to the host
include "status_response.fbs"
include "data_response.fbs"
union ResponsePacketContents { StatusResponse, DataResponse}
table ResponsePacket{
version_major:uint8=0;
version_minor:uint8=1;
contents:ResponsePacketContents;
}
root_type ResponsePacket;
Then run flatc -c *.fbs or similar, which will generate a bunch of *_generated.h files. Use RequestPacket and ResponsePacket at the appropriate ends for sending and receiving.
1 Like
To answer the question in your code: ack would only reflect a single unknown byte, unless we return more info. I was thinking that instead of a dedicated ack field, which is not used elsewhere and doesn’t really provide useful info on reads either, we can return an error message with more specific info: “write byte 5 NACK”.
In this way the software is a thin client and the bus pirate firmware provides detailed debug information (instead of codes that need to be looked up in software and printed by each dev).
I realize this doesn’t really satisfy the needs of a fully automated program (eg don’t parse error text, it might change), but I think it’s a good first approach and when the system is a bit more built out we can look at error codes as well.
It’s not a big deal. I only noted it because in embedded_hal’s I2C error system there’s NoAcknowledgeSource, giving information about whether the nack was after the address (device missing) or data (malformed command). But I know that not every implementation does (or can) provide that information (the third case in the enum is Unknown, and IIRC in my MCP2221 HAL data nacks aren’t distinguishable from any other error so they don’t even get to the level of NoAcknowledgeSource::Unknown).
One other comment from the code: should there be a data transfer limit (data sent, or requested number of bytes to be read)? I’m just wondering if there is a size beyond which it becomes painful on the Bus Pirate side. (For comparison, the MCP2221 has a limit on I2C transfer length of 65,535 bytes, which is done is 60-byte chunks.)
1 Like
I did change it to uint16. There will be a limit. It’s going to be hardware specific and mode specific.
BP5 normal config: 1000 to 2000 bytes per packet
BP5 if the “big buffer” is allocated to bpio: up to 128K
BP6: as Much as we can allocate, 256K?
BP7R2: 8megs
1 Like
I tried this a bit, but I don’t quite understand how to implement it. Don’t both directions still need to be able to decode one packet type and encode the other?
Major progress on the StatusRequest side. It now returns a fair amount of info. More still be connected/added.
- Add pin name labels
- Fix units
- Report ADC, etc
One thing that’s annoying in python is that there is no way to tell if a field set or just ‘0’ value. I guess it’s not too important at this point, but it means developers need to pay attention to the query parameters.
Bus Pirate terminal gives debug info. I will add a debug option to enable verbose output for software developers.
Status request needs some more work, but I’m going to move on to ConfigurationRequest.
1 Like
Yes, they do, but they can be treated separately.
Host builds RequestPacket and send to Bus Pirate.
Bus Pirate uses Verify and accesses fields within the verified RequestPacket.
Bus Pirate builds ResponsePacket and sends to host.
Host uses Verify and accesses fields within the verified ResponsePacket.
Both sides need to #include request_packet_generated.h and #include response_packet_generated.h in order to build (one type) and process (the other type).
I’ve been using flatc, so I don’t know how the autogenerated code differs compared to flatcc.
Edited to add: my understanding is that root_type refers to the root table per .fbs file, so we will need one file request_packet.fbs with its root_type RequestPacket and another file response_packet.fbs with its root_type ResponsePacket. Splitting up the other tables into their own .fbs files is optional but likely will be easier to maintain.
1 Like
Sent 2 bytes (length header): 3800
Sent 56 bytes (flatbuffer data)
Total bytes sent: 58
Response length header: 5000 (value: 80)
Received 80 bytes
ContentsType: 3
Data read: [255 255 255 255 255 255 255 255 255 255]
Data request error: Test error
Configuration request and Data Requests are working (well, the request is handled and a response given, still lots to hook up internally).
Today huge progress was made. It’s clear how I want to initially construct the v0.0.1 flatbuffer tables. Tomorrow I’ll do the “final” draft table. Then it’s time to start tearing up old code and organizing everything nicely.
It’s possible we can read an I2C EEPROM tomorrow, but definitely by Monday 
Next steps:
- Finalize the initial flatbuffer tables
- Connect Bus Pirate side to handler functions, add error checking, consistent debug output
- Create a Python wrapper for the flatbuffer wrapper With consistent debug output, etc.
2 Likes
Great progress today, the basic framework is all there.
StatusResponse:
Hardware version: 5 REV10
Firmware version: 0.0
Firmware git hash: unknown
Firmware date: Jul 20 2025 16:04:26
Available modes: HiZ, 1WIRE, UART, HDUART, I2C, SPI, 2WIRE, 3WIRE, DIO, LED, INFRARED, JTAG
Current mode: I2C
Pin labels: ON, SDA, SCL, , , , , , , GND
Number of LEDs: 18
Pull-up resistors enabled: True
Power supply enabled: True
PSU set voltage: 3299 mV
PSU set current: 300 mA
PSU measured voltage: 3314 mV
PSU measured current: 3 mA
PSU over current error: No
IO ADC values (mV): 3290, 3280, 3277, 3285, 3306, 3300, 3293, 3293
IO directions: IO0:IN, IO1:IN, IO2:IN, IO3:IN, IO4:IN, IO5:IN, IO6:IN, IO7:IN
IO values: IO0:HIGH, IO1:HIGH, IO2:HIGH, IO3:HIGH, IO4:HIGH, IO5:HIGH, IO6:HIGH, IO7:HIGH
Disk size: 97.69779205322266 MB
Disk space used: 0.0 MB
StatusRequests and ConfigurationRequests are now fully implemented (with a few areas for improvement). Status returns the query fields with actual values. Request sets mode, psu, io pins, pullup, and LEDs.
//psu_enabled
if(bpio_ConfigurationRequest_psu_enabled_is_present(config_request)) {
bool psu_enabled = bpio_ConfigurationRequest_psu_enabled_get(config_request);
Everything is “working”, but my design failed:
- flatbuffers don’t send values that are not set
- I depended on 1 and used _is_present functions to test if the value is included in the table
- What I’ve now learned is that ‘0’ and False values (and probably defaults) are ALSO NOT sent.
- In the above, I test if psu_enable is present, and then try to _get the true/false value. This doesn’t work because if psu_enable is false, it is also not present and we can’t disable the PSU.
mode:string; // Name of the mode to configure.
mode_configuration:ModeConfiguration; // Configuration for the mode.
psu_enabled:bool; // Enable or disable power supply.
psu_set_mv:uint32; // Set voltage in millivolts.
psu_set_ma:uint32; // Set current in milliamps.
io_direction_mask:uint8; // Bitmask for IO pin directions (true for output, false for input).
io_direction:uint8; // IO pin directions (true for output, false for input).
io_value_mask:uint8; // Bitmask for IO pin values (true for high, false for low).
io_value:uint8; // IO pin values (true for high, false for low).
led_resume:bool; // Resume LED effect after configuration.
led_color:[uint32]; // LED colors in RGB format (0xRRGGBB).
pullup_enabled:bool; // Enable or disable pull-up resistors.
Some of these can be fixed by rearranging the code a bit. For example testing for positive value of io_direction_mask and io_value_mask. led_resume is okay too, because it is only done when true.
Issues:
- psu_enable
- pullup_enabled
- DataRequest:i2c_addr (think I can fix this with an invalid default of 0xff)
I’m not sure what to do. Adding a second psu_configure bool is wasteful because bools are stored as uint8.
The easiest thing would be to change the bools to uint8, and have three values: 0 (do nothing) 1 (false) 2 (true).
To do:
FixTest is_present fix- IO pin direction/value should be routed through bio so that updates to in-use pins (SDA SCL) are forbidden
- Build the protocol struct for processing DataRequests
- Mode configuration table
Generic error packet- Development/debug mode, with respect for terminal not connected
- Fix occasional mystery crash
1 Like
Would you be able to post the program you’re using on the host side to send the flatbuffers please? I’m still crashing the firmware (today’s commit) with a status request, must be losing my marbles.
Packet structure is pretty basic (pseudocode):
RequestPacket {
contents: StatusRequest {
query: [StatusRequestTypes::Version];
}
}
1 Like
flatbuffers.zip (104.7 KB)
Here’s the python script I’m using. The thing is, it may not be in sync with any firmware. I made several updates, but I’m too exhausted to test them today.
Other tip: I tied this to the “Binmode test framework” binmode. So you’ll need to type binmode and select #2. The later firmware should support the option to save it as the default startup mode.
The pattern in the linked code looks right to me. You make a status request with query for firmware , then wrap it in a packet.
Thanks, but no joy. Must be something off with my setup. I’ll give it a rest for now since this is very much the current situation:
1 Like
If you’re following the repo it’s probably my fault. I’m also learning on the go and today is the first day I kind of almost understand how to put everything together, then realized during testing I had completely failed the assignment. Tried to do some “prep” for tomorrow and now the script is out of sync with the firmware. I’m sorry about that. Tomorrow it should stabilize.
For the bool is_present issue, I think I will have separate pullup-enable and pullup-disable bool, that should be consistent and doesn’t depend on trickery.
Don’t worry about it, really I was getting in under your feet while you were still working
I hadn’t clocked that if a field is omitted, the default value is assumed, and all types have an implicit default. Here’s what a ConfigurationRequest with only the PSU mV set (Rust debug format output):
RequestPacket {
version_major: 0,
version_minor: 1,
contents_type: ConfigurationRequest,
contents: ConfigurationRequest {
mode: None,
mode_configuration: None,
psu_enabled: false,
psu_set_mv: 1000,
psu_set_ma: 0,
io_direction_mask: 0,
io_direction: 0,
io_value_mask: 0,
io_value: 0,
led_resume: false,
led_color: None,
pullup_enabled: false,
},
}
Only psu_set_mv was actually included, but the resulting structure would imply the user wants to essentially turn everything off.
The recommendation to make fields actually optional is to either explicitly set their default to null, or to wrap them in a struct:
… we cannot disambiguate the meaning of non-presence as “written default value” or “not written at all”. …
If you care about the presence of scalars, most languages support “optional scalars.” You can set
nullas the default value in the schema. …Another option that works in all languages is to wrap a scalar field in a struct. This way it will return null if it is not present. This will be slightly less ergonomic but structs don’t take up any more space than the scalar they represent.
Perhaps this could be removed? It’s the only part of the data request that is protocol-specific, and is at odds with the terminal bus command syntax. I don’t think it’s an ergonomic win for users as probably they won’t be interacting directly with the flatbuffer representation.
1 Like