BBIO2 binary mode

There’s some scripts out there that support the BBIO1 binary interface, but the support is so old and variable I’m not sure emulating it has much value. The major targets are sigrok and flashrom. Sigroks windows SUMP driver is messed up, and flash rom still tries to parse ancient text from the terminal. Maybe also AVRdude?

I’m going to take this as license to build something new. BBIO1 is a mess because it was hacked together in stages and we just didn’t have many resources left in the PIC. It’s simple and easy to understand, but it’s inconsistent and overly involved.

Goals

  • Consistent control of all bus pirate features in every mode
  • Data first, always be sending data unless otherwise requested
  • Simple values - PWM, PSU, etc set with plain text values and the right dividers are calculated

Many wireless packet protocols use 0x7f as an escape character to indicate system commands. I think that’s the direction to go. If 7f is data, it’s escaped with 7f, otherwise commands follow. Maybe commands can be 7f with command byte and 4 data bytes (like sump commands).

Global commands

  • Change mode
  • Vreg enable (optional current limit)
  • pull ups
  • echo string to terminal
  • xon flow control ?
  • xoff flow control
  • setup mode. Possible get descriptor of mode setup options? Maybe speed and settings are separate commands?
  • LED control
  • LCD control (maybe a mode?)
  • pwm
  • measure freq
  • gpio aux stuff
  • pin direction
  • ADC measure (setup and get reading)
  • get current (plain text)
  • get pin voltage (plain text)
  • delay
  • silent (wether to acknowledge every command?)

Mode specific commands, like what we now call bus syntax in the user terminal

  • read
  • all non escaped data is write
  • start
  • stop
  • bulk read write (used by flashrom and avrdude)
  • scan for devices (I2C, 1wire)

7f 02 03 30 - command 2 set PSU at 3. 30 volts for example.

It would be cool to have a mode that debugs the binary interface from the UI. All least it would be convenient to me.

I’m going to target flashrom, avrdude, and asprogrammer. I saw dreg has an asprogrammer hack in his github. I’ll get a build server setup so we get auto compiles.

Any thoughts?

1 Like

Hello Ian! Are you focusing more on the readability of the protocol or its integration with existing tools? If the last is your main focus , a suggestion would be that you could take into consideration use of Protocol Buffers as the transporting means. It would simplify a lot the development of new scripts in virtually any language and take from you the burden of developing your own.

1 Like

Thanks for the suggestion… I’m not familiar with protocol buffers. How does that work?

You could consider it to be a serialization method, such as COBS or even JSON. It is a quite popular format, created by Google, and has a lot of tooling available.

Using this method, we could define the protocol using .proto files, a human-readable file format similar to C headers where users could create clients in virtually language.

Eg.: Imagine you had defined the protocol as:

bus_pirate_commands.proto

These are specific commands to change the state of the buspirate, their values and parameters.

syntax = "proto3";

package buspirate;

// Commands for altering the state of the Bus Pirate
message Command {
  required int32 version  = 1; // always good we do versioning
  oneof payload {
    ChangeModeCommand change_mode = 1;
    VregEnableCommand vreg_enable = 2;
   (...) we can include more as we go
  }
}

message CommandResponse {
  oneof response {
    SuccessResponse response = 1;
    ErrorResponse response = 2;
  }
}

message ChangeModeCommand {
  enum Mode {
    SPI = 0;
    I2C = 1;
    // other modes
  }
  Mode mode = 1;
}

message VregEnableCommand {
  bool enable = 1;
  // Optional current limit, if applicable
}

message SuccessResponse {
   repeated uint32 message = 1
}
message ErrorResponse {
   repeated uint32 message = 1
}

bus_pirate_queries.proto

These are queries used to request data from buspirate.

syntax = "proto3";

package buspirate;

// Queries for retrieving state from the Buspirate
message Query {
  oneof payload {
    GetCurrentQuery get_current = 1;
    GetPinVoltageQuery get_pin_voltage = 2;
    // Add other queries as needed
  }
}

message GetFrequencyQuery {
  // Details for the query, if any are needed
}

message GetPinVoltageQuery {
  // Details for the query, if any are needed
}

// Responses to queries
message QueryResponse {
  oneof response {
    FrequencyResponse current = 1;
    PinVoltageResponse pin_voltage = 2;
    // Add other response types as needed
  }
}

message frequencyResponse {
  float frequency = 1; // in hertz
}

message PinVoltageResponse {
  float voltage = 1; // Voltage in volts
}

With that, I could easily consume it using python as (after compiling it with protoc):

from buspirate_commands_pb2 import Command, ChangeModeCommand

# Create a Command message
cmd = Command()
cmd.version = 1  # Set the version
cmd.change_mode.mode = ChangeModeCommand.SPI  # Set to change mode to SPI

# Serialize the command to a byte string
cmd_bytes = cmd.SerializeToString()

# Now, cmd_bytes can be sent  over com port.

I would advise you to take a look at the nanopb project, it is a very lightweight implementation (less than 10kb size and requires less than 1kb ram memory). The best of all, it has a short permissive license, compatible with GPL.

3 Likes

Yes, protocol buffers would be a valid choice as base for a new binary mode.

If for some reason you decide against it and keep rolling your own protocol, then consider COBS instead of traditional escaping with something like 0x7f. This traditional escaping has a very bad worst case efficiency when the escape character is often part of the data to transmit and COBS is able to improve that.

Have a look at the wikipedia page for an explanation Consistent Overhead Byte Stuffing - Wikipedia

1 Like

Very nice, thank you both. That looks great.

Today I did a major rework of commands and command line arguments for the spi flash utility. The next step is the binary mode. I will check this out.

Ian, I can second the recommendation to use protocol buffers (aka protobuf)!

The beauty of protocol buffers is that client code is auto-generated for many, many languages. I did not think there was such a small implementation available, else I would have absolutely recommended it!

Strong support from me on use of protobuf!

There’s a bit of progress on this I should push, but the current in-progress thing is reworking the modes so they return stuff in a way that works with both the UI handler and the binary mode. It would be nice to maintain a single system for interacting with the bus modes.

This is the absolute next big thing to finish when my smaller to-dos are sorted. It has slipped a few weeks because of unexpected surprise projects.

Edit: The goal is to build some framework for doing all the “stuff”, and then plug whatever interface layer onto it. Maybe even multiple interfaces if I’m feeling ambitious, which would solve the issue of “how do we pretend to be other devices and have our own interface”. However, that issue is probably better solved with a second stage bootloader that jumps to different programs within our single firmware. This is something I’ve played with but not implemented because it will also mean reworking the build server.

1 Like

I have a solid block of time (fingers crossed) to wrap this up. I’m focusing on getting the functional framework implemented, and then that can be attached to any protocol layer.

In the binmode branch there is a “bin” mode in the terminal that can be used for testing. It’s kind of a loopback into the binary mode.

These are proposed commands and data formats for the global Bus Pirate commands. Things that should be available in every protocol mode. Please note that this is not a protocol, it’s a framework for interacting with the Bus Pirate in binary mode. The protocol will come later.

Test firmware

bus_pirate5_rev10-binmode-2.zip (185.3 KB)

This is tracking the binmode branch in github.

Test rig mode

BIN mode (currently mode 10) provides a loopback to the binary interface. The Bus Pirate command line and tools are available for debugging the binary interface without any clunky external software.

Return codes

Commands with no output currently return 0 for success or an error code (1+).

BM_RESET = 0,

image

Reset the Bus Pirate current mode to HiZ, reset all hardware (hopefully).

Note that in BIN mode, this resets the terminal back to HiZ.

BM_DEBUG_LEVEL, //1

image

Set debug level. Takes one argument:

  • 0 - no debug output
  • 1 - show debug output in the terminal

This handy setting prints debug info to the Bus Pirate terminal (not the binary port). It shows:

  • The command received and the number of arguments it requires
  • Output from the command used
  • The command return code (0 success, else error)

BM_POWER_EN, //2

image

Enable the Bus Pirate power supply. 4 byte arguments:

  1. Voltage integer value (0-5)
  2. Voltage decimal value (0-99)
  3. Current limit mA high byte
  4. Current limit mA low byte

Set the current limit to 0xff 0xff for no current limit.

The power supply has some return codes:

  1. Fuse blew during power on, possible short or current limit too low
  2. Voltage level too low

BM_POWER_DIS, //3

image

Disable Bus Pirate power supply.

BM_PULLUP_EN, //4

image

Enable pull-ups.

BM_PULLUP_DIS, //5

image

Disable pull-ups

BM_LIST_MODES, //6

List available modes. Gives a comma separated list of available modes. Ends with 0x00 (null terminated string).

BM_CHANGE_MODE, //7

Not yet implemented. I think the thing to do is send the name of the mode (as printed above) so we’re not reliant on fixed numbers. Should probably make it case insensitive.

BM_INFO, //8

Reserved. Returns 1 for invalid command.

BM_VERSION_HW, //9

Currently returns the a null terminated string, but this will probably change to a 3 byte reply or something.

BM_VERSION_FW, //10

Currently returns the null terminated compile timestamp.

BM_BITORDER_MSB, //11 & BM_BITORDER_LSB, //12

image

Set bit order: MSB or LSB.

BM_AUX_DIRECTION_MASK, //13

image

Set IO pin to input or output. Will not effect pins currently assigned to a peripheral. Requires two argument bytes:

  1. Bit mask of pins to change
  2. Bit mask of the pin direction (1 = output, 0 = input)

This should probably have a companion function that returns the free IO pins, and maybe the functions assigned to any pins.

Edit: I fixed the wrong direction values shown above.

BM_AUX_READ, //14

image

Return a byte with the current state of the 8 IO pins.

BM_AUX_WRITE_MASK, //15

image

Write to IO pins. Does not effect pins currently assigned to a hardware peripheral. Requires 2 byte arguments:

  1. Bit mask of pins to write
  2. Bit mask of the value to write to those pins

Edit: also fixed the incorrect output state number.

BM_ADC_SELECT, //16

image

Select the ADC channel to read. Requires one argument

  1. ADC channel according to the list below.
    HW_ADC_MUX_BPIO7, //0
    HW_ADC_MUX_BPIO6, //1
    HW_ADC_MUX_BPIO5, //2
    HW_ADC_MUX_BPIO4, //3
    HW_ADC_MUX_BPIO3, //4
    HW_ADC_MUX_BPIO2, //5
    HW_ADC_MUX_BPIO1, //6
    HW_ADC_MUX_BPIO0, //7
    HW_ADC_MUX_VUSB, //8
    HW_ADC_MUX_CURRENT_DETECT,    //9
    HW_ADC_MUX_VREG_OUT, //10
    HW_ADC_MUX_VREF_VOUT, //11
    CURRENT_SENSE //12 

Channels 0-11 are measured through the analog mux and are divided by 2. Channel 12 measures the current sense, which is not divided.

BM_ADC_READ, //17

image

Read from the currently selected ADC channel, return actual voltage. Returns 2 bytes:

  1. Voltage integer value
  2. Voltage decimal value

Note that the return values are not ASCII text, and have already been adjusted to account for the divide by 2 for analog mux channels.

BM_ADC_RAW, //18

image

Read from the currently selected ADC channel, return the raw ADC value (2 bytes).

Channels 0-11 are measured through the analog mux and are divided by 2. Channel 12 measures the current sense, which is not divided. The raw uncompensated values are returned.

BM_PWM_EN, //19

image
Enable a PWM at specified frequency in Hz. Six argument bytes:

  1. IO pin (0-7)
    2-5. 32 bit value of frequency in Hz
  2. Duty cycle in percent (0-100%)

I would like to return the actual frequency, but it needs to be balanced against the return codes. I’ll probably repurpose the raw command to fetch the actual frequency.

Return codes:

  1. Pin out of range
  2. Pin in use/not available (also if the adjacent slice is used)
  3. Frequency out of range

BM_PWM_DIS, //20

image

Disable PWM. Requires one argument byte: IO pin number. Return codes:

  1. Pin out of range
  2. PWM not active on specified IO pin

BM_PWM_RAW, //21

Enable PWM with raw register values. Not recommended as it will vary by system. Requires 7 argument bytes:

  1. IO pin number (0-7)
  2. 12 bit PWM divider value (2 bytes)
  3. 16 bit PWM top value (2 bytes)
  4. 16 bit PWM duty cycle value (2 bytes)

Return codes:

  1. Pin out of range
  2. Pin in use/not available (also if the adjacent slice is used)

BM_FREQ, //22

Not implemented. Measure frequency on pin, return value in Hz.

BM_FREQ_RAW, //23

Not implemented. Return raw frequency measurement.

BM_BOOTLOADER, //24

image

Jump to bootloader. Does not currently return 0x00 because of reasons.

BM_RESET_BUSPIRATE, //25

image
Reset the Bus Pirate. Does not currently return 0x00 because of reasons.

BM_PRINT_STRING, //26

Print a string to the Bus Pirate terminal. The string must be terminated with 0x00.

Potential additional commands

//self test
//disable all interrupt
//enable all interrupt
//LEDs?

//SUMP logic analyzer mode?

1 Like

Updated with PWM commands. Attached a new firmware, and pushed the latest code to binmode branch.

I want the PWM to return the actual frequency, so I still plan to add a command for that.

I suppose we probably really need some delays too? For bit banging?

Next I’ll add a config menu option to enable switching between different binary modes: the current SUMP logic analyzer interface, the new binary mode, and whatever else we want to hack in there in the future.

Then, getting the protocols struct working with binary mode. This was a sticking point previously, it is going to require some reorganization.

Speaking of reorganization: the PWM in the terminal and in binary mode really need to be cleaned up into a sensible library.

So… is there a reason to have BBIO2 mode use incomprehensible numbers as command identifiers? Maybe it was a right of passage in the past… Is it really a goal to make it hard to write / comprehend BBIO2 scripts?

For example, why not allow BBIO2 to use friendly names (e.g., BM_FREQ etc.) instead of hard-coded numeric values?

If this is a performance issue, would it make sense to have a “compilation” pass that translates from human-readable script?

 

NOTE: This is a GREAT starting point for a protobuf definition, and starts to highlight why protobuf was created in the first place…

Imagine the power of being able to use script the BP5 with your choice of Python, C#, etc. …

I’m in favor of protocol buffers, at least what I understand of it. Currently I’m making a framework of functions to build the protocol on top of. Today I hope to get all the syntax/protocol mode related stuff wrangled. The numbers are just elements of an array to help me debug, they don’t mean anything.

RP2040’s low speed USB does create some performance concerns for using long/human readable commands as well.

1 Like

I believe the mode stuff is under control. There is a new binmode setup function in the modes array. There is a function to change modes and configure modes, and read/write/etc should work. Untested though, pushed to binmode branch.

I totally misunderstood! Reading back, I see you communicated this … and I just didn’t get it. Sorry for my misunderstanding! :flushed:

1 Like

I added a config menu option (nice cleanup on that @henrygab) to select between different binary interfaces. These are just some examples of what you might find there.

An array of references will point at the main loop of different modes (needs to be cooperative multitasking when not active). The selected mode will be called in the main loop in pirate.c.

This removes all the trickery we used in previous binary modes to make a bunch of different protocols (SUMP, etc) play well together. It solves the issue of how to port other projects like pico probe* and dirty JTAG and the gusmanB logic analyzer, etc into our single firmware.

@lersi I believe has some SUMP improvements on the test branch that I’m going to pull over here soon as part of this update.

*pico probe newer versions use an RTOS, so I’m not 100% sure how that will go.

2 Likes

A small rework to be better prepared for an integration with a protocol buffer:

  • Removed global arguments. binmode functions now accept a reference to arguments
  • Lumped the global/local commands together for internal cleanup
  • Reworked the testing state machine, also now has a null terminated data sucker uper.
  • Change mode now uses argument reference, rather than consuming data directly.

SPI now has a binary configuration function. It currently consumes data directly, which needs to change. Here’s the thing: all modes have different length of configuration options. So, the upper layer protocol needs to know how many configuration bytes to expect from the user. I can see a few ways to do this, but none are very clean:

  • Return the number of config bytes from a new function in each node
  • Store the number of config bytes in the modes array (sloppy)
  • Return the number when entering the mode (except that we don’t actually enter the mode at config time, see below)
  • A set config packet size, zero padded for unused bytes (ugh)

Entering a mode is a two step process:

  • Enter the mode. This technically just cleans up the previous mode and sets a new mode number in the system_config struct.
  • Configure the mode. Like the Bus Pirate terminal, selecting a mode is the first step, then the mode must be configured before it is active. After this step the LCD pin numbers update and the buffered IO states are set.

I want to think about the configure step a bit. I assume it will take a new function to return the required number of bytes during the enter mode step.

1 Like

Oh, and added a function to change binmode handler within binmode. And a new binmode version command that returns BBIO2.xxx where x is a version number.

The main modes struct now has a function to query the number of configuration bytes required. Those bytes are then gathered by whatever protocol and handed to binmode_config by reference.

SPI is the only mode with a prototype config length query and binmode_config function at the moment. No sanity checking yet either.

(0x3b9aca0) (4-8 bits) (CPOL=0) (CPHA=0) (CS=1)

The SPI config is 8 bytes long. The first four bytes are the SPI speed in Hz, maximum 62500000 (realistically 12-20MHz). New byte is the number of bits in each SPI frame (usually 8). Clock polarity & clock phase. Finally CS idle state.

Read and write functions need to be updated to not consume data directly. This, I believe, is key to using a protocol buffer as the library hands us actual populated structs to work with. Maybe this is a good use of creating/destroying temp variables with the new bigbuf API.

  • Sanity check if functions are available in the current mode
  • Read and Write not consume own data
1 Like

So sorry to bring this old thread back to life. I previously did quite some work on BP3.6 especially scripting a fair bit of I2C programs with pybuspiratelite. I know this is a long shot but I wonder if there’s any way that the BP5 can be backwards compatible? Failing which I will need to refactor the code. Any help would be appreciated! I keep ending up not being able to connect to BBIO mode.

2 Likes