Also I just updated the (very much in-progress) Rust HAL to the current schema, no problems.
1 Like
To make sure I understand this, the intention is:
- Client says “I need features from evolution 123.”
- Bus Pirate returns error if firmware evolution version is <123. Proceeds if ≥ 123.
Correct?
Yes this.
The down side would be that you couldn’t include extra fields that would conceptually be better as part of ErrorResponse and deserve to be non-string types. I don’t know that’s a real concern, an error string is pretty flexible after all.
We could still add more fields to the end of the table.
Tomorrow I will try to break the compatibility (simulated BBIO3) and see if we can really actually count on being able to send compatible packets.
Sorry for lack of quotes, not sure why it didn’t work on mobile.
1 Like
Okay, some real life testing.
minimum_flatbuffers_version > firmware: OK
I absolutely mangled the BBIO2 scheme:
- Removed a bunch of random fields (no-no)
- Changed the order of all the unions (no-no)
- Added in the monster.fbs example, and added monster to the front of the union (no-no)
BPIO3 host, BPIO2 firmware: BPIO2 can read “BBIO3” RequestPacket.version_major, and returns a BBIO2 ResponsePacket.error that BPIO3 can read: Excellent!
I really didn’t expect the BBIO2->BBIO3 to work.
I’m going to make the bold assumption that BPIO3 firmware can also respond to BPIO2 hosts. Mangling the flatbuffers for python is easy because it’s a script. Doing so in the firmware would require updating absolutely everything to get it compiling.
I’ll tidy up and push updates for the firmware and tooling/demos. The docs will have to wait in case there are further updates.
ETA: how o n earth did BPIO3 pass the verification in the firmware? 
1 Like
I am a glutton for punishment so I updated the docs too.
Maybe this is it? Maybe it’s done?
1 Like
Good … this will work. Please note that technically there is still a default value for an integral integer … zero. So… don’t use zero as a valid version, and the buspirate can require that the client explicitly set the value. IIRC, there’s also a way to wrap the integer (as a single-item optional table?) if you ever need the full range of values to be valid, as you can check for the existence of the table separately from the value it stores.
1 Like
Hi Ian,
I think it would be helpful to write up a set of rules that should ensure bi-directional backwards compatibility between firmware and clients using BPIO2.
(Request == Host to BP; Response == BP to Host)
Backwards compatibility is defined as:
- BP firmware can detect unsupported protocol in a request (e.g., client evolution is not supported by firmware)
- BP firmware can report error for unsupported commands in a request (e.g., Specific request type is unsupported)
- BP firmware can report error for supported commands in a request(e.g., command is supported, but errors parsing / processing the command)
- Client can thus distinguish between unsupported evolution, unsupported command, and errors that occur when parsing a given command.
- Old clients continue to work with new firmware without recompilation and without modification.
Resulting rules for Flatbuffers
- Unique root for requests and responses.
- OK, this is technically not required, but just makes sense…
- Root tables contain only nested tables.
- Why: allows each table to grow, without impacting order of remaining tables
- Request Root first table is Evolution.
- Why: Prevent future fundamental breaking changes from being misinterpreted by firmware and crashing.
- Firmware must check evolution on each incoming packet.
- → Evolution table can be extended in non-breaking fashion only … even across evolutions ←
- Response Root first table is Errors.
- Why: Regardless of evolution or other data, the error data, if the table exists in the response, must be discoverable by clients.
- → Errors table can be extended in non-breaking fashion only … even across evolutions ←
- Request tables are to be considered immutable contracts (as to data from client → BP).
- Why: earlier firmware will entirely ignore new fields / tables. It is difficult for client to know if a field will be ignored or not. Thus, must treat the requests as immutable contracts for bi-directional backwards compatibility.
- Similarly, response table fields are to be considered immutable contracts.
- However, it is permissible to add a nested optional table for additional informative data.
- Similarly, it is permissible to add additional optional tables to the root response packet, which are provided by newer firmware.
- Why: Clients need to distinguish between data not being provided, vs. data being provided (even if wrong data).
- In designing the request / response communication flow, each table that is populated (in the root) should result in one corresponding tables in the root of the response packet.
- In simplest form, a single populated table in request root returns a single populated table in response root.
- Obviously, also must include Evolution / Error tables, respectively.
Summary
Following the above rules should ensure bi-directional backward compatibility. Hopefully, it will also prevent the need to increment the Evolution … ever. However, should that ever be required, these rules will at least prevent crashing device firmware / confusing clients.
2 Likes
Working a bit on SPI, is it possible for the Bus Pirate to return the bytes read while writing under start-alt mode { ?
My desired scenario is:
- MOSI and MISO shorted
- DataRequest with start_alt=true; write 1 byte; read 1 byte.
- Write 0xAA
- Receive 0xAA
But presently any reads take place after the write, so there’s an extra 8 clock pulses and the read data is the 0xFF dummy byte.
I’m trying to fulfil the contract of transfer and the related transfer_in_place.
1 Like
I haven’t done that because it will require a rule. Do you have a suggestion? My notes in the code suggest read with write bytes first, then read bytes after. Not to exceed total max_read_bytes. If that sounds good it is easy to implement the function and a bit more to do limits testing.
2 Likes
Yes I think that sounds right: write & read together, read more if necessary.
Would short reads (read fewer bytes than are written) cause a problem for you?
2 Likes
It’s more an issue of heap overflow. We have a 2k heap, and flat buffers builds in that heap. Nothing else is using the heap (BC I didn’t know it existed…). The issue is to make sure we don’t overflow the heap. For example 512 write with read bytes and then 512 read bytes would exceed the max_bytes_read. I need to look, because this would need to be handled at the protocol layer and I’m not sure there is a chain to push the error back up to the top at the moment.
It’s not a huge change, but it is a TODO in the code comments. 
+1 to this plan!
(Plus extra characters for posting this reply.)
1 Like
It’s a bit of a hack, but full duplex SPI is now supported when start_alt = true.
- Updated firmware
- Added support/test to python example library
- Updated the docs
Very annoying bug I have not been able to track down.
Enter SPI mode from BPIO2 and the MOSI pin label is not printed.
Enter SPI in the terminal and the MOSI pin label is printed as normal.
They both use the protocol_setup_exc function. It doesn’t make sense.
I suppose it is an issue about when the changes are picked up and redrawn by the second core, but yikes.
Ouch… Schroedingbug…
those can be some nasty fun to debug when they absolutely don’t make sense
I recommend enabling / using the RTT debugging capability when tracking these types of things down. Debug prints are then simple memory copies, so have lesser chance of creating a Heisenbug.
IIRC, you previously mentioned that BPIO2 does something odd with the screen refresh? Something about voltages not updating in BPIO2 mode?
If I get a chance to look into it, do you have steps to easily reproduce the issue? (yes, I’d need some hand-holding … first time I’d use BPIO2; Maybe the level of example docs, without needing the screencast?)
1 Like
I think the voltage display was an issue with the legacy 4 third mode. I fixed that by changing to the higher level functions.
For the SPI display issue, you can use the spi-example.py to enter spi via bbio2, and use the terminal to enter and see the correct display.
There are docs for the Python examples are here:
1 Like
the fun starts when even a single debug print changes some behaviour and a bug disappears.
Load bearing logging is nothing that you want in the code
It looks like I forgot to push the updates for SPI full duplex, sorry about that. The latest should be in the repo now.
While we wait for new plank designs to arrive, I’m going to continue linking new modes into the BPIO2 interface. DIO is probably next, then 2 and 3 wire.
Based on the bug report by @henrygab, I explicitly added HiZ mode to BPIO2. I’ve also finished 2WIRE and 3WIRE, but need to update the python wrapper so I can do testing.
Tomorrow I’ll continue on DIO, LED and infrared modes. This should come together quickly.
I’m a bit hung up on how UART and HDUART will work: data arrives asynchronously so it’s not like the other protocols. We may need another packet type, and a async service loop.