Usually the Bus Pirate is used with the main Bus Pirate firmware. But this firmware doesn’t have support for some protocols and usage scenarios. For this externally developed firmware is ported. Reasons for this are mostly code complexity and licensing. For example this is in the works right now for JTAG and SWD, but also JTAG detection, voltage glitching and other things are on the horizon.
I like that this expands the things the BP can be used for.
But currently you have to completely re-flash the BP with the new firmware whenever you want to run one of these external firmwares. When you regularly switch between tasks this becomes burdensome and makes the BP harder to use as a tool. Also you probably need to organize the files of your different firmware binaries, so that when you want to go back to another well-working one, you don’t have to download and test several versions or even recompile.
But the BP comes with much bigger flash than for example the Pico or other boards. So how about having multiple firmwares flashed in parallel and switching between them with a command?
The RP2350 has support in hardware and firmware to make using different firmwares more easy. But the BP5 with the RP2040 is quite common so I think it would be better to make this concept also usable on the RP2040.
This is how I think this could work:
There is some kind of central registry for third party firmwares that assigns a flash start address and region to each. The regular BP firmware gets the beginning of the flash. The start addresses must be aligned to flash pages.
The firmwares are compiled with a custom stage2 bootloader (modified exit_from_boot2.S) and linker script that uses this flash start address.
So the .uf2 file spans two different flash regions: the stage2 bootloader in the first 256 bytes, then a big hole, and then the respective third party firmware from it’s assigned start position.
This makes the third party firmwares usable on their own. So you can flash them as they are and run them. This is important for licensing, because then you have reasonable arguments that they are still standalone and not somehow linked among each other.
When you flash this firmware, just the pages that are in the uf2 file are overwritten, the other flash regions are left as they are.
So you can flash several third party firmwares into their own regions and then the regular BP firmware last. When you now boot, the regular BP firmware runs because it’s stage2 bootloader was written last.
The regular BP firmware gets a new command to boot into another firmware.
This probably stores some command in the watchdog scratch memory, does a reset and then the start code jumps to the other firmware. The reset also does a usb-reset, so the other firmware can present it’s own usb descriptors.
There should probably be some logic to search for installed firmwares and show their names and versions in a menu.
When you reset the BP, it starts in the regular BP firmware again. So you don’t have to reflash anything when you want to switch firmwares. You just use the new command on each boot.
The watchdog scratch memory could probably also be used to transfer information like target voltages and current limits that currently are hard to integrate into the third party firmwares because they lack an interface to set them
This isn’t 100% fleshed out yet, but I think this could work.
What do you think of this idea? Is it worth the extra work?
I won’t have time to implement this soon (the port protection stuff comes first), but I mull on this idea over the last days and want to float it.
I was reading thru this and was wondering how do you switch back to regular firmware from example from JTAG firmware but you have it sorted out that you simply reboot and by default it goes to standard. I like the idea.
But also there will be problem to track versions. As BP firmware have i command to display build number. How would you track versions of secondary firmware’s?
There should be some place in memory that is readable by main firmware with versions of the secondaries.
And then in case of new build you just shuffle correct binary
In general, there is a potential for this to work. However, the idea of many start addresses pre-assigned to specific alternative firmware is (imho) not a long-term viable option. Already the main firmware build is using ~1/3 of the flash. This is likely to grow over time.
As an alternative, perhaps define a single alternate starting address for the alternative firmware… maybe the last ~1/3 of the space?
The main firmware could list firmware files (stored in NAND). When one is selected, it could verify the contents vs. the what’s already flashed. If different, then it could offer to flash that alternate firmware to that second region, and then reboot in a way to cause the second-stage bootloader to jump to the alternate firmware location. This reboot-to-alternate-config is a well-understood hardware path, although I’ve not delved into the specifics for RP2040.
In this way, if there’s usually only one alternate firmware needed, it only needs to be flashed once. Later uses will validate the firmware w/o writing, and then just reboot. At the same time, the power user might have many alternate firmwares listed, and the BP would internally have the ability to parse the .UF2 file to re-flash the alternate firmware. The code to parse .UF2 is trivial, and could reject any .UF2 file that writes outside the reserved-for-alt-firmware addresses, for example.
So… similar to your concept, with the exception that only a single alternate firmware is used at a common fixed address (+ some glue to write it from within BP5 main firmware).
This is exactly what I had in mind for 3rd party firmware. Locating then after the main firmware, and using a linker to set the higher addresses without the hardware the 2350 has for this purpose. It’s been a while, but I did read several threads where people had accomplished exactly this on 2040.
In terms of flash space: the current firmware uses ~650K out of 16 megabytes. In the 2350 build this is correctly reported, but in the 2040 build I could not see how to override the 2MB region size. I did the same as I did for 2350, but it persists in reporting 2MB capacity.
I figured this was related to the known issue of 100% ram being used, even when it is very much not and would be fixed in the next SDK update. However, I am now using SDK 2.1.0 and neither bug seems to be fixed.
I didn’t realize it had 16MB! Ok, that does make it more possible to have more than two … but maybe not worth doing on the RP2040 due to complexity. For RP2350, using the built-in partitioning (when they fix the errata … E10?) sounds like a winning option.
One additional thing to consider: Any change in this direction would likely require post-processing the .UF2 file, to ensure it doesn’t write all-zero data to the main firmware when writing one of the other firmwares.
Because of the above risk, would it make more sense to start:
(a) on RP2350, start using the partitioning, and put the main firmware into a later partition, to at least verify this functionality works (even on the A2 revision)?
(b) on RP2040, change the firmware start address so it’s really close to the end of the address space?
This way, the alternatve firmwares might be less likely to overwrite the main firmware. And then the firmwares can “grow” towards each other over time (much like stack vs. heap).
Did you ever encounter this, that the bootloader zeroed unused flash pages for you?
The .uf2 format has metadata in each sector that explicitly says which target address it should go to. So unlike with a dumb .bin it is defined where the data should go. And in my experience this actually works, so the .uf2 file just contains data for the space you actually use, and the bootloader just zeros those pages that are written to.
So I think you don’t need any postprocessing. But you have to properly configure the linker script beforehand to make it work.
Yes, I would also suggest them growing towards each other: the main BP firmware from the beginning, and space allocated for the third party firmware backwards from the 16MB.
When you do all the sector handling in the BP firmware, it will work on the RP2040 and RP2350, regardless of errata. But even when you implement it all in the BP firmware, I would suggest to still use the sector formats as defined for the RP2350. Just to allow to re-use the tooling in the SDK and similar.
The bootloader integrated into the RP2350 has a definition for a metadata format that describes each firmware. You can store things like names, version numbers and checksums in it. I would suggest to use this format, even if the code is implemented in the BP firmware and would also run on the RP2040.
The BP firmware would probably scan through the whole memory and read the beginning of each flash page. If the data there matches this format it can read it out and verify it. Then you get a list of all available firmwares and show it.
Exactly. If the linker script isn’t properly configured, it could zero the entire preceding region, and thus the UF2 would necessarily have zero data for all those prior areas. BTW, I made significant contributions to the tinyuf2 bootrom, and am familiar with UF2 internals and interactions with the host file systems(*). It works because of duct tape / smoke / mirrors.
That layout would appear to require modifying many 3rd party firmware to be compiled with a different start address, including the aforementioned linker settings. All those 3rd party firmware would all potentially need to change if the start address of the main firmware changes.
Wouldn’t it be simpler to have the main buspirate firmware start at the later offset? Not only would this force us to verify the functionality works, if the main firmware’s start address has to grow downwards, there would be no change required in the alternate firmwares. In other words, keep the changes to one place where the greatest control is (main firmware), and make it easier for the alternate firmwares by keeping them as close to “normal” as possible?
(*) as opposed to analog / ee stuff, where I’m still a hobbyist at best.
Ok, I haven’t looked too deep into it. It just worked for me as I expected, didn’t overwrite my flash once I had the linker script right, and I didn’t notice any issues. So the smoke and mirrors was good enough for me
Exactly. Their linker script and their stage2-bootloader (when you want to be able to run them standalone, mostly for license reasons as I wrote in the first post) would have to be modified.
The pico-sdk comes with a linker script. In my experience most projects targeting the RP2040/RP2350 use this linker script unmodified. So when you have once figured out how to properly modify this linker script from the sdk, it will be trivial to adapt for all firmwares that use it.
The main BP firmware always is the first one, so it won’t change it’s start address.
Even if it would, I don’t see why it would matter for the other firmwares. They will be built in a way that they must not rely on the main BP firmware at all. A regular stage2 bootloader, with just the start address in exit_from_boot2.S adjusted, will be able to start them. And this is the same interface and calling convention the BP firmware will use.
I think it would be easier if the main firmware can just grow substantially without forcing you to adjust the start address after some time.
I think the third party firmware ports mostly cover one topic each, like JTAG, SWD, glitching,… - so they probably won’t grow in size more than 2x or 3x. But the BP firmware covers many topics and areas and I think it could still grow substantially. So it will be easier statically allocating a chunk of flash for them than for the main BP firmware.
Flash linker issue is now github Issue #169 … which is now resolved with PR #170.
Bug notifying of 100% RAM utilization is now github Issue #171marked as external per your notes, but left open as it’s not resolved. BTW, RPi has a fix … just hasn’t made it into release builds yet. … which is now resolve with PR #172.
In general, the solution by electronic_eel would have those many edge cases and hacks, but has the benefit of allowing more than one alternative firmware to be programmed at once. However, each alternative firmware would need to know, before being built, where its start address would be (and thus be recompiled with any change).
In contrast, the solution I would champion would have a restriction of allowing only one alternative firmware to be programmed at once, but avoid many edge cases and hacks. Moreover, by having the alternative firmware load at the default address, it avoids the need to keep rebuilding firmware for multiple custom start addresses. This eliminates the concerns about alternate firmware versions (including their expected start address) … and also reduces maintenance / edge cases for those alternate firmware (limiting the more technical changes / linker settings to BusPirate’s main firmware).
Both are valid ways forward.
Electronic_eel’s proposal allows many firmwares to co-exist in the 16MB firmware space. My proposal has lower overhead, management code, and edge cases … but limits to a single alternative firmware for that simplicity (on RP2040), although it allows that alternative firmware to be changed independent of the main firmware.
I guess it just depends on who gets something working and documented first.
On this, I believe there is space in the firmware layout for some kinds of information in a standard way. That might make it searchable from the “base” firmware.
$ picotool info -a
Program Information
name: hello_world
features: stdout to UART
binary start: 0x10000000
binary end: 0x1000606c
Fixed Pin Information
20: UART1 TX
21: UART1 RX
Build Information
build date: Dec 31 2020
build attributes: Debug build
Device Information
flash size: 2048K
ROM version: 2
This is specified in the datasheet for the RP2350. See section 5.1.4 (“Image” and “Image definition” and section 5.9 (“Metadata block details”).
This is what the integrated bootloader of the RP2350 uses and picotool (and the rest of the sdk) supports. I suggest to use this format, even on the RP2040, because of the sdk support with things like picotool.
