Storage architecture discussion

General
61

There are many unwritten assumptions that the code relies on to be “safe”; my goal is to find them, and add at least run-time validation that those assumptions still hold. That’s what I did here, by ensuring that the core releasing the mutex was the core that acquired it. Worked fine … until a terminal was connected … which resulted in finding (and fixing) this bug. I admit to some pride in tracking this one down … and entirely via code review / RTT output (i.e., I never connected a debugger). :exploding_head:

As for the root cause being related to the NAND… that was just fortune smiling on the project. Data loss is never good, so anything that reduces the chance of corrupting data is good in my book.

3 Likes
62

Somewhere you asked about mutex.c, I believe it is only a file on my system. Possibly from the pirate-lib. I’m sorry for the confusion.

1 Like
63

It was a really good catch! Those kinds of finds feel so good. Someday I’ll spend more time contributing to the project, but for now I’m busy using it and adding my own use-specific code.

BTW, I have RTT capabilities, so if you ever want another test with that output just let me know :slightly_smiling_face:. (I haven’t left it connected, the setup I used to connect the picoprobe is a bit sketchy, lol)

2 Likes
64

Thanks! Glad to know you got RTT available also. My setup is also… Fragile… For now.

I’m looking forward to the JLink adapter boards, as I am hoping the speed of RTT will go up tremendously. If so, then I’m going to order an upgraded JLINK… . having good tools is similar to getting a new dev box.

In hoping to get full terminal support via RTT soon. That may be hampered by the current storage architecture, which switches to read only based on terminal connection.

Might first recommend the need to change storage to be { fw:r/w, host:ro} by default, and switch to { fw:ro, host:r/w} via a command… (And then changes back on any key press).after all…

Current design is going to still see corruption; I hope to convince Ian of this risk before the customer base gets too large, which would make such change more difficult. I know I may not succeed in this, and in the end it’s his choice to accept the risk, but I have to try. :face_with_spiral_eyes:

3 Likes
65

yeah, having a controlled switch that works withhout terminal disconnect is the best since you don’t need to resetup stuff after jiggling around files

2 Likes
66

Generally I’m on board! I appreciate greatly everyone’s contributions. This is a barn raising, I can’t do it all on my own.

1 Like
67

OK, I will look at making the on-board file system read-only by default, and adding the new command to temporarily give exclusive access to the storage to the host (read/write).

details...

I will add a new command that changes the storage to be unavailable(*) to the firmware, but read-write for the host. This command will then wait for the user to “press any key”, after which it will switch the firmware back to original settings (r/w for firmware, r/o for host).

From a user perspective, there are few reasons to upload data from the host to the buspirate. The known exceptions are:

  1. When adding a script
  2. When restoring a backup of a configuration file
  3. Copying a binary image (e.g., for writing to SPI flash)

The new command will fail if the firmware has any open handles … ensuring no corruption due to host + firmware writing simultaneously.

(*) Read-only access by the firmware could give corrupt data, as the order the host writes to the media is undefined. As a hacker tool, it’s unlikely that all data parsing paths have been “hardened” against tainted input …

I think this will be my next PR (deferring RTT terminal changes) … I will give updates on this thread.

2 Likes
68

Another use case for writing is to copy a binary file that would then be written to a SPI Flash device. Probably others I can’t think of.

2 Likes
69

Interesting … you wouldn’t send that entirely from the host (or using a binmode)? I’ll add it to the list.

2 Likes
70

depends on if the host tools using the binmode support the flash or not. flashrom doesn’t use the newer SFDP standard iirc and requires a definition for each flash by itself. the buspirate-native flasher supports SFDP

2 Likes
71

Copying flash bins to and from the internal storage is certainly a use I envisioned. I’m adding a dump command for @AreYouLoco as I knock out this too long reply.

I was eager to have a flash app in the actual firmware because getting support for new hardware into a lot of the most beloved projects is a bit rocky. I am also not a desktop developer, while Linux is pretty frictionless these days I know very very little of Windows or OSx native development.

Over time, existing support seems to bitrot. The PulseView OLS/Bus Pirate v3.x support for example, it is broken under windows. phdussud did a fix and a custom compile that I upload to our github page while we hope the fixes make their way into the official build. That is why I made the silly VT100 in terminal logic graph viewer.

In summary… in 15 years when Windows 17XXXL is the common OS I want to be able to dump or burn an odd flash chip without fighting to get 15 year old code going. Drop into a terminal, use a stable hardware platform, get the job done.

2 Likes