Differential FW update

for GPRS application

  • 1
  • 2
  • Page 1 of 2
Marco
Benutzer
Avatar
Gender: n/a
Location: Italy
Posts: 270
Registered: 10 / 2004
Subject:

Differential FW update

 · 
Posted: 05.03.2012 - 12:23  ·  #1
Hello,
in my new GPRS application I have necessity to reduce at the minimum the quantity of bytes transferred for a "firmware update over the air".
For what I have understood this technique seems based on a differential concept; the hex file is not an image of the program memory, but contains only the sections which are different from previous fw version, and the informations about where these various patches must be applied.

So, is it possible instruct the compiler to assign absolute addresses to new procedures, leaving the previous version almost unchanged ?

Regards,
Marco
Merlin
Administrator
Avatar
Gender:
Age: 24
Posts: 1373
Registered: 03 / 2005
Subject:

Re: Differential FW update

 · 
Posted: 05.03.2012 - 15:37  ·  #2
Hi Marco

It can be done in assembler of course using eg ORG statements. But you should not do this in AVRCo.

I think that even if you could do it, it would be a nightmare to administer. What would you do when the size of a procedure gets too big, and what if a system function changes because of a bug fix?

Also, how do you handle skipped updates? eg if someone has had their device turned off for an extended period?
Marco
Benutzer
Avatar
Gender: n/a
Location: Italy
Posts: 270
Registered: 10 / 2004
Subject:

Re: Differential FW update

 · 
Posted: 08.03.2012 - 22:43  ·  #3
Hi Merlin,
sorry for late reply, but I was in holidays last days...

I agree it's not easy to manage the various Fw versions, but I cannot believe there is not a way to optimize at compile time the hex file...
For example if I have a mega256 with 140 KB used by firmware; if now I add a simple procedure() I must reflash almost all pages !!?

About the example above, a simple raw option I can imagine is to put the new procedure on top of previuous image file,
changing only the pages where it is called.
I understand it's "simple to say", but the result is that you have to send only few hundred of bytes instead of say 141Kb !

Of course this is a problem only when you pay for the bytes transferred (via GPRS), but when you have thousands of devices to upgrade
this can become a big problem...
Also when your competitors in a possible project claim to have such kind of feature to offer...

RFC,
Marco
Avra
Schreiberling
Avatar
Gender:
Location: Belgrade, Serbia
Age: 53
Homepage: rs.linkedin.com/in…
Posts: 653
Registered: 07 / 2002
Subject:

Re: Differential FW update

 · 
Posted: 08.03.2012 - 23:55  ·  #4
Maybe this is not a job for compiler, but for a boot loader that upgrades the firmware:

1) bootloader reports firmware version
2) If firmware needs an upgrade custom tool creates diff file between old firmware and new
3) diff file is sent to bootloader
4) bootloader does what it's paid for and reports results
;-)

Bootloader might solve step number 4 in two ways. To reprogram only changed pages from diff file, or to first reconstruct new firmware by applying diff file to old firmware file and then reprogram as usual (some sort of CRC should be reported and matched with CRC of new firmware before actual reprogramming). First way can be done online page by page, but for second way you would need some external extra storage space.
Merlin
Administrator
Avatar
Gender:
Age: 24
Posts: 1373
Registered: 03 / 2005
Subject:

Re: Differential FW update

 · 
Posted: 09.03.2012 - 10:12  ·  #5
Hi Avra.

The main problem that I see with this is the way that the assembler is organised. User code, which is the most likely to change (and least likely to be the same size) is organised at the front of the code. Everything else is shuffled down. That means that the 'difference' is likely to be most of the code.

I agree 'bootloader' is the answer, but I think that either you need some way to make the functions likely to change a fixed size (by, for example padding with NOP statements), which in larger processors should not, in principle at least, be a problem, or something smarter than a simple difference engine, i.e. one that than can calculate how to move existing blocks. The latter though also requires a pretty smart bootloader. You may need, at different times of day, to transmit different upgrade paths to handle jumps in versions as I mentioned before.

The problem, as I see it is not just to make a difference engine, but to make the differences themselves as small as possible.
Marco
Benutzer
Avatar
Gender: n/a
Location: Italy
Posts: 270
Registered: 10 / 2004
Subject:

Re: Differential FW update

 · 
Posted: 09.03.2012 - 12:16  ·  #6
Hi all,
I agree that bootloader must do some work (different than usual) to apply the new FW version, and Avra's ideas about the ways to accomplish this are interesting, but I think it's the "easier" part of the job.
As say Merlin, the difficult is find a way to keep the differences to minimum, still from compile time.
And here I Hope we can find the "Columbus egg" :-)

I am thinking to 3 situations starting from an already in place ver 1.0 and trying to upgrade at 1.1;
a) adding 1 or more new procedures -> should be done always in the next free pages
b) modify an existent procedure in one smaller -> should overwrite the existent.
c) modify an existent procedure in one bigger -> should leave there the old and place the new in the next free pages.

I think to something similar to a FAT system, where deleted files (older routines for us) are left in place for undelete purposes.
So newer files (new routines) can reuse that available space if convenient, otherwise place the file in new sectors (pages).
There will be a moment when a complete erase and reflash of the firmware will be necessary due to excessive fragmentation,
but before reach that point I think will be possible apply many differential updates !

Marco
Merlin
Administrator
Avatar
Gender:
Age: 24
Posts: 1373
Registered: 03 / 2005
Subject:

Re: Differential FW update

 · 
Posted: 09.03.2012 - 14:07  ·  #7
Hi Marco.

Situation (a) is more difficult than it seems, ignoring for the moment that it does not at comply with AVRCo. Something (maybe many things) will need to call it and these areas will also need to be updated.

Situation (b) is quite simple and OK.

Situation (c) suffers the same difficulties as situation (a) so padding (to effectively change situation (c) into situation (b) might be a better option.

Now, as I say, adding procedures at a specific page does not comply with how AVRCo works, so how could one get around that with AVRCo? Well, if you know in advance that the procedure will exist (for example with situation c) you could use indirection, which AVRCo supports, to point to the functions and then update the pointers. If all pointers are kept in the same area, then effectively moving the functions to a new page would be quite easy (well, relatively speaking!) However, you would have to be sure that you always call a function via its pointer and not directly! The first version could be a true AVRCo program and subsequent ones could be updates.

Sensibly the boot loader should be aware of these pointers, otherwise v1.1 applied directly will have a different image to v1.1 applied over the air, so an upgrade to v1.2 might fail for one or the other. However this might make the boot loader too big for the boot loader space. I don't have enough experience with boot loaders or XMEGAs to know if that is an issue or not, but I am thinking that with care this can be circumvented and that the 'intelligent' part of the boot loader can exist outside boot space, calling functions within the bootloader space to actually update the flash.
Marco
Benutzer
Avatar
Gender: n/a
Location: Italy
Posts: 270
Registered: 10 / 2004
Subject:

Re: Differential FW update

 · 
Posted: 11.03.2012 - 10:28  ·  #8
Hi Merlin,

a) of course this situation (adding new procedures) means that also other parts of the source are changed. not only for the calls to the new procedures, but much probably also some code that will use that new procedure.
So having to reflash also these pages I think it's not a problem, because I still save a lot of pages (data traffic).

c) If I must modify an existent procedure and it becomes bigger, I think it's hard reduce it to situation b) isn't easier to fully place it in the free area on top ?

But I think before going in deep discussions, we must understand by Rolf what can be done at compilation time with some kind of $ switch...

Marco
  • 1
  • 2
  • Page 1 of 2
Selected quotes for multi-quoting:   0

Registered users in this topic

Currently no registered users in this section

The statistic shows who was online during the last 5 minutes. Updated every 90 seconds.
MySQL Queries: 15 · Cache Hits: 14   134   148 · Page-Gen-Time: 0.025679s · Memory Usage: 2 MB · GZIP: on · Viewport: SMXL-HiDPI