Matasano Chargen - Latest Comments in Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

TK — Mon, 05 May 2008 13:07:04 -0000

Once again you guys produce an article so exceptional on detail and techniques that I am at a loss for words... Excellent job Eric!

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

Marcin — Thu, 01 May 2008 15:09:07 -0000

Why yes, I just noticed your post! Very cool.

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

Eric Monti — Thu, 01 May 2008 15:04:47 -0000

Marcin, I noticed that too.

I just posted up a 0.9.1 version that fixes this and adds the deezee tweak:

http://www.matasano.com/log/1048/blackbag-091-n...

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

Marcin — Thu, 01 May 2008 14:44:45 -0000

Btw, I ran into compilation problems on Linux before:

tsec.c: In function ‘main’:
tsec.c:44: error: too few arguments to function ‘setpgid’
tsec.c:44: error: too many arguments to function ‘setpgrp’
make: *** [tsec.o] Error 1

So in tsec.c, you'll want to change line 44:

if(setpgrp(0, setpgid()) == -1) {

to:

if (setpgid(getpid(),0)) {

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

Eric Monti — Thu, 01 May 2008 13:59:14 -0000

@Bill:

Wow, thanks for all the VxWorks info. I've been hunting for all I can dig up. This definitely helps clarify and solidify some of what I've been reading about.

As you have inferred, there were several signs of a shell component present in my binary. The VxWorks relationship between shell and symbol table is one of the reasons why I mentioned that finding VxWorks symbol tables might be more common than on other targets. I wasn't sure enough about this to assert that here till you mentioned it.

Anyway... for the rest of us mortals:

This FAQ helped me some:
http://www.xs4all.nl/~borkhuis/vxworks/vxfaq.html

I also found the VxWorks reference site from Colorado Space Grant Consortium pretty handy:

http://spacegrant.colorado.edu/~dixonc/vxworks/

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

Bill Paul — Wed, 30 Apr 2008 23:33:07 -0000

The image you have comes from VxWorks 5.5.1. I can tell by the pixels... er, I mean, I can tell by the GNU ld version string. Each full version of the VxWorks development platform ships with a compiler suite which is usually new for that particular release (if there's new arch support in the release, the compiler has to be updated). Starting with 5.5.1, there are two compilers available: the GNU compiler tools and the Diab tools. (The Diab suite is proprietary and costs extra.)

Also starting with VxWorks 5.5.1, all objects and images on all architectures are in ELF format. In earlier versions, there were some architectures that used ELF, but some used a.out. That said, you don't always end up with an ELF image in your toaster, because there's more than one way to embedded VxWorks in a given device, and exactly how it's done is up to the designer.

During development, a device will use a VxWorks bootrom to load the runtime image. The bootrom is usually written to flash and is the first code the CPU runs when it comes out of reset. The bootrom is really just a special instance of VxWorks that has only one job, which is to initialize the hardware and load a runtime image either from local storage (flash TFFS filesystem) or the network. (Ok that's actually two jobs, but whatever.) Being able to download a new image into the system is handy during development since it saves you from having to keep writing new images into flash all the time.

If you want, however, you can also create a VxWorks ROMable image, which allows you to do without the bootrom. Instead, your image is written directly to flash and is the only code the CPU runs. It can be configured as totally ROM resident (code and initialized data stays in flash/ROM/etc...) or it can be made relocatable, where the image is copied into RAM after basic initialization.

If you use a ROMable image, then there is no image format: the resulting ELF image is converted to a flat binary file (and the symbol table is discarded).

(It is also possible for a sufficiently clever toaster maker to create their own bootloader, since you can very easly create flat binary files from any VxWorks image.)

As for the symbol table, there are two cases where you will find one. One is if you have an unstripped ELF image. (For some reason, nobody seems to care about stripping the symbol tables from VxWorks binaries.) The other is if you have a VxWorks image that contains the shell component. Not every VxWorks image include a shell, since some embedded applications don't have any I/O devices, but if you do. However, if it is included, the shell interpreter in VxWorks is really nothing more than a function caller: when you type a command on the shell, the shell looks for a symbol name in the image that matches in the symbol table, translates the symbol name into an address, and then just calls it. This is very useful during development, since it means you can directly invoke functions from the shell to see if they work as expected. Anyway, in this case, the symbol table is not the ELF symbol table: it's a separate one that's embedded in the image at compile time. (I think this is so that the symbol parser will work even if the ELF symbol table has been stripped.)

Note that is is possible to build a VxWorks image that includes the shell component, but which does not include the embedded symbol table.

Lastly, sysInit usually is the entry point for an image. Though for a bootrom or ROMable image, the entry point is usually called romInit (which is the CPU reset entry vector).

It's hard to tell if the manufacturer of your toaster is using a regular VxWorks image or a ROMable one. I'm a little surprised the full VxWorks banner is present, because the banner component is usually only pulled in if your image include the shell. And I can't imagine why a toaster would need a shell.

-Bill

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

Robert 'Groby' Blum — Wed, 30 Apr 2008 23:22:24 -0000

Damn you for releasing just enough details that searching for the "toaster" is interesting ;)

Narrowed it down to two candidates, now trying to get my hands on them to see if their firmware matches your description....

Somebody stop me and have me do something meaningful instead, please! ;)

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

Cody — Wed, 30 Apr 2008 15:54:44 -0000

Great post. Doing these kinds of things is always fun and refreshing.

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

Antonio — Wed, 30 Apr 2008 15:11:41 -0000

Great post - makes me pine for the days when I used to reverse software as a hobby (for an old school cracking group).

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

AlexG — Wed, 30 Apr 2008 14:28:40 -0000

If I'm not mistaken, you could also find that the beast's web interface is controlled by GET requests. Yes, state changes in response to GET. Though I haven't been near the thing for 3 years, that might've been fixed by now.

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

Eric Monti — Wed, 30 Apr 2008 14:05:26 -0000

@Marcin:

I wasn't "suggesting" quite that. That puny one line patch for deezee is my sole contribution to blackbag to date.

But... stay tuned. Something along these lines is in the works, kind-of. That's all I'll say for now.

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

Eric Monti — Wed, 30 Apr 2008 12:00:12 -0000

@igorsk: I read your series of blog posts on the Kindle a while back. Great, great work! I've been checking back on your blog frequently hoping for more posts.

I'd particularly love to hear more about the Sony Reader, if you've got anything to tell.

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

Kevin — Wed, 30 Apr 2008 11:23:09 -0000

Great article. I really enjoyed reading about your process.

Kevin

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

igorsk — Wed, 30 Apr 2008 07:03:23 -0000

Nice writeup! I did some similar work on Amazon Kindle, though in that case the hardest part was getting the firmware image.
http://igorsk.blogspot.com/2007/12/hacking-kind...

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

Marcin — Wed, 30 Apr 2008 01:31:53 -0000

The latest version of blackbag I have is at http://www.sockpuppet.org/blackbag-0.9.tgz

Eric, are you suggesting a 1.0 release?

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

Chris — Tue, 29 Apr 2008 21:58:25 -0000

Good post Eric. blackbag tools are very handy

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

Eric Monti — Tue, 29 Apr 2008 21:42:18 -0000

Interesting side note: Mind you, I've not looked at a whole lot of VxWorks stuff out there. But I get the strong impression that symbol tables like the one I found are not so uncommon in VxWorks applications.

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

Eric Monti — Tue, 29 Apr 2008 21:36:16 -0000

@Nate: Networked... vxworks... mips... big endian... narrows it down too much you think? Damn, this is why we can't have nice things on our blog =(

@Tolomea: I spent about half a day working on reversing the image file headers and writing tools for working with them so I could install my patched "toaster". Another half a day looking at the actual application binary and getting it from raw binary to full disassembly with symbol names.

@Taco Master: Great question! A new link for blackbag will be back up soon. We'll post it here for sure.

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

Thomas Ptacek — Tue, 29 Apr 2008 21:12:27 -0000

Nate: I can name 3 off the top of my head, none of which make toast, so, good luck. =)

I think the symbol addresses were changed to protect the innocent.

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

Nate — Tue, 29 Apr 2008 21:01:13 -0000

Nice work! Now we only need to figure out which networked device is VxWorks + MIPS-BE and has a matching symbol table. Guesses below?

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

Taco Master 3000 — Tue, 29 Apr 2008 19:18:25 -0000

Where can we get a copy of Black Bag and Deezee?

Re: Retsaot is Toaster, Reversed: Quick ‘n Dirty Firmware Reversing

Tolomea — Tue, 29 Apr 2008 19:04:49 -0000

Out of curiosity, roughly how long did the work described above take to complete?