In a couple weeks at Recon I'll be giving a presentation with a colleague on how to do similar things with Python. We go over extracting and modifying code from .pyc and .pyd files, writing our own assembler and disassembler, as well as possible anti-reversing techniques. Although I haven't looked at EVE yet, I will be demonstrating many cheats I've injected into the Pirates of the Caribbean Online MMORPG. The abstract for our talk is located here: http://recon.cx/2008/speakers.html#python.
-- Aaron
Nate
· 1 year ago
Nice. I prefer to use read/write watchpoints though since you don't have to figure out what code is involved in descrambling the data. All you need is the input and output buffer.
sapphirecat
· 1 year ago
The m68k had a trace bit, but as far as I remember, only line-by-line tracing (and not breakpoints) trigger it. The old versions of AmigaDOS used it to block tracing of supervisor code, but you could still execute that code without giving up control of your program, using the 'run-to-line' feature of the debugger.
Effectively, unless the computer can securely watch itself (which the trace bit was a small step towards), there is no way to actually secure it.
Rolf
· 1 year ago
You're mostly right about the lameness of script obfuscators, but:
"The problem with Perl source filters is an example of a problem that afflicts all high-level language runtimes: if you want your code to run, you have to expose it to the interpreter."
For instance, PHP internally contains a function pointer called "execute" that can be replaced at runtime to allow for a custom interpreter. Ergo the PHP bytecode that you see in memory post-decryption is not "typical" bytecode and will not run if passed to the standard interpreter. That said, these protections still tend to be lame -- they usually just use the standard interpreter with some XORs thrown in here and there for obfuscation.
Mongo
· 1 year ago
> Get used to that idea, or go back to writing in C.
or write in a high level language that can compile to C or compile directly to native code.
Eric Monti
· 1 year ago
Rolf,
The PHP example you pose is coincidentally one close to home. You are right, de-obfuscating encrypted intermediate bytecode is a somewhat different beast than straight source. More so if the bytecode is handled in a proprietary manner. But many of the same principles still apply.
I chose the Perl example since it was simpler to illustrate against. However, we've had some similar experience with a popular PHP obfuscation framework a few months back based on crypto with a semi-proprietary intermediate bytecode format as well.
When we got to the point where we were plucking byte-code out of memory instead of actual source code, it became a matter of decompiling the byte-code. Definitely an extra step, and I'll concede, a somewhat non-trivial one. (having some "patience" might apply here)
Having spent that additional time reversing, we were ultimately able to discern the program structure almost completely back to original code, including original variable names and, under the right build conditions, even comments.
I should point out, also. We took more or less the same approach regarding the crypto; side-stepping the "encryption" altogether and shooting straight for byte-code while it was exposed in memory when fed to the engine's interpreter.
Earle Martin
· 1 year ago
Next time you may wish to try "perl -MO=Deparse obfuscated_file.pl" first.
lulz
· 1 year ago
Earle Martin, you're not supposed to point out B::Deparse or really any other Perl info (and not Immunity Debugger info). Imagine the humor we would have lost if Eric Monti and others had known that! The "When all you have is a hammer, everything looks like a nail. Let's show off our hammer!" Matasano approach to work time and blogging is better for everyone: more entertainment for observers and more rewarding for Matasano authors in the blogosphere.
Daniel Reynaud
· 1 year ago
I made a small attempt at formalizing the use of an interpreter as an obfuscation step, the slides of the presentation can be found here :
The idea is that if you want to publish some high-level code without actually disclosing the source code, you "just" have to compile it to bytecodes and apply low-level obfuscations on this code. Correctly implemented, this should really add a layer of complexity.
Looks like there is an upcoming presentation at REcon dealing with the same topic :
@Earle: B::Deparse is probably a valid approach to try for this. The reason I went completely underneath Perl framework to an OS-level user-land debugger was because the instance of perl the target used was packaged with the target, not just the crypto extension. There was a mix of several opensource and proprietary perl extensions to wade through and I didn't particularly want to rely on yet another 3rd party add-on which might or might not build and install, or even work at all, in the target's perl environment. Not knocking it, just not my cup of tea.
Alex Radocea, our super-intern, also pointed out another route I could have gone was to use Perl's own debugging mode, "perl -d". This, I think, is even simpler if you're thinking of staying inside the perl framework. But had I done so I still would have had to do some reversing of the framework to disable the source-filter's own perl debugger check.
@lulz: Yes. My "hammer" is frequently a debugger. I don't really have strong feelings about Immunity, gdb, Windbag, olly, softice, ida, python/ruby libraries, versus something I write myself. Given the choice, I'll use something I know of and know will work if its available. (If I don't have to write something new, so much the better).
Repeat: We used the almost the same exact method on a (technically very different) PHP obfuscation scheme.
Looks like nail? Have hammer? Hit with hammer! Hit until nail-thing go down.
I'm pretty ok with that actually.
Showing off a "hammer" was not the point of the post ,though. Still, I'm glad to see you're so easily amused, lulz.
Thomas Ptacek
· 1 year ago
Like the Matasano approach of writing debuggers and then blogging about them instead of trying Deparse and giving up ("no findings, all clear!") when it doesn't work? We're hiring.
Eric Monti
· 1 year ago
@Daniel:
This filter just happened to be a funny target because of the free perl source code thrown in at the end.
I wouldn't suggest that reversing a proprietary bytecode format is going to be as easy as the perl source filter example I posed. But I will say it's probably not going to be as hard as the obfuscation designer imagines it to be.
Hands down, Perl source filters of the variety I looked at are a very naive approach to code obfuscation. As approaches get smarter, very soon the reverser ends up dealing with bytecodes or "intermediate bytecodes" somewhere a few steps up in abstraction from machine instructions. Once out of de-obfuscation (if there is any -- in some cases, bytecode may be comparable to machine instructions, so why obfuscate?), if the attacker ends up staring at proprietary bytecode, their considerations will likely include designing a disassembler.
Depending on their ultimate goal and the likeliness of re-using the attack in the future, it may just make more sense to just trace machine instructions and render that back up to an any abstract flow picture to make sense out of.
I've not ever had to take it to this point. Hmm but I could imagine cases where I might?
For instance: Jeff Goldblum, **MY_REVERSING_IDOL**, almost certainly did this to stave off planetary annihilation in Independence Day. Take that, nasty space monsters!!!
Anyway, there's certainly a line somewhere between the effort of obfuscation versus the effort required to reverse it, and dont forget factoring in the value of what is obfuscated. The line is where obfuscation may have a *partial* pay-off. If the strength of deterrence is greater than the value of what's protected, seems like a good pay off.
At what point does the obfuscator win? Not my call, but it's an interesting question for the pundits.
lulz
· 1 year ago
Hammers are cool.
B::Deparse isn't another third-party tool that might not compile - it is maintained and packaged with perl. If there were technical limitations that excluded the use of it in your case, that could have made for a more interesting post. Instead your post suggests that the natural recourse, in general, is to jump to a debugger. It kind of caters to those who know little on the topic and can admire your hammer slamming, while leaving Perl users asking why you missed the obvious.
Thomas: Come now. I advocate practical solutions; there is intelligence in simplicity. Although not very illustrious, simple solutions gained from basic research can not only be easier, but can often gain much better results (e.g. valid Perl code instead of asm) from building off the years of hard work that others have invested in an issue. I'm not saying you should not be capable of doing your own work.
Thomas Ptacek
· 1 year ago
lulz --- I take your point. I don't think Eric can go into the details of why Deparse wasn't going to work here (it would betray the actual target). I'm just reacting to your "there's the Matasano way --- go write your own debugger and then blog about it! Good technique if what you care about is blogging!"
What our clients care about is that we find stuff. What our team cares about is that they aren't forced to run and interpret Webinspect for 80 hours a week.
Also, Eric didn't write a debugger --- he used PyDbg. He's gotten to write debuggers on other projects. I've done a few. We have an intern writing one now. We're debugger-happy. How does that sound to you, readers? We're hiring.
Eric Monti
· 1 year ago
lulz,
1. I'm aware of B::Deparse and have used it in the past. It may come packaged with your stock perl distro, but it is definitely not "built-in" to Perl. I *like* shortcuts too!
2. This should be abundantly clear by now: B::Deparse was *not part* of the perl environment I targeted. Would *you* release your own Perl dist with obfuscation packaged with Deparse?
3. You will more than likely *never* read anything written by me describing how I installed and ran a Perl module to achieve an objective unless it is something totally out of left-field. Not saying "I'd never do that sort of thing." I'm saying, "I probably won't write posts about that."
(That said, you'll probably read me describing the same under Ruby or Python... go figure!)
All Comments
In a couple weeks at Recon I'll be giving a presentation with a colleague on how to do similar things with Python. We go over extracting and modifying code from .pyc and .pyd files, writing our own assembler and disassembler, as well as possible anti-reversing techniques. Although I haven't looked at EVE yet, I will be demonstrating many cheats I've injected into the Pirates of the Caribbean Online MMORPG. The abstract for our talk is located here: http://recon.cx/2008/speakers.html#python.
--
Aaron
Effectively, unless the computer can securely watch itself (which the trace bit was a small step towards), there is no way to actually secure it.
"The problem with Perl source filters is an example of a problem that afflicts all high-level language runtimes: if you want your code to run, you have to expose it to the interpreter."
For instance, PHP internally contains a function pointer called "execute" that can be replaced at runtime to allow for a custom interpreter. Ergo the PHP bytecode that you see in memory post-decryption is not "typical" bytecode and will not run if passed to the standard interpreter. That said, these protections still tend to be lame -- they usually just use the standard interpreter with some XORs thrown in here and there for obfuscation.
or write in a high level language that can compile to C or compile directly to native code.
The PHP example you pose is coincidentally one close to home. You are right, de-obfuscating encrypted intermediate bytecode is a somewhat different beast than straight source. More so if the bytecode is handled in a proprietary manner. But many of the same principles still apply.
I chose the Perl example since it was simpler to illustrate against. However, we've had some similar experience with a popular PHP obfuscation framework a few months back based on crypto with a semi-proprietary intermediate bytecode format as well.
When we got to the point where we were plucking byte-code out of memory instead of actual source code, it became a matter of decompiling the byte-code. Definitely an extra step, and I'll concede, a somewhat non-trivial one. (having some "patience" might apply here)
Having spent that additional time reversing, we were ultimately able to discern the program structure almost completely back to original code, including original variable names and, under the right build conditions, even comments.
I should point out, also. We took more or less the same approach regarding the crypto; side-stepping the "encryption" altogether and shooting straight for byte-code while it was exposed in memory when fed to the engine's interpreter.
http://tcv.loria.fr/slides/reynaud-obfuscation-...
The idea is that if you want to publish some high-level code without actually disclosing the source code, you "just" have to compile it to bytecodes and apply low-level obfuscations on this code. Correctly implemented, this should really add a layer of complexity.
Looks like there is an upcoming presentation at REcon dealing with the same topic :
http://recon.cx/2008/speakers.html#virtualmachines
Alex Radocea, our super-intern, also pointed out another route I could have gone was to use Perl's own debugging mode, "perl -d". This, I think, is even simpler if you're thinking of staying inside the perl framework. But had I done so I still would have had to do some reversing of the framework to disable the source-filter's own perl debugger check.
@lulz: Yes. My "hammer" is frequently a debugger. I don't really have strong feelings about Immunity, gdb, Windbag, olly, softice, ida, python/ruby libraries, versus something I write myself. Given the choice, I'll use something I know of and know will work if its available. (If I don't have to write something new, so much the better).
Repeat: We used the almost the same exact method on a (technically very different) PHP obfuscation scheme.
Looks like nail? Have hammer? Hit with hammer! Hit until nail-thing go down.
I'm pretty ok with that actually.
Showing off a "hammer" was not the point of the post ,though. Still, I'm glad to see you're so easily amused, lulz.
This filter just happened to be a funny target because of the free perl source code thrown in at the end.
I wouldn't suggest that reversing a proprietary bytecode format is going to be as easy as the perl source filter example I posed. But I will say it's probably not going to be as hard as the obfuscation designer imagines it to be.
Hands down, Perl source filters of the variety I looked at are a very naive approach to code obfuscation. As approaches get smarter, very soon the reverser ends up dealing with bytecodes or "intermediate bytecodes" somewhere a few steps up in abstraction from machine instructions. Once out of de-obfuscation (if there is any -- in some cases, bytecode may be comparable to machine instructions, so why obfuscate?), if the attacker ends up staring at proprietary bytecode, their considerations will likely include designing a disassembler.
Depending on their ultimate goal and the likeliness of re-using the attack in the future, it may just make more sense to just trace machine instructions and render that back up to an any abstract flow picture to make sense out of.
I've not ever had to take it to this point. Hmm but I could imagine cases where I might?
For instance: Jeff Goldblum, **MY_REVERSING_IDOL**, almost certainly did this to stave off planetary annihilation in Independence Day. Take that, nasty space monsters!!!
Anyway, there's certainly a line somewhere between the effort of obfuscation versus the effort required to reverse it, and dont forget factoring in the value of what is obfuscated. The line is where obfuscation may have a *partial* pay-off. If the strength of deterrence is greater than the value of what's protected, seems like a good pay off.
At what point does the obfuscator win? Not my call, but it's an interesting question for the pundits.
B::Deparse isn't another third-party tool that might not compile - it is maintained and packaged with perl. If there were technical limitations that excluded the use of it in your case, that could have made for a more interesting post. Instead your post suggests that the natural recourse, in general, is to jump to a debugger. It kind of caters to those who know little on the topic and can admire your hammer slamming, while leaving Perl users asking why you missed the obvious.
Thomas: Come now. I advocate practical solutions; there is intelligence in simplicity. Although not very illustrious, simple solutions gained from basic research can not only be easier, but can often gain much better results (e.g. valid Perl code instead of asm) from building off the years of hard work that others have invested in an issue. I'm not saying you should not be capable of doing your own work.
What our clients care about is that we find stuff. What our team cares about is that they aren't forced to run and interpret Webinspect for 80 hours a week.
Also, Eric didn't write a debugger --- he used PyDbg. He's gotten to write debuggers on other projects. I've done a few. We have an intern writing one now. We're debugger-happy. How does that sound to you, readers? We're hiring.
1. I'm aware of B::Deparse and have used it in the past. It may come packaged with your stock perl distro, but it is definitely not "built-in" to Perl. I *like* shortcuts too!
2. This should be abundantly clear by now: B::Deparse was *not part* of the perl environment I targeted. Would *you* release your own Perl dist with obfuscation packaged with Deparse?
3. You will more than likely *never* read anything written by me describing how I installed and ran a Perl module to achieve an objective unless it is something totally out of left-field. Not saying "I'd never do that sort of thing." I'm saying, "I probably won't write posts about that."
(That said, you'll probably read me describing the same under Ruby or Python... go figure!)