PowerPC, Emulation, and Transition

Who am I?

Erin Cornelius / ac0rn https://onionshark.com
she/her

If Someone Sounds Like They Know What They Are Talking About...

They are full of shit
They have been working with <thing> for 3+ years
They created <thing>

Why Emulate Something?

Bare metal systems are a PITA
Full system emulation provides for additional analysis and information collection
Allows for more targeted reverse engineering
"It's useful"

2 minutes

We'll get to story time shortly but before that I want to make sure everyone has a basic idea of what the benefits of using a tool like an emulator are

Bare metal systems provide very little visibility into what is going on without sophisticated and expensive tools such as a Lauterbach TRACE32 which are also challenging to set up

Full system emulation provides many opportunities for analysis and collecting information that is useful when reverse engineering

Emulation can be extremely helpful in reverse engineering such as helping to bypass tamper protections or anti-debug features. For example in Linux the PTRACE system call can be used by a process to get debug access to itself and thus prevent a debugger for attaching, or detection of if a debugger is already attached. If we have full system emulation then we can make the system call return whatever we want.

For example vivisect uses it's emulation capabilities to identify where code is in a binary. You point it at a block of memory and it attempts to decode instructions, if it finds valid instructions from the start point to some sort of return instruction then it assumes you have found valid code

Vivisect has the ability to take a function, fills in the register arguments and stack parameters with taint values, then executes the code. Reads and writes to memory are tracked during emulation, and at the end you can see what the effects of the different input taints are. For example if the program counter is set to a value based off of an input parameter then you immediately know that this particular function is has a potential vulnerability in it.

So that's a lot of words to basically say "because it's useful"

Why A New Emulation Tool?

QEMU is popular, but it's codebase is terrible (last I looked)
Existing tools don't make me happy
Because I want to
Also it was my job for the DARPA AMP program

2 minutes

"Ok great" you say, "you've convinced me that emulation is fantastic, but aren't there already existing emulators out there? Why not use them?"

Good question. QEMU is a common tool, the user mode emulation on Linux works really well but full system emulation can be challenging to set up. If you've ever tried to get a Raspberry PI system image running in QEMU you know what I mean. Documentation of how to do things changes, drivers and features get broken by updates. Building on existing complicated open source project can be a challenge.

When I say "codebase is terrible" I just mean in my opinion

We haven't gotten to a particular topic quite yet, but the next reason is because existing tools don't do things quite the way I want, or don't approach problems the way I want them to

So short answer is because I want to

Also I was assigned to do this for work because we were partners with AIS as part of the DARPA AMP program. I'll explain more about that in a second.

Why PowerPC?

Popular in specific embedded controls
Existing emulation tools are limited, no VLE support
I was assigned to for the DARPA AMP program

2 minutes

In embedded devices, "cyber physical systems" if you must, we often see PowerPC in automotive controls and aerospace devices. Those industries have used PowerPC for years and have just kind of kept doing it. Maybe they have a technical reason, maybe it's just because the company is familiar with the architecture? who knows.

There isn't much open source that handles PowerPC well for emulation. With QEMU you can emulate an early 2000's Mac but they don't support features on new chips such as VLE (Variable Length Encoding) instructions - you can think of them like thumb2 for ARM.

As part of my past work experience in those various industries I've done a lot of development, integration and testing on PowerPC platforms, so I've somehow become a bit of an expert in PowerPC. So it makes sense that as I was working at GRIMM when we got assigned to work on a PowerPC emulator as part of the DARPA AMP program that I got assigned to it

My friend Heather says: "If it works, don't breath on it, don't fucking touch it, otherwise it's going to break" (why companies don't like changing things)

AMP

https://www.darpa.mil/program/assured-micropatching

Lift high level code from binaries
Modify that lifted code
Apply patch into binary
Assurance
Test those tools

1 minutes

Let's say you have a critical device you rely on and the vendor goes out of business or maybe news of a vulnerability is published and the vendor refuses to fix it. How do you patch the vulnerability?

Micropatching!

You have to get the firmware, find the bug, make a binary patch that fixes the issue and figure out how to apply it. Typically this requires a subject matter expert.

In short what I have highlighted in red from the DARPA webpage, create capability that can be used for rapid patching of legacy binaries in mission critical systems.

Here's what most of the teams have been doing

I won't go into detail about most of this because it's not what I worked on, and some of the tools are probably going to be talked about at one of the events this week or were last year. I know ofrak was released last summer, angr has a gui now called Angr Management that was initially released in 2019 and improvements have continued to be made. I believe Fish will talk about it his year at defcon also.

Anyways, I find the tools that have been created and improved as part of this project interesting but that's not what I've been working on.

AIS, Cummins, and GRIMM are the team that tests those tools. The end goal is to have the tools find bugs in a real embedded controller. Because Cummins is a partner one of their engine controllers is the target device.

But this controller runs a modern (ish) PowerPC chip that there is no good emulator for. By the end of the project the goal was to have a tool that the different teams could use as a reference, or one that could be used to verify the tools in some fashion.

And that is how I got assigned to create a PowerPC emulator

MPC5674F

Conway's Law

Any organization that designs a system will produce a design whose structure is a copy of the organization's communication structure

- Melvin E. Conway

1 minutes

This can be loosely interpreted to mean that systems tend to mimic the people and groups who design them. This should be fairly obvious, the tools/systems/devices that a person creates tends to mimic how they think and solve problems. The tools I make I know are a reflection of how I think about problems.

Incidentally this is one reason that I encourage people to make your own tool even if there are many existing tools that accomplish the same thing, how you use a tool is closely tied to how you think about problems so if you have a tool that conflicts with how you think about problems it will be difficult to use. It's not doing work someone else has done all over again, it helps you learn about the problem or system or protocol, and you end up with a tool that works well for you.

Anyways, back to Conway's law

When looking back at this project in preparation for doing this talk I was struck by how progress of the emulator seemed to track along with my personal progress of figuring myself out.

How To Build An Emu

^H^H^H^H^H^H

A Gender

20 minutes

So story time! Time to talk about the work I did on this emulator, what I learned, what the challenges have been, and what I expect the future challenges will be

By now I assume it's obvious that I am trans, when I said "figuring myself out" in the previous slide I mean finally coming to terms with the fact that I am a woman and have always wanted to be and finally confronting the question "what should I to do about this?"

You're not gonna get the full nitty gritty because that takes a while and does get a a little personal, already this talk is going to be way more personal than anything I've talked about before

What do these things have to do with each other? Nothing! They just happened to be occurring at the same time in my life

Everything happens at some time

Winter 2014

this is the only surviving selfie I ever attempted to take of myself before coming out, no idea how I didn't delete it. This is really only relevant here because this is around the time I started thinking more in depth about my gender.

Probably worth mentioning that I don't mind old pictures of myself? Like this is a very personal thing for trans people so it's not something everyone is comfortable sharing. I have no shame so it's whatever. I still don't enjoy old pictures but they're interesting to see how I've changed. Also it doesn't look like me, it never felt like me, so it's kind of an abstract thing.

The only reason I like old pictures of me is because of how different they look from me.

Also worth nothing that at this point in time I had already been diagnosed as ADHD as an adult, it took a few years after being diagnosed before really understanding the ways that being ADHD affected me. It's probably obvious that I was not particularly happy when I attempted to take this picture... Yet another reason I'm surprised I never deleted it.

Anyways, part of this story is is about emulation, PowerPC and reverse engineering. The other part of this story is about me essentially starting to recognize patterns in myself and trying to use the information of those patterns to learn more about myself. So let's move on and I'll let you know what I've learned about myself. Or what I learned professionally.

Anyways, enough of that, let's move on.

Summer 2018

Started work on Vivisect PowerPC https://github.com/ehntoo/binaryninja-vle

Winter-Summer 2019

Project to improve Vivisect PowerPC support

Atlas started work on PowerPC support for vivisect but didn't get very far because work kept getting in the way

Mitch (another coworker) also created a PowerPC VLE disassembler plugin for Binary Ninja at the same time. Unfortunately it was based on libvle and there were a few bugs and gaps in that library at the time.

GRIMM got an award from NMFTA to officially add and improve PowerPC support to Vivisect. Because PowerPC is so common in transportation controllers they were attempting to push disassembler/research tools to better support PowerPC.

The image there is an NXP multicore 64-bit PowerPC reference design. It appears designed to be a router platform I guess. But I did a yocto build for it so we could log in to a native PowerPC Linux platform and play around with some native applications and debugging.

I didn't help with the initial code for the disassembly/emulation of the Standard PowerPC instructions but I did help make a bunch of unit tests to ensure disassembly was right, I the output from a few other tools and took the instructions from some random PowerPC binaries so we could compare the output of Vivisect to other tools. I found a few bugs in other tools and there were a few customization's of how we wanted the output to look.

January 2020

Summer 2020

DARPA AMP kicks off

So this is the last picture I have before the pandemic started and I started growing my hair out. I always wanted to grow my hair out but for some reason felt it was silly and I should look "professional" instead. That was a stupid idea, but the pandemic was a good excuse for me to not get a haircut at least.

I'd been working at GRIMM about 3 years at this point and I had mostly overcome my initial impostors syndrome. Hadn't fully grown confident in my ability to teach or mentor yet.

By this point I had finally realized that I was probably autistic. As a result of that realization I had started learning to be gentler with myself. I started recognizing the situations and stresses that would lead me to want to shut out the world and instead of trying to force my self through those situations I learned it was better to get some time to myself, be quiet and just let myself recover first.

Poor Erin had no idea what the future would hold for her. At this point I assumed I was probably nonbinary of some sort but that gender was bullshit anyways and I didn't care.

I do want to emphasize here that this doesn't mean that everyone nonbinary just hasn't made up their mind yet because that is far from the truth. All I'm describing is what I was thinking. I'd ended up on nonbinary because for me it was less scary than accepting that I was trans. And to a large part I wasn't yet ready to accept that I was trans.

So then the AMP project kicks off in the summer of 2020. And my colleagues start working on the project. I was on another project at this point.

Fall 2020

Bare metal emulator framework created

Late Fall 2020

I started working with Matt on the PowerPC emulator.

My colleague Matt created the initial bare metal emulator framework along with defining the way Memory Mapped IO reads and writes will be able to plug into a generic peripheral "module" framework.

Which really meant learning how Vivisect was designed and structured. This took a little while. I still am not great at using Vivisect, but I can write code for it

I will also point out that there are other colleagues who worked on the emulator also. I've been the primary developer for it but others have helped along the way.

In December we hired a new junior researcher, Jordan Cyr, who didn't have much experience with assembly or programming python, but he wanted to learn. So through much of the next year or so I helped mentor Jordan on writing python, how assembly works, using GDB, etc.

April 2021

Created the SWT Peripheral

May 2021

Created the SIU and FMPLL peripherals

May 2021

Software Watchdog Timer

Forced me to create a way to manage and track time within the emulator.

System Integration Unit and Frequency Modulated Phase-Locked Loop (FMPLL)

With those peripherals created we were finally able to emulate the start of the Cummins CM2350 booloader in the emulator.

So now we've got something that should be able to start trying to run the firware of the actual Cummins CM2350 controller. So I extracted the firmware. Actually we already had a firmware that was extracted from another controller already, but I wanted multiple images so I could see what changes between the different versions. Also getting it working with the debugger meant that I could do spot checks on how the actual NXP MPC5674F processor behaved in certain sections of code. Initially I wanted to confirm that the FMPLL and SIU register states matched my first guess at how I implemented the emulation.

May 2021

MMU and virtual memory addressing

June 2021

Flash

August 2021

Interrupt handling & PowerPC Exceptions

October 2021

CAN & External IO peripherals

November 2021

SPI

December 2021

ADC, EBI

Summer - Winter 2021

Integration & Testing on real AMP challenge

If you don't know what CAN feel free to stop by the Car Hacking Village at DEF CON and there will be plenty of people to help you learn

Serial Peripheral Interface. The wikipedia page is a pretty good description of how it works

Analog to Digital Conversion.

External Bus Interface. Used to talk to external RAM on the MPC5674F. On many embedded systems the processor has on-chip SRAM, but usually they can also interface with external parallel bus components such as external SRAM.

Lots of integration and testing on real AMP challenge, these components were the last (ok, just mostly the last) parts bnecessary to run the CM2350 application so after these were implemented I was able to run a demo AMP challenge binary that was created to run on the CM2350

I got the basics of these peripherals done in August 2021, and then spent... well about 5 months actually fully tested and implemented.

On a personal note I think it was around October 2021 that one of the people I followed on twitter just happened to mention that trouble forming long-term memories was a symptom of cPTSD. Up until this point I had assumed it was just because I was ADHD and tended to be forgetful. Like I really don't remember much of my life. So I finally started to grapple with the idea that maybe I had cPTSD.

January 2022

Bare metal PPC firmware analysis in Vivisect

February 2022

Emulator moved to github!

https://github.com/Assured-Micropatching/CM2350-Emulator

February 2022

Added raw PowerPC binary contained in an ELF file support to vivisect

With the addition of a these peripherals I now had the minimum necessary functionality to try and get CM2350 application to run!. Most of these were initially implemented in August it just took quite a few months to actually fully test the peripherals and work the bugs out. The peripherals were getting more complicated.

Move to github and split emulator and vivisect back into their independent parts

Also early in Feburary I finally accepted myself and came out to my partner, this feels very symbolic in a silly way how the timeline aligns with the emulator project but sometimes things just happen at the same time as other things.

You'll notice from here that my timeline gets a little less populated with technical stuff, that's somewhat because as the emulator became more functionally complete and complicated it both takes more time to figure out how best to fit new functionality into the structure, and also a lot of the work started to be small fixes and changes that are less easy to make a bullet point for.

Also this talk is already in danger of going too long...

March 2022

The first week of March I had a training to teach to company, and had to decide if I wanted to hide or not. But I knew how my brain works by this point and if I didn't come out right now I knew that it would never feel like the right time.

Also I was so happy! I wanted to share the reason for my joy.

Lastly there were some people I followed on twitter who had come out publicly and I knew that I found them being out to be motivational. I travel quite a bit for work and perhaps if I was out and loud about who I was I could help other people feel like they aren't alone.

My first work trip after coming out was for the AMP program to CSU and it fell right on top of Trans Day of Visibility. Well I had already come out on twitter and linkedin, no way I wasn't going to dress as myself!

Buuuut I was incredibly nervous, as I said I'd come out twitter, and linked in, to people I worked with, but that wasn't to people who I knew but saw maybe one or two times a year so far on this project. This was a whole new challenge.

But it was amazing, and as you see from the picture here that day felt incredible.

This was the first day I wore a skirt professionally! It was a good day

May 2022

June 2022

Placeholder
peripherals

DC30

Summer - Fall 2022

Floating Point Instructions

Summer - Fall 2022

Vector Instructions

November 2022

SIM

Fall 2022 - Winter 2023

DMA & Interrupts

By this point just about every PowerPC instruction was completed but we floating point instructions still needed to be implemented. Remember that this isn't just the PowerPC standard floating point instructions, it also included the NXP custom SPE floating point instructions. This was work that Jordan worked on, so in addition to learning how to write python, and how to disassemble binaries, and emulate assembly instructions, he was now getting to learn how floating point representation works

Another colleague, Dan Fego, joined us on the emulator project briefly and had fun writing the standard PowerPC vector instructions, and the NXP custom SPE vector instructions.

Jordan wrote his first PowerPC emulator peripheral

Direct Memory Access. When implementing increased the complexity of how interrupts could also be used to trigger automatic reads and writes of memory. This one took a while to fully implement and test.

March 2022 - 2023

Remote GDB server

April 2023

Accurate Timing

May 2023

External Watchdog

The simple way we did timing early on ended up not being good enough when needing to track events accurately across both external peripherals (like the External Watchdog) but also because the emulator can't execute instructions at the same relative speed as the real hardware so the internal timed events can end up happening too soon.

The last change made in AMP Phase 2 was some of the board-level integration envisioned from the start. The CM2350 has some hardware that causes the CPU to get reset if it hasn't taken the correct action recently enough. This is common in a lot of embedded devices, there are on-chip watchdogs that help ensure the software is working properly and then typically another watchdog that sits external to the processor to handle the condition where the internal software keeps the watchdog alive but some higher-level task processing is not working properly.

Phase 3

Performance Improvements
A few peripherals still using placeholders
Improve ease of use
Needs more documentation
Abstract generic emulation framework

So that's where the emulator stands at the end of Phase 2 of the AMP project.

DARPA projects typically have 3 phases, the first is doing a basic proof of conept the second phase is the bulk of the work and the third phase is the "transition" phase where the funding is less but the goal is that other industry partners will start to take an interest in the things being developed and invest in them.

Get it? Now we're in the transition phase?

There are quite a few smaller things that we hope to get done in phase 3, or outside of the AMP program. It feels like there is a lot more to go but really it's basically done.

If you are changing less than 10% of the overall functionality of a thing, you are done. It doesn't feel like you are done, but you are done enough to let other people mess with it.

Conclusion

Emulator framework that works well for me
The emulator provides a bunch of hooks and features for additional analysis
I've learned a great deal about disassembly, emulation
Learned more about PowerPC than I really wanted to know

Don't be afraid to make your own tool

More Conclusions!

I like being me for the first time in my life
I enjoy life?
(Is this how life is supposed supposed to feel?)

PowerPC, Emulation, and Transition

Who am I?

If Someone Sounds Like They Know What They Are Talking About...

Why Emulate Something?

Why A New Emulation Tool?

Why PowerPC?

AMP

MPC5674F

MPC5674F

Conway's Law

How To Build An Emu

^H^H^H^H^H^H

A Gender

Winter 2014

Summer 2018

Winter-Summer 2019

January 2020

Summer 2020

Fall 2020

Late Fall 2020

April 2021

May 2021

May 2021

May 2021

May 2021

June 2021

August 2021

October 2021

November 2021

December 2021

Summer - Winter 2021

January 2022

February 2022

February 2022

March 2022

May 2022

June 2022

DC30

Summer - Fall 2022

Summer - Fall 2022

November 2022

Fall 2022 - Winter 2023

March 2022 - 2023

April 2023

May 2023

YOU ARE HERE

Phase 3

Conclusion

Don't be afraid to make your own tool

More Conclusions!

It's never too late to be yourself

It's never too late to love yourself