lopqto's adventures

Punched In: Extracting Firmware from an ESP8266 RFID Terminal

Thu, 12 Feb 2026 03:30:00 +0330

I was looking to buy a time tracker device for the office. In my searching process I found a very cheap device that immediately triggered my curiosity. As you may already know by reading my older posts, I love dissecting this kind of hardware (unknown & random manufacturer), so I decided to purchase one just to reverse it.

The Device

This device is a time tracking device. After you turn it on for the first time, it starts a wireless access point. You can connect to it and open the control panel to configure your office’s WiFi credentials. After the reboot it will act as a WiFi client and connect to the AP.

Each user has a unique RFID card, and they will scan it using the terminal. The terminal will send the RFID tag number plus the timestamp to a remote API.

Hardware Parts

After I configured the WiFi credentials and restarted the device, I headed to the DHCP page of my router to find the IP address of the device so I can open the control panel. In that list of entries, I found something very interesting:

That was the AHA moment!

As per Wikipedia’s description:

The ESP8266 is a low-cost Wi-Fi microchip, with built-in TCP/IP networking software, and microcontroller capability, produced by Espressif Systems

It makes sense. To keep the hardware cost very low, they used a very cheap microcontroller with WiFi connectivity and to compensate for that, they delegated all the main functionalities like leave tracker, timesheet, etc. to a remote API.

I decided to open the device and take a look at it:

To be honest, I felt down a bit. The hardware is far from an intricate device and more like a hobby project.

Let’s review:

Microcontroller: Wemos D1 Mini -> an ESP8266-based development board.
Display: 1602 LCD with I2C Backpack
RFID Reader: RC522 (MFRC522) - datasheet
Base Board: Custom “Shield” / Carrier PCB -> Acts as a “motherboard” to fix the messy jumper wires issue. It breaks out the pins from Wemos D1 Mini to the specific headers for the RFID and LCD.
LED Indicators

I decided to focus on the firmware instead of hardware and go as deep as possible.

Getting the Firmware

We can easily see the mini USB port that’s connected to an AC socket to act as the power source.

We can connect the Wemos to our PC using a simple mini USB cable, and the dumping part would be a piece of cake for these reasons:

Hardware Disabling: You cannot disable the port via software. You must physically destroy the UART traces (TX/RX lines) or remove the USB-to-Serial IC (e.g., CH340) after flashing.
Flash Encryption: The ESP8266 does not support hardware flash encryption. If someone has physical access, they can bypass the USB port and wire directly to the SPI flash chip pins to dump data.
eFuse Limitations: Unlike the ESP32, the ESP8266 lacks the flash_crypt_cnt or JTAG_DISABLE eFuses.
Physical Protection: The only real protection is potting the board in epoxy to prevent physical probing of the flash memory chip. Which in this case, they didn’t!

To verify my guess, I connected the microcontroller to my PC and checked the flash-id

esptool --port /dev/cu.usbserial-A5069RR4 flash-id

To Dump it:

esptool --port /dev/cu.usbserial-A5069RR4 read-flash 0 0x400000 wemos_firmware_dump.bin

Note: Wemos D1 Mini has exactly 4MB of memory, so we read it all.

And finally to verify the dump:

esptool --chip esp8266 image-info wemos_firmware_dump.bin

Cool, we got the entire (probably) unencrypted firmware. Let’s move to the next part!

Reading the Binary

I found this guide that describes ESP8266’s processor architecture like this:

The ESP8266 has an Xtensa lx106 processor at its core. This is a 32 bit RISC processor with 16 registers.

In the previous step we used esptool to analyze the image info. So obviously, esptool knows how to parse the image.

I digged a little bit deeper in the esptool source code and found this bin_image.py file which is responsible to parse the headers.

By looking at the file I found these headers:

Common Header:

magic, segments, self.flash_mode, self.flash_size_freq, self.entrypoint = struct.unpack("<BBBBI", load_file.read(8))

Which is equivalent to:

struct header {
    u8  magic;
    u8  segments; // count
    u8  flash_mode;
    u8  flash_size_freq;
    u32 entrypoint;
};

Segment Header:

(offset, size) = struct.unpack("<II", f.read(8))

Which is equivalent to:

struct segment_header {
    u32 offset; // load address
    u32 size;
};

And for the segments, data comes right after the segment header so we can write it like:

struct segment_header {
    u32 offset;
    u32 size;
    u8  data[size]; 
};

I dug way deeper and wrote a pattern file for the entire image with some helper variables for ImHex and sent a pull request (PR #492) for it. At this moment the PR is still open.

Pattern file

After verifying everything about the legitimacy of the firmware and writing a pattern file for it, we are ready to load it in Ghidra for further analysis.

Into Ghidra

Ghidra already supports Xtensa pretty well (added in version 11.0) and can disassemble and generate pseudocode for these processors easily. The issue resides with the custom proprietary firmware format of ESP8266. Ghidra must know where to look, where to disassemble as code, and where to look for data. For these kinds of situations, Ghidra uses loaders.

Ghidra doesn’t have a native loader that understands ESP8266 firmware. Unlike standard formats like ELF or Mach-O, ESP8266 uses a proprietary binary format with a custom header structure featuring:

An 8-byte header with magic byte (0xe9) for identification
Segment-based layout where each section has its own offset and size
Multiple headers including a user ROM header at offset 0x1000
Specific memory mappings for IROM, user code, and data segments

Without a proper loader, Ghidra would need you to manually extract and position each binary segment into the correct memory regions. The loader handles all this automatically by parsing the firmware structure and creating properly mapped memory blocks with correct permissions.

I found an older ESP8266 loader here!, but the codebase was outdated and needed updates. So I decided to send a pull request (PR #6) to contribute the fixes back to the community.

After installing and enabling the custom loader, we can easily import the dumped firmware into Ghidra:

Loader parses the firmware headers like what we did in the previous sections, then creates memory layouts and fixes the permissions:

Address	Name	Loader Section Name –	Size	Description
0x3FFE8000	dram0	`.user_data`	14000h	User data RAM. Available to applications
0x40100000	iram1	`.user_code`	8000h	Instruction RAM. Used by bootloader
0x40200000	SPI Flash	`code`	-	SPI Flash is mapped here

After loading the firmware into Ghidra, we can see that it was able to analyze the firmware. One of the simplest ways to verify this is by looking at the strings. Seeing meaningful strings usually means that the loader did its job and was able to map the data sections to the correct memory addresses.

If we head to the function lists, we can spot the other issue. Ghidra was unable to find function signatures using their signature db and all of the functions are labeled like FUN_*.

To solve this issue we can use a FidDb. It’s a database file used by Ghidra’s Function ID (FID) analyzer to identify and match functions across different binaries.

Because we are in microcontroller land, usually we are limited in terms of public FidDbs. For that reason, we need to create one ourselves.

To do that, we need to compile another firmware that uses same SDK and libraries with debug symbols on. To build a firmware for an ESP8266, there are two viable options:

Arduino IDE + ESP8266 SDK
PlatformIO + ESP8266 SDK

We can have an educated guess and based on the hardware design, guess that they used Arduino IDE to develop the firmware because it is much easier and user friendly.

You can read about installing ESP8266 SDK for Arduino IDE here!

Generating function signatures heavily depends on the SDK and library version. Even minor version differences can affect function signatures due to compiler optimizations or code changes. While ESP8266 SDKs share core functionality across versions, you shouldn’t assume signature compatibility. To be honest, you cannot expect 100% function signature coverage. FidDb is a helper utility to make the reversing process easier, not an ultimate solution. That’s why I decided not to play the cat-and-mouse game. of hunting down the exact SDK version to match the FidDb perfectly. The reality is that even with a perfectly matched FidDb, there will always be functions that need manual analysis. Since ESP8266 firmware is relatively small and the FidDb provides useful partial matches even across versions, I chose to just use what’s available and move forward with the actual reverse engineering. The time spent hunting for the exact SDK version is better spent on the analysis itself.

I created this simple sketch:

#include <ESP8266WiFi.h>
#include <ESP8266WebServer.h>
#include <ESP8266HTTPClient.h>
#include <ESP8266mDNS.h>
#include <WiFiClient.h>
#include <WiFiUdp.h>
#include <DNSServer.h>

void mega_stub() {

  // Never execute
  if (false) mega_stub();
}

void setup() {}
void loop() {}

Exporting it generates two files: .bin and .elf. We need to load the .elf one because it has the debug symbols.

I then loaded it in Ghidra, let Ghidra analyze it, and generated a FidDb out of it.

Steps:

Create an empty FidDb: Tools -> Function ID -> Create new empty FidDb
Populate the FidDb: Tools -> Function ID -> Populate Fid Database (Use Xtensa Little Endian for the language field)

After that, we can head to the dumped firmware and attached the created FidDb (Tools -> Function ID -> Attach existing FidDb). Make sure to let the Ghidra to analyze it again!

Tadaaa!

With meaningful function names now identified, Ghidra can show us the actual structure of the firmware. Instead of cryptic FUN_40200000 addresses, we now see recognizable functions like setup(), loop(), WiFi_init(), and others.

Everything is ready for our adventure.

What’s Next

In Part 1, we identified the hardware, successfully dumped the firmware and made some helper utilities along the way. We created a pattern for ImHex, and were able to load the firmware in Ghidra and adjust it so we can reverse it later with more ease.

I think we are at a good point now and ready to take the next step. In the next post (that hopefully I can allocate some time to write soon), we will take a look at the actual firmware to see how it actually works, if there are any vulnerabilities with it and what hidden functionalities exist. We will talk about ESP8266 firmware much deeper.

Feel free to ping me if you had any questions! Peace!

References & Tools

Via Negativa: On Reducing Entropy in Systems

Mon, 22 Sep 2025 00:30:00 +0330

On the way home after a short vacation, the peace of a bus ride sparked a reflection. Opening my notebook, one thought stood out: stability, both in my role as a DevSecOps engineer and in personal life, had come simply by subtraction rather than addition.

What is Via Negativa?

Via Negativa, “the negative way”, is a philosophical and theological idea. Leaving theology aside, I like the philosophy behind it: a way of thinking that removes what is false or unnecessary.

As Taleb put it:

Progress is often made by removing harmful things, not adding more.

Whether we call it Via Negativa, minimalism, essentialism, or the Pareto principle (80/20), the message is the same: remove the unnecessary, and what remains is what truly matters.

Subtraction vs. Addition

Addition is seductive. It is measurable, tangible, visible.

Add another CI/CD pipeline. One more security scanner. Another policy, rule, habit-tracking app, supplement, or note-taking tool.

The world rewards addition:

Stack more.
Optimize more.
Buy more.
Squeeze every last drop of time and energy to do more.

Addition feels productive because the results are visible.

Subtraction, on the other hand, feels empty. You remove something, and it looks like loss. It feels like doing less, being less. It goes against what the world advertises.

But subtraction has one thing addition lacks: it reduces entropy.

Less moving parts.
Less bugs.
Less crashes.
Less alerts.
Less noise.
Less distraction.
Less wasted energy.

And with that reduction comes stability.

Subtracting as an Engineer

In the world of technology, the desire to add surrounds us. I have seen it, and I have done it. We get a new security tool, and it feels like progress. We add a new dashboard with dozens of graphs, and we feel more in control. For a while, this works. We are catching more things, we have more “visibility.” It feels like a win for the whole team.

Then the noise begins. The new tool sends a message for every little thing that occurs. The dashboard is so cluttered that nobody ever looks at it anymore. The CI/CD pipeline is now a muddle of scripts that only one person fully comprehends. The important signals are drowned out by the constant noise.

The bravest engineering decision is often not to add a new feature, but to delete a redundant one. It’s the courage to say, “We don’t need this alert,” or “Let’s simplify this pipeline.” When we eliminate the noise, we can now listen to the signal. The result is a less noisy, more focused team that can respond to actual problems faster. We design systems that are not only robust but also simple to understand. Simplicity provides us strength.

Enjoying the Sunset

This philosophy of subtraction is not just limited to our systems at work; it’s extended to our lives.

It starts by removing the unneeded, the applications that don’t assist you, the alerts that get in the way of the here and now. The quiet at first feels awkward. The absence of the continuous hum of input is frightening. Our brains, trained for distraction, don’t know what to do with quiet.

But soon that emptiness manifests as empty space. Time itself becomes open. A gap of five minutes is no longer just time to glance at a phone, but time to complete a whole page in a notebook. A simple meal, untouching of a screen, is a whole experience.

The purpose of this idea, this subtraction exercise, is not to be an machine of maximum efficiency. It’s the opposite. It is to be a space quiet enough to finally hear your own mind. To be the author of your own day, not a reactor to the demands of a thousand notices. It is to understand that by removing the things that consumed your time, you finally have time to live..

I experienced it myself, resting on a bench as the sun set, music in the background, a tree’s shadow dancing on the wall, nothing urgent, just the quiet fullness of the moment.

Life is simple. Living is even simpler. The hard part was trusting that less could be enough.

Closing Toughts

Subtraction isn’t glamorous. It doesn’t win you quick applause. It seems to be like doing less, being less.

But it is subtraction that reduces entropy, in systems, and in life. And clarity appears expensive when entropy is low.

And occasionally the most heroic act of design, as an engineer or in life, is not what we add, but what we have the courage to subtract.

Reversing a Remote Controller: A Case Study in RF Engineering

Sat, 16 Aug 2025 03:30:00 +0330

Feeling bored, I decided to take on a personal challenge: reverse-engineer a toy’s remote controller. My goal was to gain practical experience with radio communication and create a custom controller with new features, like controlling it remotely over Wi-Fi. It’s a classic hands-on project that starts with a simple question: what makes this thing tick?

Exploring the Target Device

I had a toy with a remote controller laying around. The remote controller has 3 buttons for 3 different functions: power, change the rhythm of the bottom part, change the rhythm of the top part. I was suspecting that the remote controller uses radio waves to communicate with the toy like other similar remote controllers.

After opening the remote controller, My suspicion was confirmed.

A few things immediately jumped out that pointed toward a specific frequency. The board is labeled AYKJ006-TX, with TX being the standard abbreviation for a transmitter. More importantly, the distinct, curvy copper line labeled ANT is clearly a PCB trace antenna. Simple, short-range remotes like this almost always operate in the unlicensed ISM bands, with 433 MHz being one of the most common frequencies worldwide. This made 433 MHz my primary suspect.

Let’s look at the components.

The Encoder and Transmitter IC (U1)

The most important component on this board is the large black chip labeled U1. It has 2 primary functions:

Encoder: When user presses a button, it converts that action into a command.
Transmitter: After generating the command, the chip modulates it onto a 433 MHz radio frequency carrier wave.

The Crystal Oscillator (Y1)

The shiny, metallic component labeled Y1 is a crystal oscillator. It generates a stable clock signal that is used by the main chip (U1).

If you look closely, you will notice it’s marked “13.560,” which means it vibrates at 13.560 MHz and not 433 MHz. This is a common engineering trick. The main chip (U1) contains a circuit called a Phase-Locked Loop (PLL), which acts as a frequency multiplier. It takes the stable 13.560 MHz signal and multiplies it by 32 to generate the final transmission frequency:
```
13.560MHz × 32 = 433.92 MHz
```
It’s often cheaper and more stable to use a lower-frequency crystal and multiply it up.
The User Interface Components

There are 3 buttons for 3 different functions: power, change the rhythm of the bottom part, change the rhythm of the top part labeled as S1, S2, and S3. Also there are 4 different LEDs labeled as D1, D2, D3, and D4. They provide a visual feedback when the button is pressed.
Power Management Components

Battery Terminals (BAT+ / BAT-): These are the metal contacts where the battery (CR2032) is connected to power the entire circuit.
Voltage Regulator (U2): Its job is to take the variable battery voltage and provide a constant, clean voltage (3.3V) that the main chip (U1) needs to operate on.

Planning the Next Steps

My goal was to reverse engineer the remote controller and create a custom one that can control the device. I needed to understand the communication protocol used by the remote controller to be able to replicate those signals. To achieve this, I had 2 options:

Using a Logic Analyzer: A logic analyzer is a tool that captures and displays digital signals within an electronic circuit. To use this method, I would have had to solder tiny wires directly to the pins of the encoder chip (U1). This would let me “eavesdrop” on the exact digital commands the chip generates when a button is pressed, before they are turned into a radio wave.
- Pros: It provides a very clean, noise-free look at the digital data.
- Cons: It requires delicate soldering skills, risks damaging the board, and tells you nothing about the radio transmission part of the signal (the modulation).
Using a Software Defined Radio (SDR): An SDR is essentially a radio scanner for a computer. It’s a device that can be tuned to almost any frequency to receive raw radio signals directly from the air. I could use it to capture the exact 433 MHz signal the remote sends out, just as the toy’s receiver would hear it.
- Pros: It requires no physical modification to the remote. It captures the complete, final signal as it’s transmitted.
- Cons: The captured signal can have noise from other radio sources, and it requires software on a computer to process and decode the raw radio data back into a digital command.

Since I had an SDR device lying around and wanted a practical project to learn more about radio hacking, I decided to go with option 2.

Capturing the Radio Waves

To capture the radio waves, I plugged my RTL-SDR device into my computer and used a software suite called Universal Radio Hacker (URH).

The Universal Radio Hacker (URH) is a complete suite for wireless protocol investigation with native support for many common Software Defined Radios.

With the software running, I configured my setup to get the cleanest signal possible:

I tuned the SDR to the frequency of the remote (433 MHz).
Next, I adjusted the gain. Since the remote was right next to the SDR’s antenna, the signal was very strong. A high gain would only amplify unnecessary background noise, so I lowered it significantly.
Then, I created a ground plane to improve reception. For the simple monopole antenna that comes with most SDRs, placing it on a metal plate (or any large metal surface) acts as a ground plane, which helps stabilize the signal and reduce interference.
Finally, I hit the record button in URH to start capturing.

Voila! I had a clean, noise-free sample of the transmission. Now I could jump into the next part.

Probing for a Rolling Code

With everything ready, I began transmitting. The button for the toy’s top part (S1) cycles through four different rhythm modes. To capture the full sequence and check for consistency, I pressed the button eight times, ensuring I recorded two full cycles of all four modes.

There was a critical reason for this repetition. I needed to check if the protocol had any “moving parts,” like a rolling code or a timestamp. More secure systems, like car key fobs, change the code with every press to make replay attacks impossible. By capturing two sets of signals for each of the four modes, I could compare them. If the first signal for “mode 1” was identical to the second signal for “mode 1,” it would mean the code is static, and a simple replay attack would likely work.

This was a make-or-break step. If the code wasn’t static, I’d have to figure out how to generate the new code for each transmission. That would likely mean reverse-engineering the U1 chip itself, and an SDR isn’t the right tool for that kind of task.

URH attempts to automatically analyze the signal and show the bits. In my case, its interpretation wasn’t perfect, but it didn’t need to be. It was clear enough to see that the payload was static. After each cycle, it sent the exact same payload for that mode.

As you can see, in the following image, searching for the single pattern resulted in 2 different groups of matches.

That was the confirmation I needed. The device uses a static code, which means it’s completely vulnerable to a replay attack. Game on.

A Quick Note on Security: It’s worth pointing out that finding a “vulnerability” like this in a simple toy isn’t an indictment of the manufacturer. Implementing secure rolling codes (cryptography) adds complexity and cost. For low-cost consumer electronics, manufacturers make a deliberate trade-off, opting for simple, static codes that are cheap and reliable over robust security. This is expected and perfectly normal for this class of device.

Demodulating the Signal

In simple terms, modulation is the process of encoding digital data (our ones and zeros) onto a radio wave, and demodulation is getting that data back off the wave at the other end.

Figuring out the modulation type was straightforward once I looked at the captured signal. The pattern was a dead giveaway for Amplitude Shift Keying (ASK), one of the simplest forms of digital modulation.

The key thing to look for with ASK is the “on-off” nature of the transmission. When I viewed the signal in URH, I didn’t see a continuous wave that changed its shape. Instead, I saw a series of clean, distinct bursts of radio energy separated by total silence. This is the classic signature of ASK, meaning the transmitter sends a 1 by turning the 433 MHz carrier wave ON (high amplitude) and sends a 0 by turning it OFF (low amplitude, or silence).

But how does it know how long each 1 or 0 should be? That’s where the encoding scheme comes in.

What is a PWM?

The specific method this remote uses to structure its ones and zeros is a form of Pulse Width Modulation (PWM). Don’t let the name intimidate you; the concept is really simple. It uses pulses of different lengths (or widths) to represent different data.

Think of it like sending a message with a flashlight in the dark:

To send a 1, you might do a long flash followed by a short pause.

To send a 0, you’d do a short flash followed by a long pause.

The receiver on the other end just has to measure the duration of each flash and each pause to reconstruct the original message. My remote does the exact same thing, just with radio waves instead of light and with timings measured in microseconds. By analyzing the captured signal, I could see two distinct pulse lengths, a short one and a long one, confirming that this was the method being used.

Dissecting a Command

With the encoding method figured out, I could finally dissect a complete command. My first step was to lean on URH’s built-in analysis tools, which gave me a bird’s-eye view of the entire transmission structure.

A single press of a button wasn’t just one command; it was a whole conversation. The remote was clearly designed for reliability, sending its message multiple times to make sure the toy heard it correctly. The structure for every button press was identical:

1001111111111111111111 [Pause: 1898 samples]
111110111110111110111000000011100000001111101110000000111110111000000011100000001110000000111110111110111000000011111011100000001010001111111111111111111 [Pause: 1897 samples]
111110111110111110111000000011100000001111101110000000111110111000000011100000001110000000111110111110111000000011111011100000001010001111111111111111111 [Pause: 1897 samples]
111110111110111110111000000011100000001111101110000000111110111000000011100000001110000000111110111110111000000011111011100000001010001111111111111111111 [Pause: 1898 samples]
111110111110111110111000000011100000001111101110000000111110111000000011100000001110000000111110111110111000000011111011100000001010001111111111111111111 [Pause: 1895 samples]
11111011111011111011100000001110000000111110111000000011111011100000001110000000111000000011111011111011100000001111101110000000101 [Pause: 490234 samples]

The Preamble: A short, distinct initial pulse. This acts like a wake-up call for the receiver, letting it know a real command is about to follow and allowing it to synchronize its clock.

1001111111111111111111 [Pause: 1898 samples]

The Command Repeats: The main data payload was sent back-to-back four times. This is a common strategy in simple RF devices. If the first transmission is garbled by interference, the receiver has more chances to get a clean copy.

111110111110111110111000000011100000001111101110000000111110111000000011100000001110000000111110111110111000000011111011100000001010001111111111111111111 [Pause: 1897 samples]
111110111110111110111000000011100000001111101110000000111110111000000011100000001110000000111110111110111000000011111011100000001010001111111111111111111 [Pause: 1897 samples]
111110111110111110111000000011100000001111101110000000111110111000000011100000001110000000111110111110111000000011111011100000001010001111111111111111111 [Pause: 1898 samples]
111110111110111110111000000011100000001111101110000000111110111000000011100000001110000000111110111110111000000011111011100000001010001111111111111111111 [Pause: 1895 samples]

The Trimmed Packet: The transmission ended with a final, shorter burst of data. This was a truncated version of the main command, likely acting as a message over signal.

11111011111011111011100000001110000000111110111000000011111011100000001110000000111000000011111011111011100000001111101110000000101 [Pause: 490234 samples]

With the overall structure understood, I could focus on one of those identical, repeated blocks. By analyzing the repeating patterns of high (1) and low (0) samples, I could deduce the encoding “alphabet” for a single bit of data:

Logical 1: A long high pulse followed by a short low pulse. In the sampled data, this corresponds to the pattern 111110.
Logical 0: A short high pulse followed by a long low pulse. In the sampled data, this corresponds to the pattern 1110000000.

With these rules, I could now decode the full data payload. For example, applying these rules to one of the captured packets for the “top part” function gave me the 16-bit binary message:

1110 0101 0001 1010

By doing this for all the captured modes, a clear pattern emerged.

The Final Command Format

The command is a 16-bit (2-byte) packet with a clear and simple structure.

[ Address Nibble (4 bits) | Command Nibble (4 bits) | Checksum (8 bits) ]

Address Nibble: A 4-bit chunk that acts as a group identifier. For example, all commands for the “top part” used the address 1110 (Hex: 0xE). This tells the receiver which function to modify.
Command Nibble: The next 4-bit chunk is the actual instruction. For the “top part,” this value changed for each of the four rhythm modes, telling the receiver which specific pattern to activate.
Checksum: The final 8 bits are a simple checksum. It’s a bitwise NOT of the first 8 bits (the address and command combined). Every 1 is flipped to a 0, and every 0 is flipped to a 1. This is a computationally cheap way for the receiver to verify that it received the message without errors.

Here is a breakdown of the example command we decoded (1110010100011010):

Part	Binary	Hex	Meaning
Address Nibble	`1110`	`0xE`	Address for the “top part” function
Command Nibble	`0101`	`0x5`	Instruction for a specific rhythm mode
Full Byte 1	`11100101`	`0xE5` The complete command byte
Checksum	`00011010`	`0x1A` The bitwise NOT of `0xE5`, used to verify the message

Timing is Everything!

Knowing the sequence of ones and zeros is only half the job. To build a perfect clone, I had to replicate the signal’s timing with microsecond precision. A radio receiver is incredibly picky; if your pulses are too long or too short, it will reject the command as noise.

While a logic analyzer is the ideal tool for getting perfect, noise-free timing measurements directly from the hardware, it’s possible to get a very good estimate from a clean radio capture in URH. By zooming in on the demodulated signal, you can measure the number of “samples” for each pulse and pause. By establishing a base time unit (e.g., one sample equals ~5µs at a sample rate of 200k), you can calculate the approximate duration of each part of the signal.

Time per sample = 1 / Sample Rate
1 / 200,000 = 0.000005s or 5µs

This method requires some trial and error. I started with the calculated estimates and then “hand-tuned” the microsecond values in my code, testing each time until the device responded reliably.

Creating a Custom Remote Controller

With the protocol fully reverse-engineered, the next step was to build the hardware. The goal was to create a new remote from scratch that could replicate the toy’s signals.

For the main controller, I chose a NodeMCU ESP8266 board. It’s inexpensiveyet powerful (and also has a built-in WiFi module), which is great for this kind of hardware project. For the transmitter, I used a standard 433 MHz ASK module, the same type found in the original remote.

Finally, a crucial component was a decoupling capacitor. During early tests, I discovered the transmitter’s power draw was causing signal instability. Adding a capacitor directly across the transmitter’s power pins provides the instant current needed for clean, strong pulses.

I soldered all components onto a piece of perfboard to create a compact and sturdy device. The NodeMCU is the main board, with the transmitter and its supporting capacitor placed nearby.

The final step was to translate our findings into C++ code for the NodeMCU. This involved creating functions to precisely replicate the signal’s unique structure.

The Building Blocks: Timing is Key

The foundation of the code is built on generating pulses with microsecond accuracy. The delayMicroseconds() function is perfect for this, but to ensure perfect stability and prevent timing jitter from the microcontroller’s background tasks, each pulse is sent inside a “critical section” by temporarily disabling interrupts.

The core of the signal is the PWM “alphabet” we discovered, which defines a logical 1 and a 0. The timings were hardcoded directly from our analysis:

// Timings in microseconds
const int PULSE_SHORT_US = 250;
const int PULSE_LONG_US = 520;
const int PAUSE_SHORT_US = 40;
const int PAUSE_LONG_US = 680;

void send_logical_1() {
  // A long HIGH pulse followed by a short LOW pause
  send_high_pulse_critical(PULSE_LONG_US);
  delayMicroseconds(PAUSE_SHORT_US);
}

void send_logical_0() {
  // A short HIGH pulse followed by a long LOW pause
  send_high_pulse_critical(PULSE_SHORT_US);
  delayMicroseconds(PAUSE_LONG_US);
}

Assembling the Full Command

With the functions for sending a 1 and a 0 defined, the full command structure could be built. Each transmission starts with a unique preamble, which acts as a “wake-up” call for the receiver.

// Preamble timings in microseconds
const int PREAMBLE_PULSE_1_US = 70;
const int PREAMBLE_PAUSE_US = 160;
const int PREAMBLE_PULSE_2_US = 1860;

void send_preamble() {
  send_high_pulse_critical(PREAMBLE_PULSE_1_US);
  delayMicroseconds(PREAMBLE_PAUSE_US);
  send_high_pulse_critical(PREAMBLE_PULSE_2_US);
}

Finally, the main function assembles the full 16-bit packet. It takes the 4-bit address and the 4-bit command, combines them into a single byte, and calculates the checksum by performing a bitwise NOT. It then sends the full sequence: the preamble, four full repetitions of the data packet with its trailer, and a final trimmed packet.

void send_command() {
  // Hardcoded for one of the "top part" rhythm modes
  byte address_nibble = 0b1110; // Address for the "top part"
  byte command_nibble = 0b0110; // Command for a specific mode

  // Construct the full 8-bit command
  byte byte1 = (address_nibble << 4) | command_nibble;
  // Calculate the checksum
  byte checksum = ~byte1;

  // Send the preamble once
  send_preamble();
  delayMicroseconds(PAUSE_INTER_PACKET_US);

  // Send the command + checksum + trailer 4 times
  for (int i = 0; i < REPETITIONS; i++) {
    send_byte(byte1);
    send_byte(checksum);
    send_trailer();
    delayMicroseconds(PAUSE_INTER_PACKET_US);
  }

  // Send the final trimmed packet (command + checksum only)
  send_byte(byte1);
  send_byte(checksum);
}

After uploading the code and powering it on, it worked flawlessly. The custom-built remote could control the toy’s functions just like the original, now functioning as a platform for future expansion.

I did the same thing for the other 2 buttons as well and created a fully functional replica.

Conclusion

This project was more than just building a new remote; it was a deep dive into the practical side of reverse-engineering. I started with a simple problem and a basic tool—an SDR—and worked step-by-step to a complete solution. By looking closely at the signals, I uncovered the device’s secrets: a static protocol, ASK modulation, and a simple PWM encoding scheme. From there, I was able to build the same logic into a new device.

The main takeaway is that even with simple tools, you can figure out how complex systems work. The static code made the project much easier to tackle, and the bit-flipping checksum was a great example of a simple but effective way to check for errors. This hands-on process strengthened my understanding of radio communication and embedded systems, giving me a solid base for more complex projects in the future. Especially with radio communications!

References

Tools

From Laziness to Control: Reversing an IoT device using Frida

Tue, 13 Jun 2023 03:30:00 +0330

Have you ever had that irresistible urge to take apart a cheap IoT device you bought from a random store? You know, that feeling that says, “Let’s dive into the mystery and figure out how this thing actually works!” But then you stop and ask yourself, “Why am I even doing this?” and struggle to come up with a good reason. Well, let me tell you, if there was a Ph.D. for taking apart useless things, I’d probably have earned it!

Today, we’re going to explore the exciting world of reverse engineering, with a focus on understanding the device’s communication protocol. And here’s my little secret: this is my go-to trick when I’m feeling a bit lazy. Get ready for some tech magic, my friends!

Exploring the Target Device

Alright, let’s kick off our journey by addressing the question: What are we going to reverse engineer? Last week, I purchased an LED strip stand (as shown in the image), which came with a controller and a questionable phone app. From my experience, these types of apps usually communicate through Bluetooth. However, I’m not entirely certain about the controller itself. It could be using infrared, Bluetooth, or maybe even something entirely different (but probably not)?

Now, here’s the deal: I want to ditch the app because it’s been tickeling me crazy. To achieve this, I need to uncover the commands that are being sent from the app to the LED strip. Once I have a clear understanding of these commands, I can rewrite them in a programming language that I’m comfortable with. So, our goal is set: let’s crack the code.

Choosing the Right Approach

In order to achieve our goal of taking down the app and gaining control over our LED strip, we have several options available. Let’s explore each one:

Dump the LED strip’s firmware and reverse it: This approach involves extracting and analyzing the firmware of the LED strip itself. However, it requires extensive knowledge of hardware and may consume a significant amount of time and effort. Considering the complexity and potential challenges, this approach may not be the most convenient choice for a simple strip light.
Dump the controller’s firmware: This method involves extracting and examining the firmware of the controller. Similar to the previous approach, it requires hardware knowledge and can be time-consuming. While it provides valuable insights, it may not be the most suitable option for our scenario.
Use a magical device to sniff Bluetooth communication: This option involves employing a specialized device to intercept and analyze the Bluetooth communication between the app and the LED strip. Unfortunately, obtaining such a device might be difficult or costly, making it less reliable for our purposes.
Dissect the mobile app and reverse engineer it: This is the approach I have chosen, and let me tell you why. Tackling the mobile app allows us to dive into its inner workings, understand the commands it sends to the LED strip, and ultimately rewrite those commands according to our preferences. It’s a more accessible option, requiring reverse engineering skills but without the hardware-intensive aspects of the other methods.

Considering my inclination toward efficiency and laziness, focusing on dissecting and reverse engineering the mobile app appears to be the most favorable path forward. Let’s dive into the app’s code and unveil its secrets!

When we’re faced with a big challenge like reading lots of messy code written in Java or Kotlin, it can be really tiring to go through it all by hand. But don’t worry, I’ve got a solution that’s both efficient and effective. Our main goal is still clear: we want to figure out how the app communicates and extract the important commands, without getting too caught up in the app’s inner workings.

Here’s where dynamic analysis comes in. It’s a smart technique that lets us skip the hard work of reading all that code manually. Instead, we can focus on the most important parts. And to make things even easier, we’ll use Frida, a handy tool that can help us with our analysis.

By using dynamic analysis and Frida, we’ll be able to work smarter, not harder (I made up this excuse to cover up my laziness :d ). We’ll be able to find the key details of the communication protocol without getting lost in all the other stuff. Get ready to see how this combination of analysis and Frida can make our job much simpler and more efficient.

Environment Setup

Let’s set up the environment for our project. Here are the steps:

Rooted Android device: We’ll need a real Android device that has been rooted for this project. Emulators won’t work because they don’t support the Bluetooth stack. Some emulators might have a simulated Bluetooth connection, but we need to connect to a real device. Also, install the HappyLighting app.
ADB (Android Debug Bridge): We’ll use ADB to communicate with the Android device. You can install ADB on Debian-based Linux distributions by running the following command:

sudo apt install adb

Note: Make sure you have enabled Developer Options on your Android device.

Frida: We also need to install Frida (You need Python) on both the computer and the Android device.

On your computer, run the following command to install the frida-tools:

pip install frida-tools

On the Android device, first download the appropriate frida-server binary for your device’s CPU architecture from here!. Then, push the frida-server to the device by running these commands:

adb push /path/to/frida-server /data/local/tmp/
adb shell chmod 755 /data/local/tmp/frida-server
adb shell /data/local/tmp/frida-server &

To verify that everything is working correctly and that you can communicate with the frida-server, run the following command on your computer:

frida-ps -U

You should see a list of running processes on your Android device.

With these steps completed, we have our environment set up and ready to go.

Reversing the Communication Protocol

Finally, we’ve reached the exciting part where we dive into reversing and getting our hands dirty. But before we begin, let’s discuss our approach. If you’re already familiar with Frida and how it works, great! If not, take a quick look here! to get acquainted.

In simple terms, our goal is to find the specific function responsible for sending commands to the LED strip device, hook it using Frida, and log those commands. This way, we can reimplement a basic Bluetooth client and send the commands ourselves.

Now, you might be wondering how we can achieve this without delving into the complexities of reversing the Android application. Well, let’s take a step back and think outside the box for a moment. Do we really need to locate specific functions? In fact, what is the “right” function we want to hook?

To answer that, we need to consider a different perspective. We’re interested in the last function in the chain of the functions that handles and sends the commands to the device. Why the last one? Because we want to capture the raw payload being sent to the device, including any checksums, encodings, or padding applied. We need the raw payload as our starting point. We can capture that payload, send it to the device, and see if it works. If it does, great! If not, we can backtrack and analyze the chain further.

So, what is this last function responsible for sending the Bluetooth payload? Well, you guessed it! It’s the Java Bluetooth function from the Android SDK. Ultimately, Java and the Android SDK are responsible for delivering the payload to Android’s Bluetooth stack. Therefore, we can conveniently ignore all the code and simply hook the Java Bluetooth function. That’s all there is to it. We can reverse the communication protocol without actually having to reverse the entire application (if we’re fortunate enough that capturing and emulating the raw payload works for us).

With this approach, we’re ready to uncover the secrets of the communication protocol and create our own Bluetooth client. It’s time to embark on this reverse engineering journey and see where it takes us!

To implement hooking with Frida, we need to find the Bluetooth function’s signature. After some searching, we come across the android.bluetooth.BluetoothGattCharacteristic class, which has a function called setValue responsible for sending payloads through Bluetooth. The function signature is as follows:

public boolean setValue (String value)

Take your time and look at the examples provided by Frida docs here!. Now, let’s write a proper hook for the setValue function using Frida:

Java.perform(function () {
    
    var BluetoothGattCharacteristic = Java.use('android.bluetooth.BluetoothGattCharacteristic');
  
    // setValue(String)
    BluetoothGattCharacteristic.setValue.overload('java.lang.String').implementation = function (value) {
      console.log('[BluetoothGattCharacteristic.setValue] Value (String): ' + value);
  
      // Call the original method and return the result
      var result = this.setValue(value);
      return result;
    };

  });

This code sets up a hook on the android.bluetooth.BluetoothGattCharacteristic.setValue() function. Whenever this function is called, it will log the inputs (hopefully the commands we’re looking for) to the console.

However, we notice that there are three more overloads for this function, as mentioned in the Android Developer documentation here!.

In Java, method overloading is the ability to define more than one method with the same name in a class. The compiler is able to distinguish between the methods because of their method signatures1. This means that multiple methods can have the same name as long as the number and/or type of parameters are different. Method overloading is mainly used to increase the readability of the program; to make it look better. For example, instead of defining two methods that should do the same thing, it is better to overload one.

To ensure we don’t miss anything, let’s implement hooks for these overloads as well:

Java.perform(function () {
    var BluetoothGattCharacteristic = Java.use('android.bluetooth.BluetoothGattCharacteristic');
  
    // Overload 1: setValue(String)
    BluetoothGattCharacteristic.setValue.overload('java.lang.String').implementation = function (value) {
      console.log('[BluetoothGattCharacteristic.setValue] Value (String): ' + value);
  
      // Call the original method and return the result
      var result = this.setValue(value);
      return result;
    };
  
    // Overload 2: setValue(byte[])
    BluetoothGattCharacteristic.setValue.overload('[B').implementation = function (value) {
      // Convert byte array to hex string
      var hexValue = bytesToHex(value);
  
      console.log('[BluetoothGattCharacteristic.setValue] Value (byte[]): ' + hexValue);
  
      // Call the original method and return the result
      var result = this.setValue(value);
      return result;
    };
  
    // Overload 3: setValue(int, int, int)
    BluetoothGattCharacteristic.setValue.overload('int', 'int', 'int').implementation = function (value1, value2, value3) {
      console.log('[BluetoothGattCharacteristic.setValue] Values (int, int, int): ' + value1 + ', ' + value2 + ', ' + value3);
  
      // Call the original method and return the result
      var result = this.setValue(value1, value2, value3);
      return result;
    };
  
    // Overload 4: setValue(int, int, int, int)
    BluetoothGattCharacteristic.setValue.overload('int', 'int', 'int', 'int').implementation = function (value1, value2, value3, value4) {
      console.log('[BluetoothGattCharacteristic.setValue] Values (int, int, int, int): ' + value1 + ', ' + value2 + ', ' + value3 + ', ' + value4);
  
      // Call the original method and return the result
      var result = this.setValue(value1, value2, value3, value4);
      return result;
    };
  
    // Helper function to convert a byte array to a hex string
    function bytesToHex(bytes) {
      var hexArray = [];
      for (var i = 0; i < bytes.length; ++i) {
        var byteString = (bytes[i] & 0xff).toString(16);
        hexArray.push(byteString.length === 1 ? '0' + byteString : byteString);
      }
      return hexArray.join('');
    }
  });

With this code, we have implemented hooks for all the overloads of the setValue function. Each hook will log the inputs to the console, allowing us to capture the commands we’re interested in. This way, we can move forward with analyzing and emulating the captured commands.

To run Frida with the hooks and capture the output, execute the following command in the terminal:

frida -U -f com.xiaoyu.hlight -l bt_hooks.js

bt_hooks.js is the filename of the JavaScript code containing the hooks and com.xiaoyu.hlight is the package name of the HappyLighting app.

Once the Frida script is running, you can test the functionality by turning the LED strip on and off using the app. The console output should display the captured commands.

In this case, we see that cc2333 is the command for turning on the LED strip, while cc2433 is the command for turning it off. The static parts cc and 33 indicate some command indicator, and we can assume that 23 is for turning on the LED strip and 24 is for turning it off.

Hopefully, by sending these commands (cc2333 to turn on and cc2433 to turn off) to the LED strip ourselves, we can control its status accordingly.

To further explore the functionality and reverse-engineer the commands, let’s change the LED strip’s mode through the application and observe the corresponding commands. By switching between the “Rainbow pulsating” and “Red pulsating” modes multiple times, we can capture the commands associated with each mode.

After analyzing the captured commands, we can observe that bb260f44 corresponds to the “Rainbow pulsating” mode, while bb250f44 corresponds to the “Red pulsating” mode. From our previous understanding of the command structure, we can deduce that the bb and 44 parts likely serve as command indicators.

However, we notice another part in the commands: 0f. By further experimentation, specifically by adjusting the speed of the effect, we can observe that the 0f portion changes accordingly. For example, 01 represents the maximum speed, while 1f corresponds to the minimum speed.

Therefore, we can conclude that the 0f portion of the command likely represents the speed parameter, with different values indicating various speed settings.

I believe we can wrap things up at this point since I think I have successfully demonstrated the concept I had in mind. Now, let’s proceed to the next section where we will develop a straightforward client application to control the LED strip.

Writing a Custom Client

To write a simple client, I opted for Python as my preferred language since it’s the one I usually turn to when exploring new concepts. However, you are free to choose any programming language that you’re comfortable with.

Here’s the code:

import asyncio
from bleak import BleakClient

# Bluetooth device information
device_address = "92:3B:12:00:0D:B2"  # MAC address of the device
characteristic_uuid = "0000ffd9-0000-1000-8000-00805f9b34fb"  # UUID of the characteristic

# Payload in hex string format
payload = bytearray.fromhex("cc2333")

async def write_payload():
    async with BleakClient(device_address) as client:
        await client.is_connected()

        # Write the payload to the characteristic
        await client.write_gatt_char(characteristic_uuid, payload, response=False)

        # Disconnect from the device
        await client.disconnect()

# Run the write_payload coroutine
loop = asyncio.get_event_loop()
loop.run_until_complete(write_payload())

In the provided code snippet, the device_address variable represents the MAC address of the Bluetooth device you want to connect to. It is a unique identifier assigned to each Bluetooth device. You should replace the device_address value with the actual MAC address of your target device.

The characteristic_uuid variable represents the UUID (Universally Unique Identifier) of the specific characteristic on the Bluetooth device that you want to interact with. Characteristics are attributes of a Bluetooth device that enable specific functionalities or data exchange. You should replace the characteristic_uuid value with the actual UUID of the characteristic you intend to communicate with.

By knowing the MAC address and characteristic UUID, you can establish a connection with the Bluetooth device and utilize the corresponding characteristic to perform operations such as reading, writing, or subscribing to data.

There are multiple methods to obtain the MAC addresses and characteristics UUID of the target device. One approach is to write a simple code snippet using the bleak library, while another option is to utilize Android applications such as Bluetooth Terminal.

By following these steps, we have successfully achieved control over the LED strip using our custom client. While we could continue expanding the functionalities of our client, for the purpose of this blog post, it is sufficient to conclude our exploration here.

Conclusion

In conclusion, through our exploration, we have demonstrated the process of reverse engineering a communication protocol and controlling an LED strip. By utilizing the power of Frida, we were able to identify the relevant functions responsible for sending commands to the LED strip. By hooking these functions and capturing the command payloads, we gained insights into the communication protocol.

With this knowledge, we successfully developed a custom client that allows us to control the LED strip independently of the original mobile application. By sending the captured commands, we could turn the LED strip on and off, change its colors, and switch between different lighting modes.

It is important to acknowledge that achieving the desired outcome quickly is not always guaranteed in every reverse engineering scenario. The complexity of protocols and security measures can vary greatly, often necessitating more extensive analysis and reverse engineering efforts. However, in our case, the dynamic analysis approach we employed proved to be highly effective, significantly accelerating the process and saving us many hours of traditional reverse engineering work.

Overall, this post demonstrates the excitement and possibilities that arise from reverse engineering and taking control over IoT devices, opening up new avenues for customization and creativity.

I hope it was helpful. If you have any further questions, please feel free to reach out.

References

Building highly interactive honeypots: CVE-2021-41773 case study

Sun, 17 Oct 2021 03:30:00 +0330

Every day, as we drink our coffee in the office, new vulnerabilities pop out, some of which are highly critical and need quick reactions. Exploiting some of these vulnerabilities is a cinch, like the one found in Apache HTTPD “CVE-2021-41773”, which is why they attract many attackers. In such situations, a precise solution is required to get information around the attack as quickly as possible. The gathered information can be used for different goals, for example, to assist security engineers in knowing the attack patterns to defend themselves against it, or maybe for security researchers to gather intel and knowledge as much as possible; therefore, they can share it publicly. One of those solutions is a honeypot. Essentially, a honeypot acts as a decoy-based intrusion detection system to help us detect attacks and their patterns, and defend ourselves against them. This post (or maybe a series of posts) will discuss how to build a highly interactive honeypot for a vulnerability immediately and analyze the generated logs after successful or unsuccessful attacks.

The approach

While numerous honeypot applications are available for free (like dionaea and cowire), these programs attempt to emulate a service and present the attacker with a fake service. Honeypots of this type are effective against autonomic attacks, but they fail to detect manual attacks or attacks with more than one stage. In addition, there are several techniques an attacker can use to uncover a honeypot.

The aforementioned approach has some shortcomings such as limited emulation on specific services and challenging customization. About customization, researchers find it a time-consuming task where in some specific scenarios, they cannot get the advantage of customizatoin. Moreover, it is good to point out that there is no honeypot for dozens of services.

To overcome these hardships, rather than trying to emulate every possible aspect of the faked system, we can grant access to a real system as it is easy, so the attacker has no way of knowing whether they are logged on to a honeypot or not. Securing a real system is not easy; however, it can be done using virtualization or containerization.

The honeypot is a vulnerable Docker container affected by the CVE-2021-41773. Instances of Filebeat and Auditbeat will collect the logs, and Elasticsearch will colorate them for us. Finally, Kibana can provide a visualized dashboard.

Vuleranble container

A path traversal vulnerability and exploit just dropped in the wild for a specific version of Apache (Apache/2.4.49). This vulnerability allows an unauthenticated attacker to execute a path traversal attack (and now shown RCE if MOD_CGI is enabled) to read files outside the virtual directory path bounds. .

To build a vulnerable container for these types of vulnerabilities, we need the source code of that specific vulnerable version. Luckily, there is a mirror of Apache HTTPD at here!. After downloading the source code, we compile the vulnerable version and build a docker image we can deploy quickly.

The assumption is you know how to install and use Docker; if you’re not familiar, please take a look at This Article!.

Let’s take a look at the content of the Dockerfile:

FROM ubuntu:20.04

MAINTAINER lopqto <morpix@protonmail.com>

# Install the required packages
RUN apt-get update  && apt-get install -y \
    build-essential zlibc libapr1-dev \
    libaprutil1-dev libpcre3-dev zlib1g zlib1g-dev wget \
    subversion python3 autoconf libtool-bin

WORKDIR /honeypot

# Download the vulnerable version
RUN wget https://github.com/apache/httpd/archive/refs/tags/2.4.49.tar.gz \
    && tar -xvf 2.4.49.tar.gz

WORKDIR /honeypot/httpd-2.4.49/

# Compile the vulenrable version
RUN svn co http://svn.apache.org/repos/asf/apr/apr/trunk srclib/apr \
    && ./buildconf \
    && ./configure --prefix=/usr/local/apache2 \
    --enable-mods-shared=all --enable-deflate --enable-proxy \
    --enable-proxy-balancer --enable-proxy-http \
    && make && make install

RUN mkdir -p /var/www/html && mkdir /var/log/apache2/

# Update the required permissions for www-data
RUN chown -hR www-data:www-data /var/www/html \
    && chown -hR www-data:www-data /var/log/apache2/ \
    && chown -hR www-data:www-data /usr/local/apache2/logs/

USER www-data

WORKDIR /var/www/html

# Run apache in foreground mode
ENTRYPOINT ["/usr/local/apache2/bin/apachectl", "-D", "FOREGROUND"]

To build the vulnerable image:

docker build . -t honeypot:latest

Test the vulnerable container

The vulnerability requires specific permissions to be configured. Grab the default httpd.conf file and append these lines end of it:

<VirtualHost *:8080>
	DocumentRoot /var/www/html

	ErrorLog /var/log/apache2/error.log
	CustomLog /var/log/apache2/access.log combined

	<Directory />
		Require all granted
	</Directory>
</VirtualHost>

Then try to run the vulnerable container with the new config file:

docker run -p 80:8080 \
    -v $(pwd)/httpd.conf:/usr/local/apache2/conf/httpd.conf \
    apache:latest

Test the vulnerability by running:

curl http://localhost/cgi-bin/.%2e/.%2e/.%2e/.%2e/bin/sh -d 'echo;whoami'

You should see www-data in output.

Logging

The Elasticsearch ELK stack (Elasticsearch, Logstash, and Kibana) is an ideal solution for search and analytics platforms on honeypot logs.

There are various how-to’s describing how to get ELK running (see here and here for example), so I assume you already have a working ELK system.

HTTP requests logs

A logger with these advantages is needed:

No need to modify the container’s configuration
Prevent attackers from disabling the logger within the container

By default, Apache stores the access logs inside /var/log/apache2/access.log and error logs inside /var/log/apache2/error.log. We need to export these logs from the container and store them inside an ElasticSearch instance. To do this, we will use Filebeat:

Filebeat is a lightweight shipper for forwarding and centralizing log data. Installed as an agent on your servers, Filebeat monitors the log files or locations that you specify, collects log events, and forwards them either to Elasticsearch or Logstash for indexing.

The main configuration file will look like this:

filebeat.config:
  modules:
    path: ${path.config}/modules.d/*.yml
    reload.enabled: false

output.elasticsearch:
  hosts: ["${ELASTICSEARCH_HOST}:9200"]
  username: ${ELASTICSEARCH_USERNAME}
  password: ${ELASTICSEARCH_PASSWORD}

setup.dashboards:
  enabled: true

setup.kibana:
  host: "${KIBANA_HOST}:5601"
  username: ${ELASTICSEARCH_USERNAME}
  password: ${ELASTICSEARCH_PASSWORD}

And for the apache.yml we have:

- module: apache
  access:
    enabled: true
    var.paths: ["/log/access.log"]
  error:
    enabled: true
    var.paths: ["/log/error.log"]

Another option for this problem is to configure the containers to forward their Syslog data to the host, but this can be disabled if the attacker has root access, so it is not ideal.

Executaion logs

Docker runs on the same kernel as the host machine, so we can use kernel-level logging to see what is happening inside the container. An off-the-shelf solution is to use a Linux audit system and configure it to log execve and execveat system calls.

The Linux Audit system provides a way to track security-relevant information on your system. Based on pre-configured rules, Audit generates log entries to record as much information about the events that are happening on your system as possible.

The sad news is Linux audit system does not support kernel namespaces, so logs cannot be filtered for specific containers and the host machine. To make things easier, we defined a custom user inside the Dockerfile named www-data to filter out the related logs by this user.

USER www-data

To log the execution actions, we will use Auditbeat:

Auditbeat is a lightweight shipper that you can install on your servers to audit the activities of users and processes on your systems. For example, you can use Auditbeat to collect and centralize audit events from the Linux Audit Framework. You can also use Auditbeat to detect changes to critical files, like binaries and configuration files, and identify potential security policy violations.

The Auditbeat configuration file will look like this:

auditbeat.modules:

- module: auditd
  audit_rules: |
    -a always,exit -F arch=b64 -S execve,execveat -k exec

output.elasticsearch:
  hosts: ["${ELASTICSEARCH_HOST}:9200"]
  username: ${ELASTICSEARCH_USERNAME}
  password: ${ELASTICSEARCH_PASSWORD}

setup.dashboards:
  enabled: true

setup.kibana:
  host: "${KIBANA_HOST}:5601"
  username: ${ELASTICSEARCH_USERNAME}
  password: ${ELASTICSEARCH_PASSWORD}

Putting it all together

At this point, we have an ELK stack set up and running, Kibana dashboards ready to visualize the logs, a vulnerable container prepared to be exposed on the internet, a Filebeat instance ready to capture HTTP logs, and an Auditbeat instance to capture executed commands.

We can create a simple docker-compose.yml file to deploy the honeypot as fast as possible and make things much more manageable. To install the docker-compose, take a look at here!.

Content of the docker-compose.yml will be something like this:

version: "3.8"
services:
  honeypot:
    build: ./
    hostname: "honeypot"
    networks:
      - honeypot
    ports:
      - "80:8080"
    volumes:
      - logs:/var/log/apache2/
      - $PWD/httpd.conf:/usr/local/apache2/conf/httpd.conf

  auditbeat:
    image: docker.elastic.co/beats/auditbeat:7.15.0
    hostname: "auditbeat"
    user: root
    pid: host
    cap_add:
      - AUDIT_CONTROL
      - AUDIT_READ
    networks:
      - honeypot
    volumes:
      - auditbeat:/usr/share/auditbeat/data
      - $PWD/auditbeat.yml:/usr/share/auditbeat/auditbeat.yml:ro
    environment:
      - ELASTICSEARCH_HOST=elk.host # Change
      - KIBANA_HOST=kibana.host # Change
      - ELASTICSEARCH_USERNAME=elastic # Change
      - ELASTICSEARCH_PASSWORD=changeme # Change
    command: ["--strict.perms=false"]
    depends_on:
      - honeypot

  filebeat:
    image: docker.elastic.co/beats/filebeat:7.15.0
    hostname: "filebeat"
    user: root
    networks:
      - honeypot
    volumes:
      - filebeat:/usr/share/filebeat/data
      - $PWD/filebeat.yml:/usr/share/filebeat/filebeat.yml
      - $PWD/apache.yml:/usr/share/filebeat/modules.d/apache.yml
      - logs:/log/:ro
    environment:
      - ELASTICSEARCH_HOST=elk.host # Change
      - KIBANA_HOST=kibana.host # Change
      - ELASTICSEARCH_USERNAME=elastic # Change
      - ELASTICSEARCH_PASSWORD=changeme # Change
    command: ["--strict.perms=false"]
    depends_on:
      - honeypot

networks:
  honeypot:

volumes:
  auditbeat:
  filebeat:
  logs:

To bring up the honeypot stack, run the following command:

docker-compose up -d

That’s it :). Now, we can trace the logs with the help of Kibana and some beautiful dashboards.

Auditbeat executions Dashboard:

Filebeat Apache access and error logs dashboard:

Conclusion

We can build interactive honeypots for a variety of vulnerabilities in such a short period of time using containers. The ELK stack additionally provides us with some tools to gather valuable logs, manage records easier, and visualize them for better understanding. Spending time and effort, researchers can build honeypots to trace active threats and threat actors, such as miners, and share attack knowledge with other researchers. The information can be used to create threat feeds with a low likelihood of false positives. Additionally, engineers can use honeypots as a decoy system inside of an organization.

The project has been shared on Github. You can use it as a starting point. I hope you find it useful.

You’re more than welcome to share your thoughts and ideas. Feel free to ping me if you have questions about this topic.

Enjoy!

Automated dynamic import resolving using binary emulation

Tue, 08 Sep 2020 04:30:00 +0430

Analyzing malwares is often not an easy task because there are lots of tricks and techniques that malwares use to evade detection and classification or to make the post-analysis more difficult. One such trick is to resolve windows API calls dynamically (called “dynamic import resolving”).

In this blog post, we will talk about dynamic import resolving and a pattern to detect it when reversing malwares, how to defeat this trick using binary emulation and Qiling framework (resolve API calls and extract function names), and finally we will integrate our emulation framework with Ghidra.

In the last section, we will talk about a solution to run Python version 3 and Qiling trough Ghidra so we can see the result of our script inside the decompiler/disassembler view. It will make post-analysis easier.

As a real-life example, we will analyze Netwalker which used this technique and we will discuss our idea around that sample.

What is dynamic import resolving

Let’s talk about dynamic import resolving and indirect function calls. It’s a common technique that malwares use to hide their intention, make the static analysis more difficult, bypass some red flags, etc.

In this technique, the malware tries to create an IAT (Import Address Table) during the execution so there is no sign of used API calls in the PE header.

This technique often shows up in a specific pattern; At the beginning of the execution, the program will build an array of function pointers which works like an IAT and the malware can use stored function pointers with indirect calls as shown below:

It’s rather difficult to determine which function would be called by these indirect function calls without actually executing the binary.

To dynamically make a function pointer, the two API calls LoadLibraryA() and GetProcAddress() are often used.

According to the Microsoft docs, LoadLibraryA():

Loads the specified module into the address space of the calling process. The specified module may cause other modules to be loaded.

HMODULE LoadLibraryA(
  LPCSTR lpLibFileName
);

And GetProcAddress():

Retrieves the address of an exported function or variable from the specified dynamic-link library (DLL).

FARPROC GetProcAddress(
  HMODULE hModule,
  LPCSTR  lpProcName
);

Look at this pseudo-code as a demonstration:

typedef ret_type (__stdcall *f_func)(param_a, param_b);

HINSTANCE hLibrary = LoadLibrary("ntdll.dll");
f_func LocalNtCreateFile = (f_func)GetProcAddress(hLibrary, "NtCreateFile");

LocalNtCreateFile is a function pointer which points to NtCreateFile, which can be stored in an array a.k.a IAT.

To make things more spicy, sometimes malware authors also encrypt the strings passed to LoadLibrary() and GetProcAddress() like what Netwalker did. It will be near to impossible to analyze malware without solving this problem first.

Choosing the approach

To solve these types of techniques and tricks there are a few approaches. For example, we can sometimes decrypt passed strings statically or we can develop an IDA plugin (or any disassembler and decompiler that supports plugins) but that would be a rather time-consuming task. Alternatively, we can use debuggers to execute the malware step by step, and rename variables according to dynamically resolved functions but this is a lot of repetition.

I chose binary emulation because it gives us the best of both worlds, We can have the power of automation and the ease of debugging. ‌It’s worth mentioning that emulating can be very slow at times, especially when dealing with encryption and decryption algorithms. Personally, I think this is an acceptable trade-off.

For binary emulation we will use Qiling. Read my previous post to see why.

Analyzing Netwalker

Today’s sample is NetWalker link! . Netwalker used dynamic import resolving technique with encrypted strings so it is a good example for us to demonstrate our idea and approach around that.

As discussed before, most of the time malwares will try to build an IAT at the beginning of the execution - and NetWalker does this.

After disassembling the malware, we can see a function call right after the entry.

Jumping to that function, we can see the pattern mentioned above; A function is called multiple times and the return value is stored in an array.

This pattern is a sign of dynamic import resolving. We can confirm our guess with a debugger like below:

Let’s jump to the code and write a script to extract these function names.

I’ve discussed the basics of the Qiling like hook_code() and ql.mem.read in the previous post.

In such scenarios, we don’t need to emulate the entire malware, we just need to execute the dynamic import table resolution bit. So we need to find the start and the end of that section. This is rather easy because our target is inside a function, so we only need to emulate that specific function.

ql.run(begin=0x0040c1a0, end=0x0040c1a5)

In this process of analyzing malwares with binary emulation, you need only be creative. For example, in this sample, there are plenty of approaches that you can use; however I chose the easiest and fastest (specifically development time, this solution performs rather badly).

Let’s talk about the approach. As you can see in the image below, the return value of the (probably) decrypter and resolver function is stored in the eax register and then moved to dword ptr [ecx + int]. So we just need to hook the code and extract the value of eax in the right location.

We can run the emulator and try to hook_code() to catch every instruction that is going to be executed.

ql.hook_code(extract_eax)

As you may notice, extract_eax() is a callback function that is designed to extract the value of eax. Qiling will pass the ql (sandbox) object, the address and the size of the instruction to this callback function.

We can extract the instruction inside extract_eax() with mem.read() as below:

buf = ql.mem.read(address, size)

buf is a Python bytearray of our instruction. The next step is detecting the right location to extract eax. By looking at the disassembler we can see a pattern. the first part of the opcode is similar.

Next if will detect the right location:

if "8941" in buf.hex():

to extract eax value we need to do this:

eax_value = ql.reg.eax

eax_value is an address that points to an API call. We can search that address inside import_symbols to extract the API name.

func = ql.loader.import_symbols[eax_value]
func_dll = func["dll"]
func_name = func["name"].decode("ascii")

print(f"found {func_dll}.{func_name} at {hex(address)}")

Fulll code will be:

def extract_eax(ql, address, size):
    buf = ql.mem.read(address, size)

    if "8941" in buf.hex(): # dword ptr [ECX + hex],EAX
        eax_value = ql.reg.eax
        func = ql.loader.import_symbols[eax_value]
        func_dll = func["dll"]
        func_name = func["name"].decode("ascii")
        
        print(f"found {func_dll}.{func_name} at {hex(address)}")

This was easy! right? Next, we need to integrate our scipt with Ghidra to actually use the information we got here. This will help us to see extracted API names inside Ghidra.

Integrating Qiling with Ghidra

As you probably know Ghidra uses Jython and Jython only supports Python version 2 but Qiling is based on Python version 3. I found an interesting project called ghidra_bridge link! that helps us solve this problem.

So Ghidra Bridge is an effort to sidestep that problem - instead of being stuck in Jython, set up an RPC proxy for Python objects, so we can call into Ghidra/Jython-land to get the data we need, then bring it back to a more up-to-date Python with all the packages you need to do your work.

After installing ghidra_bridge you can find an example inside the installation directory called example_py3_from_ghidra_bridge.py. By opening this file we will have an idea about how to write scripts based on ghidra_bridge. Let’s dissect it.

Most scripts should use this minimal template:

def run_script(server_host, server_port):

    import ghidra_bridge 
    with ghidra_bridge.GhidraBridge(namespace=globals(), response_timeout=500):
        pass


if __name__ == "__main__":

    in_ghidra = False
    try:
        import ghidra
        # we're in ghidra!
        in_ghidra = True
    except ModuleNotFoundError:
        # not ghidra
        pass

    if in_ghidra:
        import ghidra_bridge_server
        script_file = getSourceFile().getAbsolutePath()
        # spin up a ghidra_bridge_server and spawn the script in external python to connect back to it
        ghidra_bridge_server.GhidraBridgeServer.run_script_across_ghidra_bridge(script_file)
    else:
        # we're being run outside ghidra! (almost certainly from spawned by run_script_across_ghidra_bridge())

        parser = argparse.ArgumentParser(
            description="py3 script that's expected to be called from ghidra with a bridge")
        # the script needs to handle these command-line arguments and use them to connect back to the ghidra server that spawned it
        parser.add_argument("--connect_to_host", type=str, required=False,
                            default="127.0.0.1", help="IP to connect to the ghidra_bridge server")
        parser.add_argument("--connect_to_port", type=int, required=True,
                            help="Port to connect to the ghidra_bridge server")

        args = parser.parse_args()

        run_script(server_host=args.connect_to_host,
                   server_port=args.connect_to_port)

We only need to focus on run_script() function. The other part is static and probably there is no need to change. Only inside run_script() you are allowed to use Python 3 syntax and only here you are allowed to load Python 3 libraries (like Qiling). As you may notice I added response_timeout to the GhidraBridge object and sets it’s value to 500 seconds. Why? because as we discussed earlier emulating is a time-consuming task and emulating decryptor functions is likely more time-consuming because there is so much instruction code that needs to be emulated. So we need to set response_timeout to prevent any timeout-related errors.

Leaving aside the base template, we can now write our Qiling code inside run_script().

def run_script(server_host, server_port):
    from qiling import Qiling

    import ghidra_bridge 
    with ghidra_bridge.GhidraBridge(namespace=globals(), response_timeout=500):

        ql = Qiling(["/home/lopqto/w/automated/samples/netwalker.exe"], "/home/lopqto/w/automated/rootfs/x86_windows", output = "debug")
        ql.hook_code(extract_eax)
        ql.run(begin=0x0040c1a0, end=0x0040c1a5)

Back to the extract_eax() function, we need to integrate it with Ghidra and add extracted API names as a comment into Ghidra. To add a comment from a script first of all we need an address (location). We have the address value from Qiling but we need to convert this value to Ghidra’s Address type.

To do this we need memory.blocks object from currentProgram API. But there is a challenge here. currentProgram API only is accessible inside run_script(). But we need this API inside extract_eax() callback. There is a cool trick to handle this situation. You need to pass things around with ql object like below:

ql.target_block = currentProgram.memory.blocks[0]

Now we can access to ql.target_block inside extract_eax(). target_block (memory.blocks[0]) points to the PE entrypoint at 0x00400000. to convert address to Address type we need to calculate offset and do something like this:

target_address = ql.target_block.getStart()
target_address = target_address.add(address - 0x00400000)

Now we have our target_address so we need one more step. accessing comment API is similar to above. First we need getListring() object:

ql.listing = currentProgram.getListing()

And to add a comment we can do:

codeUnit = ql.listing.getCodeUnitAt(target_address)
comment_message = "{}.{}".format(func_dll, func_name)
codeUnit.setComment(codeUnit.PRE_COMMENT, comment_message)

Full source code for extract_eax() will be this:

def extract_eax(ql, address, size):
    buf = ql.mem.read(address, size)
    if "8941" in buf.hex(): # dword ptr [ECX + hex],EAX
        
        eax_value = ql.reg.eax
        func = ql.loader.import_symbols[eax_value]
        func_dll = func["dll"]
        func_name = func["name"].decode("ascii")
        target_address = ql.target_block.getStart()
        target_address = target_address.add(address - 0x00400000)
        codeUnit = ql.listing.getCodeUnitAt(target_address)
        comment = "{}.{}".format(func_dll, func_name)
        codeUnit.setComment(codeUnit.PRE_COMMENT, comment)

Now we have a Ghidra script that will use Python3 to run samples trough Qiling and extract dynamic resolved function names and comment them into Ghidra. See the final result:

And we are done. :)

Tips and tricks

Two tricks helped me to make this script. First of all, tracing the binary and printing assembly instructions can help a lot while debugging source!:

md = Cs(CS_ARCH_X86, CS_MODE_64)

def print_asm(ql, address, size):
    buf = ql.mem.read(address, size)
    for i in md.disasm(buf, address):
        print(":: 0x%x:\t%s\t%s" %(i.address, i.mnemonic, i.op_str))

ql.hook_code(print_asm)

You can compare emulation result with your disassembler to debug your program.

The second tip is when you try to run a time-consuming script and write something back to Ghidra (like adding a comment) you may face with an error like this:

ERROR (BackgroundCommandTask) Command Failure: An unexpected error occurred while processing the command: Auto Analysis java.lang.RuntimeException: Timed-out waiting to run a Swing task--potential deadlock!

It’s because java closed the file and to solve this problem you need to increase timeout. Open the file in ghidra/support/launch.properties and add this line:

VMARGS=-Dghidra.util.Swing.timeout.seconds=3600

Conclusion

The idea described in this article can be extended and used to analyze any other malware families that dynamically resolve imports. It’s not an ultimate general solution and you need to change things a little bit to match it against your target binary. I tried to explain my mindset behind the scene as much as possible to help you in this process. Hope this post was helpful.

Don’t hesitate to ping me if there is something wrong or if you want to discuss about the post. I dropped the final script and the malware sample here!.

Automated malware unpacking with binary emulation

Sun, 24 May 2020 04:30:00 +0430

Probably most of the malwares out there use some sort of packer to evade detection and classification or to make the post-analysis more difficult. So in this blog post, I will talk about one of the most-used packing techniques and how to defeat that with the power of binary emulation. Also, I’ll drop a PoC of the new project that I’m working on. Note that this is a universal generic solution for packers that rely on unpacking code in heap memory and execute it.

Background

For packers that encrypt or compress a payload, a stub (a piece of code that contains the decompression or decryption routine) acts as a loader, which executes before the malware. So, we have a loader that decrypts or decompress the real malware to execute it. Stub can use so many techniques to achieve its goal.

There is a technique called self-modifying. In this technique, stub tries to acquire a block of writeable, executable memory, unpack (decrypt/decompress and write) code to the newly allocated memory and finally, transfer execution to the unpacked code in the newly allocated memory. So, if we manage to somehow read that allocated memory and dump the executable payload, right before the execution, we probably unpacked the real malware.

Digging down

Let’s dig down more and talk about some Windows APIs. To acquire a new block of memory, malwares will try to use VirtualAlloc(). VirtualAlloc() is a Windows API inside kernel32.dll. According to Microsoft docs, VirtualAlloc():

Reserves, commits, or changes the state of a region of pages in the virtual address space of the calling process. Memory allocated by this function is automatically initialized to zero.

LPVOID VirtualAlloc(
  LPVOID lpAddress,
  SIZE_T dwSize,
  DWORD  flAllocationType,
  DWORD  flProtect
);

Important variables here are dwSize, flProtect, and the return value. dwSize will define allocated memory block’s size, flProtect will define the memory protection and the return value will be the address of the newly allocated memory.

First of all, we need to catch flProtect, since in order to execute code inside a memory region, protection value must be one of the following values:

PAGE_EXECUTE (0x10)
PAGE_EXECUTE_READ (0x20)
PAGE_EXECUTE_READWRITE (0x40)
PAGE_EXECUTE_WRITECOPY (0x80)

So, we need to monitor dynamically allocated memories with executing protection cause they suggest that malware is about to unpack (or otherwise obtain) code to store there.

There is another interesting API called VirtualProtect(). VirtualProtect() changes the protection of a memory region. Malware can use this API call to change the protection of the allocated memory region if it is not already executable.

BOOL VirtualProtect(
  LPVOID lpAddress,
  SIZE_T dwSize,
  DWORD  flNewProtect,
  PDWORD lpflOldProtect
);

Important variable is flNewProtect. like VirtualAlloc() if flNewPortect contains one of the excution values (0x10, 0x20, 0x40 or 0x80) we need to monitor that specific region for unpacked code.

After we managed to list memory regions with execute protection, we need to set a memory execution breakpoint at those locations. So, if any code executes inside that region (probably unpacked code), we can catch and dump it.

Choosing tools and solutions

I needed a solution to write a cross-platform tool for automatically unpacking malwares with minimal requirements. as a Python developer, I prefer a framework written in Python or with a Python binding. So I chose a binary emulator Qiling!.

Qiling Framework is aimed to change IoT security research, malware analysis and reverse engineering landscape. The main objective is to build a cross-platform and multi-architecture framework and not just another reverse engineering tool. Qiling Framework is designed as a binary instrumentation and binary emulation framework that supports cross-platform and multi-architecture. It is packed with powerful features such as code interception and arbitrary code injection before or during a binary execution. It is also able to patch a packed binary during execution. Qiling Framework is open source and it is written in Python, a simple and commonly used programming language. This will encourage continuous contributions from the security and open-source community making it a sustainable project.

Also, there is a comparison between Qiling with other tools. take a look at this!.

Writing a PoC

First of all, we need to re-implement VirtualAlloc() and VirtualProtect() APIs in a format that Qiling recognizes. To make this happen, we need to write something like this:

@winapi(cc=STDCALL, params={
    "lpAddress": POINTER,
    "dwSize": SIZE_T,
    "flAllocationType": DWORD,
    "flProtect": DWORD
})
def hook_VirtualAlloc(ql, address, params):
    dwSize = params["dwSize"]
    addr = ql.os.heap.mem_alloc(dw_size) # allocate memory in heap
    return addr

winapi is a decorator that helps to define the structure of the API call. Here we have an API with calling convention of STDCALL and 4 inputs with the type of POINTER, SIZE_T, DWORD, and yet another DWORD.

Qiling will pass ql (sandbox) object as a parameter and address of the location that called the API. params will be a dictionary that contains all passed parameters to that API call.

For VirtualProtect() we can write something like this:

@winapi(cc=STDCALL, params={
    "lpAddress": POINTER,
    "dwSize": UINT,
    "flNewProtect": UINT,
    "lpflOldProtect": POINTER
})
def hook_VirtualProtect(ql, address, params):
    return 1

To override default API hooks, Qiling provides set_api:

ql.set_api("VirtualAlloc", hook_VirtualAlloc)
ql.set_api("VirtualProtect", hook_VirtualProtect)

Inside hooked APIs we need to catch allocated memory regions with executable protection:

addr = params["lpAddress"]
dw_size = params["dwSize"]
fl_new_protect = params["flNewProtect"]

# PAGE_EXECUTE (0x10), 
# PAGE_EXECUTE_READ (0x20), 
# PAGE_EXECUTE_READWRITE (0x40), 
# PAGE_EXECUTE_WRITECOPY (0x80),
if fl_new_protect in [0x10, 0x20, 0x40, 0x80]:
    # add newly allocated memory to list of memory regions
    mem_regions.append({"start": addr, "size": dw_size})

The next step will set up a memory execution breakpoint. Qiling offers a function called hook_code(). hook_code() takes 3 parameters. a callback function, beginning, and end of the memory region.

ql.hook_code(dump_memory_region, begin=addr, end=addr + dw_size)

If any code get excuted inside that memory region, Qiling will call dump_memory_region(). To dump that region, Qiling offers ql.mem.read(). read() takes 2 parameters, start location and size.

excuted_mem = ql.mem.read(address, size)

Full code for PoC will be:

import sys

from qiling import *
from qiling.os.windows.const import *
from qiling.os.const import *
from qiling.os.windows.fncc import *
from qiling.os.windows.utils import *
from qiling.os.windows.thread import *
from qiling.os.windows.handle import *
from qiling.exception import *

mem_regions = []

def get_mem(addr):
    for mem_region in mem_regions:
        start = mem_region['start']
        end = mem_region['start'] + mem_region['size']
        size = mem_region['size']
        if addr in range(start, end + 1): # check if address in that memory region
            return (start, size)
    
    return None
            
def dump_memory_region(ql, address, size):
    mem_region = get_mem(address) # check if memory region exists
    # check if that memory region got removed before (duplication)
    if mem_region is not None:
        # read memory, start =  first excuted address ; end = end of the region
        excuted_mem = ql.mem.read(address, mem_region[1])
        with open(f"{hex(address)}.bin", "wb") as f:
            f.write(excuted_mem) # write extracted code to a binary file

        # delete that region to overcome duplication
        mem_regions.remove({"start": mem_region[0], "size":mem_region[1]})

@winapi(cc=STDCALL, params={
    "lpAddress": POINTER,
    "dwSize": UINT,
    "flNewProtect": UINT,
    "lpflOldProtect": POINTER
})
def hook_VirtualProtect(ql, address, params):
    addr = params["lpAddress"]
    dw_size = params["dwSize"]
    fl_new_protect = params["flNewProtect"]

    # PAGE_EXECUTE (0x10), 
    # PAGE_EXECUTE_READ (0x20), 
    # PAGE_EXECUTE_READWRITE (0x40), 
    # PAGE_EXECUTE_WRITECOPY (0x80),
    if fl_new_protect in [0x10, 0x20, 0x40, 0x80]:
        # add newly allocated memory to list of memory regions
        mem_regions.append({"start": addr, "size": dw_size})
        # add memory on excute breakpoint to newly allocated memory
        ql.hook_code(dump_memory_region, begin=addr, end=addr + dw_size)
    return 1

@winapi(cc=STDCALL, params={
    "lpAddress": POINTER,
    "dwSize": SIZE_T,
    "flAllocationType": DWORD,
    "flProtect": DWORD
})
def hook_VirtualAlloc(ql, address, params):
    dw_size = params["dwSize"]
    addr = ql.os.heap.mem_alloc(dw_size) # allocate memory in heap

    # PAGE_EXECUTE (0x10), 
    # PAGE_EXECUTE_READ (0x20), 
    # PAGE_EXECUTE_READWRITE (0x40), 
    # PAGE_EXECUTE_WRITECOPY (0x80),
    fl_protect = params["flProtect"]
    if fl_protect in [0x10, 0x20, 0x40, 0x80] :

        # add newly allocated memory to list of memory regions
        mem_regions.append({"start": addr, "size": dw_size})
        # add memory on excute breakpoint to newly allocated memory
        ql.hook_code(dump_memory_region, begin=addr, end=addr + dw_size)

    return addr

def sandbox(path, rootfs):
    # create a sanbox for windows x86
    ql = Qiling([path], "rootfs/x86_windows", output = "debug")

    # set API breakpoints
    ql.set_api("VirtualAlloc", hook_VirtualAlloc)
    ql.set_api("VirtualProtect", hook_VirtualProtect)

    ql.run()

if __name__ == "__main__":
    if not len(sys.argv) == 2:
        exit(-1)

    path = sys.argv[1]
    sandbox(path, "rootfs/x86_windows")

Demonstration

For demonstration, i wrote a small program that runs a shellcode.

#include <windows.h>
#include <iostream>

char shellcode[113] =   "\x31\xdb\x64\x8b\x7b\x30\x8b\x7f"
                        "\x0c\x8b\x7f\x1c\x8b\x47\x08\x8b"
                        "\x77\x20\x8b\x3f\x80\x7e\x0c\x33"
                        "\x75\xf2\x89\xc7\x03\x78\x3c\x8b"
                        "\x57\x78\x01\xc2\x8b\x7a\x20\x01"
                        "\xc7\x89\xdd\x8b\x34\xaf\x01\xc6"
                        "\x45\x81\x3e\x43\x72\x65\x61\x75"
                        "\xf2\x81\x7e\x08\x6f\x63\x65\x73"
                        "\x75\xe9\x8b\x7a\x24\x01\xc7\x66"
                        "\x8b\x2c\x6f\x8b\x7a\x1c\x01\xc7"
                        "\x8b\x7c\xaf\xfc\x01\xc7\x89\xd9"
                        "\xb1\xff\x53\xe2\xfd\x68\x63\x61"
                        "\x6c\x63\x89\xe2\x52\x52\x53\x53"
                        "\x53\x53\x53\x53\x52\x53\xff\xd7";

int main() {
    DWORD old_protect;
    LPVOID executable_area = VirtualAlloc(NULL, 113, MEM_RESERVE, PAGE_READWRITE);
    executable_area = VirtualAlloc(executable_area, 113, MEM_COMMIT, PAGE_READWRITE);

    if (executable_area == nullptr) {
        std::cout << "error in allocating memory" << std::endl;
    }

    memcpy(executable_area, shellcode, 113);
    VirtualProtect(executable_area, 113, PAGE_EXECUTE, &old_protect);

    int(*f)() = (int(*)()) executable_area;
    f();
    VirtualFree(executable_area, 113, MEM_RELEASE);
}

We can try the PoC for that program and see output:

We managed to dump the shellcode successfully. I hope this was helpful. :)

Please do not hesitate to ping me if there is something wrong!

Decrypting NetWire's keylog files

Sun, 29 Mar 2020 04:30:00 +0430

NetWire is recently back to the malware trends again. This new variant of NetWire uses Guloader to distribute itself. After some observation, it seems that NetWire creators changed the encryption routine. In this analysis, I am going to present you how to reverse the new encryption routine and get a clean version of the keylog file.

Background

According to the malpedia:

Netwire is a RAT, its functionality seems focused on password stealing and keylogging, but includes remote control capabilities as well. Keylog files are stored on the infected machine in an obfuscated form. link!

Netwire has lots of functionalities such as taking remote control of infected PCs, keylogging, taking screenshots and extracting system information. NetWire creators added multiple data encryption layers to make a hard time for researchers. there are some sources out there about the decryption of implemented custom C&C binary protocol but there are limited sources (almost nothing) about decrypting keylog files.

NetWire encryption routine

Let’s follow the white rabbit down to the rabbit hole. After execution, NetWire makes a folder at %APPDATA%/Logs and saves keylog files there. I won’t analyze the whole malware in this blog post, only the encryption routine since with some search, you can find public researchers about analyzing NetWire.

We need to open NetWire in a disassembler (ida free will be enough) and find the correct function that writes to the log file. To achieve this goal we can look at the import section and search for WriteFile Windows API call. xrefing shows there are 2 locations that WriteFile got used.

Let’s look at the first one.

Dissecting more shows us an interesting string. Probably in this block, NetWire tries to generate keylog file names like what we saw earlier.

So Let’s assume this is the right one and dig in more. To cheat we can fire up Ghidra with default config and try to analyze decompiled version of this function.

We can clean up the code to make it more human-readable. Investigating more shows us an interesting piece of code at the middle of the function. to make it more complicated NetWire tries to load every buffer in 4-bit chunks and do encryption stuff at every bit separately.

And to make things clear we can estimate the algorithm is something like this:

for index in range(0, number_of_bytes_to_write):
    buffer[index] = ( buffer[index] ^ 0x9D ) + 0x24

Reversing this opration is easy. we need to do an operation like this:

(buffer[index] - 0x24) ^ 0x9D

NetWire decryptor

I wrote a small python script that you can find here. Just pass the filename as argv and you can get the decrypted version of the keylog file in output.

Crackmes for lazies: angr demonstration

Sun, 01 Mar 2020 19:10:00 +0330

I have been playing crackmes and CTFs all the time to boost my reverse engineering knowledge and learn new stuff, but there are times that you find some challenges boring or without new unique technics so you develop some automation tools to pass the challenges. So in this blog post, I’m gonna show you my ultimate tool to solve these types of challenges.

What is angr and why?

According to the angr’s website:

angr is a multi-architecture binary analysis toolkit, with the capability to perform dynamic symbolic execution (like Mayhem, KLEE, etc.) and various static analyses on binaries.

why am I using angr? cause angr is a pain-free toolkit with a supporting binding for Python programming language and makes binary analysis easy. the scenario is simple: find the location or function that prints the flag and give the address to angr. If you wait long enough, angr will give you the right input to reach the requested flag

Symbolic execution

Let’s make things clear for ourselves. What is the goal of a typical crackme? Find the right input that passes the checks (some conditional branches) and prints the flag. According to Wikipedia:

In computer science, symbolic execution (also symbolic evaluation) is a means of analyzing a program to determine what inputs cause each part of a program to execute. An interpreter follows the program, assuming symbolic values for inputs rather than obtaining actual inputs as normal execution of the program would. It thus arrives at expressions in terms of those symbols for expressions and variables in the program, and constraints in terms of those symbols for the possible outcomes of each conditional branch. link!

So angr will assume the input as a symbolic value and with the power of symbolic execution, it will find the path which leads to eventual printing of the flag. After finding the path, we have a series of conditions that our input needs to pass them to reach that location. angr will solve these conditions using SMT solver, like z3, in order to ask questions like “given the output of this sequence of operations, what must the input have been?”

Demo time

I found an interesting crackme challenge that has 13 checks. It will be much harder to follow these checks manually but with the power of angr, the challenges will be done with ~10 lines of code and ~30sec of computing time.

for this challenege,i wrote this code:

import angr
import claripy

project = angr.Project('./keygen', load_options={'auto_load_libs': False})
argv1 = claripy.BVS('argv1', 0xF * 8)
initial_state = project.factory.entry_state(args=['./keygen', argv1]) 
sm = project.factory.simulation_manager(initial_state)
sm.explore(find=0x401a5a, avoid=0x401a73)
result = sm.found[0]
print(result.solver.eval(argv1, cast_to=bytes))

Now it’s the time to dissect the code. First of all, you need to install angr with pip. angr has dropped python2 support and you need python3 to use angr.

pip install angr
# or 
pip3 install angr

In this line, we create a Project object and ./keygen is the relative address to our binary. what about load_options?

The CFG analysis does not distinguish between code from different binary objects. This means that by default, it will try to analyze control flow through loaded shared libraries. Since processing the shared libraries will probably prolong the analysis to even days, it’s almost never intended behavior. To load a binary without shared libraries, add the following keyword argument to the Project constructor: load_options={'auto_load_libs': False}. link!

project = angr.Project('./keygen', load_options={'auto_load_libs': False})

After that, we need to define our input (argv1) as a symbolic value. BVS creates a 32-bit symbolic bit vector with the size of 0xF * 8. you need to define the size equal or bigger than the correct input. link!

argv1 = claripy.BVS('argv1', 0xF * 8)

angr has 4 initial states:

blank_state() constructs a “blank slate” blank state, with most of its data left uninitialized. When accessing uninitialized data, an unconstrained symbolic value will be returned.
entry_state() constructs a state ready to execute at the main binary’s entry point.
full_init_state() constructs a state that is ready to execute through any initializers that need to be run before the main binary’s entry point, for example, shared library constructors or preinitializers. When it is finished with these it will jump to the entry point.
call_state() constructs a state ready to execute a given function.

Our binary gets input as an argv and according to the docs:

If you’re executing in an environment that can take command line arguments or an environment, you can pass a list of arguments through args and a dictionary of environment variables through env into entry_state and full_init_state. The values in these structures can be strings or bitvectors, and will be serialized into the state as the arguments and environment to the simulated execution. link!

so in this line, we picked entry_state to pass the symbolic value argv1.

initial_state = project.factory.entry_state(args=['./keygen', argv1]) 

The most important control interface in angr is the SimulationManager, which allows you to control symbolic execution over groups of states simultaneously, applying search strategies to explore a program’s state space. link!

We need to define SimulationManager and path the initial_state to it.

sm = project.factory.simulation_manager(initial_state)

An extremely common operation in symbolic execution is to find a state that reaches a certain address, while discarding all states that go through another address. Simulation manager has a shortcut for this pattern, the .explore() method.

When launching .explore() with a find argument, execution will run until a state is found that matches the find condition. find condition can be the address of an instruction to stop, a list of addresses to stop, or a function which takes a state and returns whether it meets some criteria. When any of the states in the active stash match the find condition, they are placed in the found stash, and execution terminates. You can then explore the found state, or decide to discard it and continue with the other ones. You can also specify an avoid condition in the same format as find. When a state matches the avoid condition, it is put in the avoided stash, and execution continues.

sm.explore(find=0x401a5a, avoid=0x401a73)

Let’s look at the binary:

The address of ACCESS GRANTED print is 0x401a5a and the adress of ACCESS DENIED print is 0x401a73.

I wrote the last two lines to grab the first value from the found stash and cast that value to byte and print the final result.

result = sm.found[0]
print(result.solver.eval(argv1, cast_to=bytes))

Let’s run our program to see the result:

That’s it, I hope you find this useful.

References

Frida by example: bypassing IsDebuggerPresent() check

Sat, 20 Apr 2019 20:00:00 +0430

Almost every malware exists out there has a functionally to make the post-detection analysis more difficult. Threat actors use various anti-debugging techniques, one of the more common ones used to check whether a debugger is on via IsDebuggerPresent() Windows API call. In this blog post, we will discuss how to bypass this technique by Frida.

why Frida?

Frida is a dynamic instrumentation toolkit. It gives a simple interface where you can develop complex hooking logic rapidly and make changes to it as your requirements. Frida supports Windows, macOS, GNU/Linux, iOS, Android, and QNX.

Prepare a sample PE

To allow us to dynamically test our function hooks I wrote a small Windows test application harness in C++. You can see the main functionality below, it’s short and easy to understand.

// target.cpp

#include <Windows.h>
#include <iostream>

int main()
{
        bool check = IsDebuggerPresent();
        if(check == 0)
        {
            std::cout << "cool!" << std::endl;
        }
        else
        {
            std::cout << "common .." << std::endl;
        }

        return 0;
}

Lets try to look at IsDebuggerPresent() C++ function prototype.

BOOL WINAPI IsDebuggerPresent(void);

According to Microsoft docs, if the current process is not running in the context of a debugger, the return value will be zero.

Bypassing IsDebuggerPresent()

First of all, you need to install Frida, start from here: documentation. You can test your installation by running frida-ps.exe in Powershell or CMD.

How The Frida works? Frida will inject Google’s V8 engine to a specific process. After that it will run JavaScript code and generate a dynamic hook. Lets make our hand dirty and write some JavaScript codes. :)

For hooking purpose we will use Interceptor API.

Interceptor.attach(target, callbacks): intercept calls to target function. it’s a NativePointer specifying the address of the function you would like to intercept calls to.

To begin, we need to find the address of IsDebuggerPresent() function by using DebugSymbol API.

// poc.js

isDebuggerPresentAddr = DebugSymbol.getFunctionByName("IsDebuggerPresent")
console.log("function address : " + isDebuggerPresentAddr)

Using the Interceptor, we can quickly hook the application and write some basic JS to make IsDebuggerPresnt to always return 0. Notice the use of isDebuggerPresentAddr in the code.

// poc.js

Interceptor.attach(isDebuggerPresentAddr, {
        onEnter: function (args) {
			console.log("IsDebuggerPresent() get called ...");
        },
        onLeave: function (retval) {
			retval.replace(0);
        },
});

retval is a NativePointer-derived object containing the raw return value. We can use replace() to change the return value.

Now it’s time to test the bypass method. First we spawn process without injecting our code:

frida.exe .\target.exe

Frida will stop on Entrypoint.Now try to attach your debugger to the target.exe process and then type %resume command to continue execution. debugger will be detected by the process. Run the following command to inject our code and bypass the check:

frida.exe .\target.exe -l .\poc.js

You can see the result below.

The IsDebuggerPresent() has been bypassed successfully. :)

References

Getting started with Dynamic Binary Analysis by Ali Mosajjal

lopqto's adventures

Punched In: Extracting Firmware from an ESP8266 RFID Terminal

The Device

Hardware Parts

Getting the Firmware

Reading the Binary

Into Ghidra

What’s Next

References & Tools

Via Negativa: On Reducing Entropy in Systems

What is Via Negativa?

Subtraction vs. Addition

Subtracting as an Engineer

Enjoying the Sunset

Closing Toughts

Reversing a Remote Controller: A Case Study in RF Engineering

Exploring the Target Device

Planning the Next Steps

Capturing the Radio Waves

Probing for a Rolling Code

Demodulating the Signal

What is a PWM?

Dissecting a Command

The Final Command Format

Timing is Everything!

Creating a Custom Remote Controller

The Building Blocks: Timing is Key

Assembling the Full Command

Conclusion

References

Tools

From Laziness to Control: Reversing an IoT device using Frida

Exploring the Target Device

Choosing the Right Approach

Environment Setup

Reversing the Communication Protocol

Writing a Custom Client

Conclusion

References

Read More

Building highly interactive honeypots: CVE-2021-41773 case study

The approach

Vuleranble container

Test the vulnerable container

Logging

HTTP requests logs

Executaion logs

Putting it all together

Conclusion

Read more

Automated dynamic import resolving using binary emulation

What is dynamic import resolving

Choosing the approach

Analyzing Netwalker

Integrating Qiling with Ghidra

Tips and tricks

Conclusion

Read more

Automated malware unpacking with binary emulation

Background

Digging down

Choosing tools and solutions

Writing a PoC

Demonstration

Read more

Decrypting NetWire's keylog files

Background

NetWire encryption routine

NetWire decryptor

Read more

Crackmes for lazies: angr demonstration

What is angr and why?

Symbolic execution

Demo time

References

Frida by example: bypassing IsDebuggerPresent() check

why Frida?

Prepare a sample PE

Bypassing IsDebuggerPresent()

References