In part one of this research, the Cybereason Nocturnus Incident Response Team provided a unique glimpse into the Winnti intrusion playbook, covering the techniques that were used by the group from initial compromise to stealing the data, as observed and analyzed by the Cybereason Incident Response team.
This part of the research zeroes in on the Winnti malware arsenal that was discovered during the investigation conducted by the Cybereason IR and Nocturnus teams. In addition, our analysis of the observed malware provides a deeper understanding of the elaborate and multi-layered Winnti infection chain, including evasive maneuvers and stealth techniques that are baked-in to the malware code, as well as the functionality of the various malware.
Perhaps one of the most interesting and striking aspects of this report is the level of sophistication introduced by the malware authors. The infection and deployment chain is long, complicated and interdependent–should one step go wrong, the entire chain collapses - making it somewhat vulnerable, yet at the same time provides an extra level of security and stealth for the operation.
These steps have proven themselves effective time and time again, as the operation remained under-the-radar for years. While there have been past reports describing some aspects of these intrusions, at the time of writing this report there was no publicly available research that discussed all of the tools and techniques and the manner in which they all fit together, as mentioned in this report.
The following graph describes the infection chain presented in this attack:
The Spyder loader is the first malicious binary the attackers execute on a targeted machine. This malware is executed from the batch files we discussed in our blog’s part 1 - cc.bat or bc.bat:
The loader’s purpose is to decrypt and load additional payloads and is being delivered in 2 variations. The first variation, is a modified SQLite3 DLL, that uses the export’s ordinal number 138 to serve malicious code, that loads and executes a file argument provided at runtime, in our case C:\Windows\System32\x64.tlb:
As seen above, the loader is executed via the famous LOLBIN rundll32.exe, in the following manner:
rundll32.exe <modified sqlite3.dll file>,#138 C:\Windows\System32\x64.tlb
Interestingly, Cybereason found this loader in different names and in different locations across infected machines:
The attackers utilized the System32 directory, which holds a multitude of TLB and DLL files, to hide their external “TLB” payload and DLL loader to make it harder to detect.
This DLL wasn’t the only Spyder Loader we found, as Cybereason discovered a second variation of this malware in the form of a standalone executable called sqlite3.exe, masquerading as a SQLite3-related executable as well.
This version featured some improvements, such as logging messages, which shed some light on some of its functionality and capabilities:
Throughout its execution, the Spider Loader implements a few interesting methods to evade detection and to maintain stealth:
At the beginning of execution, the loader checks if the file argument exists:
If it does, the loader checks for its size: if it is larger than 1.04 MB, it deletes it; if it is smaller than 1.04 MB or equal to it, it decrypts it in memory using the open-source CryptoPP C++ library and then deletes it from disk.
Cybereason assesses this condition is intended to validate that the loader won’t try to decrypt the wrong file, and as a precaution against analysis environments or Sandboxes.
After decrypting the payload, the attackers copy the system ntdll.dll file to C:\Windows\System32\TN{random_characters}.dll, and load it to memory:
Then, they acquire the NtProtectVirtualMemory address from the copied and loaded file and call a specific routine (which we named as “BypassEdrHook”) multiple times using 2 arguments:
The BypassEdrHook function will compare the first bytes in memory of the native API functions in the loaded ntdll.dll image to the first bytes in memory of the same function in the loaded/copied DLL memory image.
If the ntdll function’s first bytes are different from the first bytes of the copied DLL in memory, the attackers will conclude that this native function is hooked by an EDR tool.
To override it, the attackers count the number of different bytes at the beginning of these two functions, then they change the permissions of the relevant patched bytes in the original ntdll.dll image to READWRITE_EXECUTE, copy the original bytes from the loaded/copied DLL memory, and restore the previous page protection settings:
This procedure will occur for the following native API functions:
Right afterward, the Spyder Loader will execute the payload reflectively, and lastly, will delete TN{random_characters}.dll to leave no traces.
The above-mentioned PE files share similar code with other known Spyder loaders, such as the oci.dll payload mentioned in a SonicWall blog from March 2021:
Moreover, our PE files also share a similar evasion technique in masquerading as a legitimate executable. In the aforementioned blog post, it disguised as D3D DLL - a Direct3D 11 runtime DLL, now it disguise as SQLite3:
These similarities, in addition to others, have led us to conclude that this file is an evolution of the Winnti Spyder Loader.
After deploying the initial payload, Winnti employs a sophisticated and unique multi-staged infection chain with numerous payloads. Each payload fulfills a unique role in the infection chain, which is successful only upon the complete deployment of all of the payloads.
In the upcoming sections, we will discuss the following payloads:
Several unique techniques are used by the Winnti group to store data, evade detection, and thwart investigations during this infection flow. One of those techniques, which Winnti heavily uses, is the CLFS mechanism.
CLFS (Common Log File System) is a logging framework that was first introduced by Microsoft in Windows Server 2003 R2, and is included in later Windows operating systems.
This mechanism provides a high-performance logging system for a variety of purposes ranging from simple error logs to transactional systems and data stream collection.
One of the main uses of CLFS in the Windows operating system is in the Windows Kernel Transaction Manager (KTM) for both Transactional NTFS (TxF) and Transactional Registry (TxR) operations. Transactional operations bring the concept of atomic transactions to Windows, allowing Windows to log different operations on those components and support the ability to roll back if needed.
The high-performance aspect of this framework is based on the concept of storing the log data in memory buffers for fast writing, reading and flushing them to disk–but not continuously, according to a stated policy.
CLFS employs a proprietary file format that isn't documented, and can only be accessed through the CLFS API functions. As of writing this report, there is no tool which can parse the flushed logs. This is a huge benefit for attackers, as it makes it more difficult to examine and detect them while using the CLFS mechanism.
On disk, the CLFS log store consists of:
As will be discussed, Winnti group used this mechanism to store and hide the payload that will be extracted from the CLFS file and used by other PEs in the execution chain to build the attacker’s next steps.
STASHLOG (shiver.exe / forsrv.exe) is a 32 bit executable that is being used to prepare the victim machine for further compromise, and to “stash” a malicious, encrypted payload to a CLFS log file. This payload will be decrypted at each phase to deliver the next phase in the infection.
Both STASHLOG and SPARKLOG, which will be described further in the next section, are executed using a second cc.bat file in the following form:
STASHLOG has 2 modes of execution:
When no arguments are passed, STASHLOG prints the following output:
This output consists of the machine Globally Unique Identifier (GUID) along with a 56 byte string:
This example demonstrates the process of building the 56 byte string:
The final string will be:
The random GUID is then registered as a global atom entry in the form of win::{GUID}:
If an atom prefixed with win:: already exists during execution, then the existing atom will be used instead of the newly generated one.
Upon execution with command line arguments, STASHLOG checks the existence of the global atom table entry win::{GUID}, and the process will quit immediately unless the value exists.
Then, STASHLOG loads the file found in the given argument into memory, checks its content, and stores it in a CLFS log file. STASHLOG prints log information about this process to the terminal:
This output in this operation was redirected to a log file, as can be seen on cc.bat. Breaking down the transformation steps from the log:
Throughout the execution, STASHLOG uses the same string decryption technique used in other samples by XORing strings with pre-defined bytes, words or qwords, using the XMM registers. This decryption sequence is present at the beginning of every function that uses strings, where those strings are not saved globally, likely in an attempt to protect them.
As can be seen in the example below from STASHLOG, it uses it to decrypt several debug strings:
This technique is also being used in other samples in the chain, including SPARKLOG, PRIVATELOG and DEPLOYLOG.
This sample uses an interesting Anti-Disassembly technique which thwarts the disassembly process and makes the investigation job harder. Each function inside STASHLOG contains a jump list of every node in the function:
This jump list is then used as a flow control during the function execution. In some cases the address of the next node will be pushed to a register and then there will be a JMP to this register:
The more interesting use of the jump list is replacing the conditional jump commands like JZ and JNZ with a SET command that changes the register given as an argument to the value of the corresponding checked flag. For example, when using SETZ eax, the EAX register will be changed to “1” if the “0” flag is checked.
Using the set register, there will be a JMP to an address on the function’s jump list. This JMP is usually resolved for a switch-case mechanism, and IDA Pro disassembles it as one:
Using this method, the disassembler displays all the available JMP options based on the function’s jump list even though there are only two jump options - when the assigned register is either 0 or 1 (in the example above it’s the ECX register). These methods make the disassembled function very hard to read and investigate:
SPARKLOG (spark.exe) is a 32 bit executable written in C++, employed in this attack to extract a DLL from the CLFS file, decrypt it and then launch it for side-loading by Windows services running as SYSTEM. Executing this phase of the attack successfully enables the attackers to gain Privilege Escalation and also Persistence in a specific case.
The execution of SPARKLOG starts by creating a non-visible window followed by a message posted to trigger the execution of the main thread:
The PE then retrieves an encrypted DLL content from the CLFS log file, decrypts it and gets the OS version in use. This OS version will be required later to understand how to deploy the DLL in the compromised machine. Then, it decrypts the strings Global\HVID_ and Global\APCI#. First, it uses the GetVolumeNameForVolumeMountPointA API call to get the GUID of the operating system volume and acquires a handle to a HVID_<OS Volume GUID> event.
Then, it queries the MachineGUID value from the registry HKLM\SOFTWARE\Microsoft\Cryptography key and creates an event by the name of Global\APCI#<Machine GUID>. Using these events is a means of communication between the modules in the attack, and it will be used in further modules as well:
Next, it starts to deploy PRIVATELOG based on the OS version. From Windows Vista to Windows 7, SPARKLOG uses a popular DLL side loading technique that involves dropping the DLL with the name wlbsctrl.dll to the %SYSTEM32%\WindowsPowershell\v1.0 directory. It then stops IKEEXT, a service that was compromised by WINNTI in the past, changes the configuration using ChangeServiceConfigA based on the argument count, then starts it again:
After a successful service start, the service executes and triggers the DLL side-loading vulnerability using SYSTEM privileges. It is interesting to note that in this case, if a command line argument is provided, the DLL will be deleted after the execution starts. This might be due to the fact that abusing wlbsctrl.dll is pretty common, and might trigger EDR vendors later on.
From Windows Server 2012 to Windows 10, SPARKLOG acts in a similar fashion, but with a different name and location:
In both cases, the attackers gain stealth by deploying PRIVATELOG while masquerading legitimate file names in privileged locations, as well as gaining persistence and privilege escalation to execute their next step as SYSTEM, the most privileged user in a local machine:
PRIVATELOG is a module that exists in 2 forms:
As both of the DLLs are being loaded by native Windows services, they both live in the context of the svchost process, but differ in their execution flow.
At the beginning of its execution, wlbsctrl.dll is loaded by the IKEEXT service and verifies similarly to prntvpt.dll that it’s being executed from svchost using the right command line (netsvcs). After this check, wlbsctrl.dll goes straight to dropping DEPLOYLOG.
At the beginning of its execution, prntvpt.dll verifies it is being loaded by the correct process with the right command line (svchost -k print), similar to wlbsctrl.dll. This command line is the one being executed upon starting the PrintNotify service.
When the PrintNotify service starts, it also loads PrintConfig.dll, which is being executed from its ServiceMain function. To hijack the execution flow, prntvpt.dll loads PrintConfig.dll and acquires the address of its ServiceMain function. Then, it patches this function, and adds a jump instruction to itself, to continue its execution.
The prntvpt.dll component is different, as it is also the persistence tool for the infection, as opposed to the previous samples we discussed which execute only once to infect the machine, this tool runs every time the PrintNotify service is executed. From this point on, the different DLL files act almost the same.
PRIVATELOG decrypts DEPLOYLOG in memory from the CLFS log file, then it copies dbghelp.dll from its original place in C:\Windows\System32\dbghelp.dll to C:\Windows\System32\WindowsPowerShell\v1.0\dbghelp.dll. Next, the attackers use a rather unique technique to overwrite the copied dbghelp.dll with the aforementioned decrypted buffer using Windows Transactional NTFS (TxF).
Transactional NTFS is a component introduced in Windows Vista that allows developers to create, edit and delete files and directories while giving them the option to roll back in case of errors. This mechanism is used in major operating system components like Windows Update, Task Scheduler and System Restore.
Using Transactional NTFS, the attackers can perform file operations using unconventional methods that can be hard to detect for some security products. They leverage it to create a new malicious dbghelp.dll using the following steps:
DEPLOYLOG (dbghelp.dll) is a 64 bit DLL, with two purposes:
It’s noteworthy to mention that to evade detection, the attackers deployed DEPLOYLOG as dbghelp.dll, a generic, widely used name leveraged to masquerade as a legitimate file at the same location as PRIVATELOG (C:\Windows\System32\WindowsPowerShell\v1.0).
Once DEPLOYLOG is executed, it starts with a sleep of 7777 milliseconds:
Afterward, it tries to acquire a handle to the earlier HVID_<OS Volume GUID> event, and If it doesn’t exist it creates it. Then, it initializes the communication channel with the future deployed rootkit:
Upon succeeding the earlier steps, DEPLOYLOG extracts the rootkit from the CLFS log file, decrypts its content, then stops the amdk8 service. This service is the AMD K8 processor kernel driver service. Aiming for this specific service can tell something about the Winnti modus operandi, indicating they aim only for AMD-related machines to be infected, which could also indicate having a prior knowledge of the victim's infrastructure.
Then, DEPLOYLOG decrypts the string SystemRoot\System32\drivers\bqDsp.sys and changes the amdk8 service configuration to point to this path:
Next, DEPLOYLOG writes the bqDsp.sys rootkit driver from the CLFS log file to the C:\WINDOWS\system32\drivers directory and starts the service again, this time to execute its malicious payload. By doing so, DEPLOYLOG finishes deploying the rootkit. To cover its tracks, DEPLOYLOG will then stop the service, restore its previous configuration to point to the amdk8.sys driver, and finally delete WINNKIT:
Once deployed, DEPLOYLOG turns to its second mission, effectively acting as a user-mode agent whose purpose is to bridge the newly deployed rootkit and the remote C2. The DLL will start communicating with the C2 servers which will send data that will be intercepted by the driver, as will be explained in the next section discussing WINNKIT. Using this agent, the attackers will load new modules, pop a CMD shell, drop MFSDLL.exe for credential grabbing and more:
The final payload deployed by Winnti is also the most evasive and sophisticated: a driver acting as a rootkit, dubbed WINNKIT. WINNKIT’s previous version was researched in the past, and its purpose is to act as a kernel-mode agent, interacting with the user-mode agent and intercepting TCP/IP requests, by talking directly to the network card. The almost zero detection rate in VirusTotal, together with the compilation timestamp from 2019, illustrates just how evasive this rootkit really is, staying in the shadows for 3 years:
WINNKIT contains an expired BenQ digital signature, which is leveraged to bypass the Driver Signature Enforcement (DSE) mechanism that requires drivers to be properly signed with digital signatures in order to be loaded successfully. This mechanism was first introduced in Windows Vista 64-bit, and is affected for all versions of Windows since then:
After successfully loading, WINNKIT hooks network communication, and operates based on custom commands that are being sent from the aforementioned user-mode agent, DEPLOYLOG.
At the beginning of its execution, the driver validates the NDIS version, making sure the system is Windows Vista or above. By using the NDIS API, it communicates directly with the network card, skipping higher level communication protocols:
After establishing a connection with the network card, the rootkit tries to open the event \\BaseNamedObjects\\{75F09225-CD50-460B-BF90-5743B8404D73}. In case it fails, it creates this event, and then hooks the \\Device\\Null device. Hooking this device is somehow risky, as this device is often being targeted by modern rootkit, thus making it relatively exposed to detection. Nevertheless, it enabled the authors to stay undetected for years.
Using the above mentioned mechanisms enables WINNKIT a mean of communication with the user mode agent:
A summary of the communication flow of DEPLOYLOG and WINNKIT, can be seen in the following diagram:
Below are the functionality we believe that each code represents, according to our findings and previous conducted research:
|
Command |
Operation |
|
0x100 |
Hide driver |
|
0x200 |
Determine version |
|
0x300 |
Access IRQL shared data |
|
0x400 |
Map and allocate buffer |
|
0x500 |
Map a buffer |
|
0x800 |
Clean up |
Winnti used reflective loading injection in order to evade detection. The malicious modules are reflectively injected into the legitimate svchost processes. The following modules were detected by Cybereason and seem consistent with previously reported Winnti plugins:
In part two of the research, we provided a deep dive into the Winnti malware arsenal that was observed by the Cybereason IR and Nocturnus teams. Our analysis provides a unique and holistic view of Winnti operational aspects, capabilities and modus operandi. While some of the tools mentioned in the research were previously reported on, some tools such as DEPLOYLOG were previously undocumented and first analyzed in this report. In addition, our analysis provides further insights regarding some of the known Winnti tools.
Perhaps one of the most interesting things to notice is the elaborate and multi-phased infection chain Winnti employed. The malware authors chose to break the infection chain into multiple interdependent phases, where each phase relies on the previous one in order to execute correctly. This demonstrates the thought and effort that was put into both the malware and operational security considerations, making it almost impossible to analyze unless all pieces of the puzzle are assembled in the correct order.
Furthermore, the rare abuse of the Windows’ own CLFS logging system and NTFS manipulations provided the attackers with extra stealth and the ability to remain undetected for years.
We hope that this report helps to shed light on Winnti operations, tools and techniques, and that it will assist to expose further intrusions.
This research has not been possible without the tireless effort, analysis, attention to details and contribution of the Cybereason Incident Response team. Special thanks and appreciation goes to Matt Hart, Yusuke Shimizu, Niamh O’Connor, Jim Hung, and Omer Yampel.
LOOKING FOR THE IOCs? CLICK ON THE CHATBOT DISPLAYED IN LOWER-RIGHT OF YOUR SCREEN FOR ACCESS. Due to the sensitive nature of the attack, not all IOCs observed by Cybereason can be shared in our public report. Please contact us for more information.
|
Reconnaissance |
Initial Access |
Execution |
Persistence |
Privilege Escalation |
Defense Evasion |
|
Credential Access |
Discovery |
Lateral movement |
Collection |
Exfiltration |
Command and Control |
Chen has almost a decade of experience in Threat Intelligence & Research, Incident Response and Threat Hunting. Before joining Cybereason, Chen spent three years dissecting APTs, investigating underground cybercriminal groups and discovering security vulnerabilities in known vendors. Previously, he served as a Security Researcher in the military forces.
Fusao spent over 10 years in the security industry. Before joining, he worked as a mobile malware researcher and a developer at the security vendor and then worked at the global mobile phone manufacturer for the development of AntiVirus, VPN client on their Android mobile phone.
Fusao joined Cybereason in 2019 and was previously the Senior Security Analyst at the Advanced Services Team in Cybereason Japan where delivered various security professional services, Incident Response, consultation and triage malware activity alerts in SOC.
Ofir is a Incident Response Engineer at Cybereason who has a keen interest in Windows Internals, reverse engineering, memory analysis and network anomalies. He has years of experience in Cyber Security, focusing on Malware Research, Incident Response and Threat Hunting. Ofir started his career as a Security Researcher in the military forces and then became a malware researcher focusing on Banking Trojans.
Akihiro is the Senior Manager of Global Security Practice, leading Incident Response team in the APAC region and Japan. Akihiro has led a substantial number of large-scale Incident Response, Digital Forensics and Compromise Assessment engagements during recent years. Akihiro was also a former Team lead of Advanced Security Services team responsible for managing, developing, delivering a variety of professional services including Proactive threat hunting, Security Posture Assessment, Advanced security training and consulting services at Cybereason.
Niv, IR Practice Director, leads Cybereason's incident response practice in the EMEA region. Niv began his career a decade ago in the Israeli Air Force as a team leader in the security operations center, where he specialized in incident response, forensics, and malware analysis. In former roles at Cybereason, he focused on threat research that directly enhances product detections and the Cybereason threat hunting playbook, as well as the development of new strategic services and offerings.
With a decade in malware research, Daniel uses his expertise with malware analysis and reverse engineering to understand APT activity and commodity cybercrime attackers. Daniel has previously shared research at RSA Conference, the Microsoft Digital Crimes Consortium, and Rootcon.
Assaf has over 15 years in the InfoSec industry. He started his career in the military forces Cybersecurity unit where he developed extensive experience in offensive security. Later in his career he led Red Teams, developed penetration testing methodologies, and specialized in malware analysis and reverse engineering.