Igor’s Blog

APT Encounters of the Third Kind

2021-03-24T00:00:00-04:00

A few weeks ago an ordinary security assessment turned into an incident response whirlwind. It was definitely a first for me, and I was kindly granted permission to outline the events in this blog post. This investigation started scary but turned out be quite fun, and I hope reading it will be informative to you too. I'll be back to posting about my hardware research soon.

How it started
What hell is this?
The NFS Server
2nd malicious binary
Further forensics
Eureka Moment
The GOlang thingy
How the kernel got patched? and why not the golang app?
What we have so far
Q&A

How it started

Twice a year I am hired to do security assessments for a specific client. We have been working together for several years, and I had a pretty good understanding of their network and what to look for.

This time my POC, Klaus, asked me to focus on privacy issues and GDPR compliance. However, he asked me to first look at their cluster of reverse gateways / load balancers:

I had some prior knowledge of these gateways, but decided to start by creating my own test environment first. The gateways run a custom Linux stack: basically a monolithic compiled kernel (without any modules), and a static GOlang application on top. The 100+ machines have no internal storage, but rather boot from an external USB media that has the kernel and the application. The GOlang app serves in two capacities: an init replacement and the reverse gateway software. During initialization it mounts /proc, /sys, devfs and so on, then mounts an NFS share hardcoded in the app. The NFS share contains the app's configuration, TLS certificates, blacklist data and a few more. It starts listening on 443, filters incoming communication and passes valid requests on different services in the production segment.

I set up a self contained test environment, and spent a day in black box examination. Having found nothing much I suggested we move on to looking at the production network, but Klaus insisted I continue with the gateways. Specifically he wanted to know if I could develop a methodology for testing if an attacker has gained access to the gateways and is trying to access PII (Personally Identifiable Information) from within the decrypted HTTP stream.

I couldn't SSH into the host (no SSH), so I figured we will have to add some kind of instrumentation to the GO app. Klaus still insisted I start by looking at the traffic before (red) and after the GW (green), and gave me access to a mirrored port on both sides so I could capture traffic to a standalone laptop he prepared for me and I could access through an LTE modem but was not allowed to upload data from:

The problem I faced now was how to find out what HTTPS traffic corresponded to requests with embedded PII. One possible avenue was to try and correlate the encrypted traffic with the decrypted HTTP traffic. This proved impossible using timing alone. However, unspecting the decoded traffic I noticed the GW app adds an 'X-Orig-Connection' with the four-tuple of the TLS connection! Yay!

I wrote a small python program to scan the port 80 traffic capture and create a mapping from each four-tuple TLS connection to a boolean - True for connection with PII and False for all others:

4.254.254,443,[Redacted],43404,376106847.319,False
4.254.254,443,[Redacted],52064,376106856.146,False
4.254.254,443,[Redacted],40946,376106856.295,False
4.254.254,443,[Redacted],48366,376106856.593,False
4.254.254,443,[Redacted],48362,376106856.623,True
4.254.254,443,[Redacted],45872,376106856.645,False
4.254.254,443,[Redacted],40124,376106856.675,False 
...

With this in mind I could now extract the data from the PCAPs and do some correlations. After a few long hours getting scapy to actually parse timestamps consistently enough for comparisons, I had a list of connection timing information correlated with PII. A few more fun hours with Excel and I got histogram graphs of time vs count of packets. Everything looked normal for the HTTP traffic, although I expected more of a normal distribution than the power-low type thingy I got. Port 443 initially looked the same, and I got the normal distribution I expected. But when filtering for PII something was seriously wrong. The distribution was skewed and shifted to longer time frames. And there was nothing similar on the port 80 end.

My only explanation was that something was wrong with my testing setup (the blue bars) vs. the real live setup (the orange bars). I wrote on our slack channel 'I think my setup is sh*t, can anyone resend me the config files?', but this was already very late at night, and no one responded. Having a slight OCD I couldn’t let this go. To my rescue came another security? feature of the GWs: Thet restarted daily, staggered one by one, with about 10 minutes between hosts. This means that every ten minutes or so one of them would reboot, and thus reload it’s configuration files over NFS. And since I could see the NFS traffic through the port mirror I had access to, I recokoned I could get the production configuration files from the NFS capture (bottom dotted blue line in the diagram before).

So to cut a long story short I found the NFS read reply packet, and got the data I need. But … why the hack is eof 77685??? Come on people, its 3:34AM!

What's more, the actual data was 77685 bytes, exactly 8192 bytes more then the ‘Read length’. The entropy for that data was pretty uniform, suggesting it was encrypted. The file I had was definitely not encrypted.

Histogram of extra 8192 bytes:

When I mounted the NFS export myself I got a normal EOF value of 1!

What hell is this?

Comparing the capture from my testing machine with the one from the port mirror I saw something else weird:

For other NFS open requests (on all of my test system captures and for other files in the production system) we get:

Spot the difference?

The open id: string became open-id:. Was I dealing with some corrupt packet? But the exact same problem reappeared the next time blacklist.db was send over the wire by another GW host.

Time to look at the kernel source code:

The “open id” string is hardcoded. What's up?

After a good night sleep and no beer this time I repeated the experiment and convincing myself I was not hullucinating I decided to compare the source code of the exact kernel version with the kernel binary I got.

What I expected to see was this (from nfs4xdr.c):

static inline void encode_openhdr(struct xdr_stream *xdr, const struct nfs_openargs *arg)
{
    __be32 *p;
 /*
 * opcode 4, seqid 4, share_access 4, share_deny 4, clientid 8, ownerlen 4,
 * owner 4 = 32
 */
    encode_nfs4_seqid(xdr, arg->seqid);
    encode_share_access(xdr, arg->share_access);
    p = reserve_space(xdr, 36);
    p = xdr_encode_hyper(p, arg->clientid);
    *p++ = cpu_to_be32(24);
    p = xdr_encode_opaque_fixed(p, "open id:", 8);
    *p++ = cpu_to_be32(arg->server->s_dev);
    *p++ = cpu_to_be32(arg->id.uniquifier);
    xdr_encode_hyper(p, arg->id.create_time);
}

Running binwalk -e -M bzImage I got the internal ELF image, and opened it in IDA. Of course I didn’t have any symbols, but I got nfs4_xdr_enc_open() from /proc/kallsyms, and from there to encode_open() which led me to encode_openhdr(). With some help from hex-rays I got code that looked very similiar, but with one key difference:

static inline void encode_openhdr(struct xdr_stream *xdr, const struct nfs_openargs *arg)
{
    ...
    p = xdr_encode_opaque_fixed(p, unknown_func("open id:", arg), 8);
    ...
}

The function unknown_func was pretty long and complicated but eventually sometimes decided to replace the space between 'open' and 'id' with a hyphen.

Does the NFS server care? Apparently this string it is some opaque client identifier that is ignored by the NFS server, so no one would see the difference. That is unless they were trying to extract something from an NFS stream, and obviously this was not a likely scenario. OK, back to the weird 'eof' thingy from the NFS server.

The NFS Server

The server was running the 'NFS-ganesha-3.3' package. This is a very modular user-space NFS server that is implemented as a series of loadable modules called FSALs. For example support for files on the regular filesystem is implemented through a module called libfsalvfs.so. Having verified all the files on disk had the same SHA1 as the distro package, I decided to dump the process memory. I didn't have any tools on the host, so I used GDB which helpfully was already there. Unexpectadly GDB was suddenly killed, the file I specified as output got erased, and the nfs server process restarted.

I took the dump again but there was nothing special there!

I was pretty suspicious at this time, and wanted to recover the original dump file from the first dump. Fortunately for me I was dumping the file to the laptop, again over NFS. The file had been deleted, but I managed to recover it from the disk on that server.

2nd malicious binary

The memory dump was truncated, but had a corrupt version of NFS-ganesha inside. There were two libfsalvfs.so libraries loaded: the original one and an injected SO file with the same name. The injected file was clearly malicious. The main binary was patched in a few places, and the function table into libfsalvfs.so as replaced with the alternate libfsalvfs.so. The alternate file was compiled from NFS-ganesha sources, but modified to include new and improved (wink wink) functionality.

The most interesting of the new functionality were two separate implementations of covert channels.

The first one we encountered already:

When an open request comes in with 'open-id' instead of 'open id', the file handle is marked. This change is opaque to the NFS server, so unpatched servers just ignore it and nothing much happens.
For infiltrated NFS server, when the file handle opened this way is read, the NFS server appends the last block with a payload coming from the malware's runtime storage, and the 'eof' on-the-wire value is changed to be the new total size. An unpatched kernel (which shouldn’t really happen, since it marked the file in the first place) will just ignore the extra bytes. The EOF value is used as a bool, e.g. checked for 0 or not and not a specific value, so having a large integer values doesn’t change anything in the flow of an unmodified kernel.

The second covert channel is used for command and control, and is implemented in the VFS code as a fake directory.

Any writes to /<export>/.snapshot/meta/<cmdid> are handled by the malware code and not passed on to the FS. They are pseudo-files that implement commands through read and write operations.

The malware implemented the following commands:

1701 - self destruct
1702 - set auto self destruct time
1703 - run shell command
1704 - load SO file from buffer specified in command
1706 - get basic system description
1707 - get network connections
170A - upgrade to new SO file
74201 - put buffer in memory dict by ID
74202 - get buffer from memory dict by ID
74650 - put a payload in memory for the first covert channel
74651 - arm the first covert channel
74652 - disarm the first covert channel

For example cp payload.so /mnt/server/.snapshot/meta/1704 will load the SO file in the NFS ganesha process on the target server, and echo 1616580589 > /mnt/server/.snapshot/meta/1702 will set an autodestruct time. Reading this file will retrieve the time.

The self destruct command (0x1701) is very interesting - it sends a UDP broadcast on port 41701 with a random payload of size 321 bytes, and then restarts a clean NFS-ganesha. I guess this is some kind of network signalling.

It appears the malware has a watchdog that iterates over all /proc/<pid>/fd/* links every 25ms, and if a link is open to /proc/<malware pid>/maps or /proc/<malware pid>/smaps or /proc/<malware pid>/mem it:

kills the offending process,
deletes any writeable file handle it had opened (before it was killed) in /proc/<pid>/fd/, first trying to wipe it,
calls the self destruct command (0x1701).

Further forensics

Lets go back to that awful Saturday I triggered the self-destruct. Having a faint understanding I triggered something (that being before I reversed the devil) I asked Klaus to disconnect all the network connections to the outer world and we started taking memory dumps of whatever we could, storing them all on the laptop. In hindsight we destroyed quiet a bit of evidence by triggering more self destructs in other subnets, but I think the self destruct signal has already gone out to the bad guys through a different piece of malware that I later partially recovered, and probably "heard" the UDP distress signal (that's what is was called in the binary, not my naming).

After getting all the forensics the client insisted on reconnecting his systems to the web, they were "losing money". I switched from forensics to reversing. In the process, while inspecting the malicious libfsalvfs.so I discovered the commands I mentioned above, and discovered a "feature" that helped me fill more paces of the puzzle.

Reversing malware you always find some feeble attempt to obfuscate string using XOR or RC4, or just scrambling the letter ordering. In this case I pretty quickly found a function I called get_obfuscated_string(buffer, string_id). The difference however, was that this one was just horrendous, practically irreversible:

It had like a billion nested switches:

I think they let some intern fresh out of college write that one. It seems the complete list of strings used by the tool are encoded inside in a tree of nested switches, with a variable length encoding, e.g. in one branch the 2nd level might have 3 bits and in another it might have 5 and in a third only a single bit. Some kind of prefix tree if I remember anything from Uni.

Eventually I managed to write code to just brute force the function:

#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <string>
#include <set>

int main(int argc, char* argv[])
{
	// error handling code omitted
	const char* filename = (argc > 1) ? argv[1] : "reconstructed.elf";
	unsigned long offset = (argc > 1) ? strtol(argv[2], NULL, 16) : 0x22a0;

	int fd = open(filename, O_RDONLY);
	struct stat stbuf;
	fstat(fd, &stbuf);
	const char* addr = (char*)mmap(NULL, stbuf.st_size, PROT_READ | PROT_EXEC, MAP_PRIVATE, fd, 0);
	close(fd);
	const char* base = addr + offset;

	typedef int (*entry_t)(char* outbuf, int id);
	entry_t entry = (entry_t)base;
	std::set<std::string> found;
	char buffer[1024];
	
	for(long bits = 1; bits < 64; ++ bits) {
		bool any_new = false;
		for(long id = (bits == 1) ? 0 : (1 << (bits - 1)); id < (1<<bits); ++ id) {
			int status = entry(buffer, id);
			if(status == 0)
				continue;
			if(found.find(buffer) != found.end())
				continue;
			found.insert(buffer);
			printf("Got '%s'! [0x%x]\n", buffer, id);
			any_new = true;
		}
		if(!any_new)
			break;
	}

	return 0;
}

This first binary had the following strings (I am keeping 3 to myself as they have client related info):

'/proc/self/mem', 
'/proc/self/maps',
'/proc/self/cwd',
'/proc/self/environ',
'/proc/self/fd/%d',
'/proc/self/fdinfo/%d',
'/proc/self/limits',
'/proc/self/cgroup',
'/proc/self/exe',
'/proc/self/cmdline',
'/proc/self/mounts',
'/proc/self/smaps',
'/proc/self/stat',
'/proc/%d/mem', 
'/proc/%d/maps',
'/proc/%d/cwd',
'/proc/%d/environ',
'/proc/%d/fd/%d',
'/proc/%d/fdinfo/%d',
'/proc/%d/limits',
'/proc/%d/cgroup',
'/proc/%d/exe',
'/proc/%d/cmdline',
'/proc/%d/mounts',
'/proc/%d/smaps',
'/proc/%d/stat',        
'nfs',
'nfs4',
'tmpfs',
'devtmpfs',
'procfs',
'sysfs',
'WSL2',
'/etc/os-release',
'/etc/passwd',
'/etc/lsb-release',
'/etc/debian_version',
'/etc/redhat-release',
'/home/%s/.ssh',
'/var/log/wtmp',
'/var/log/syslog',
'/var/log/auth.log',
'/var/log/cron.log',
'/var/log/syslog.log',
'/etc/netplan/*.yaml',
'/etc/yp.conf',
'/var/yp/binding/',
'/etc/krb5.conf',
'/var/kerberos/krb5kdc/kdc.conf',
'/var/log/ganesha.log',
'/etc/ganesha/ganesha.conf',
'/etc/ganesha/exports',
'/etc/exports',
'Error: init failed',
'DELL',
'/usr/lib/x86_64-linux-gnu/libnfs.so.4',
'/tmp/.Test-unix/.fa76c5adb8c04239ff3034106842773b',
'Error: config missing',
'Error: sysdep missing',
'Running',
'LOG',
'/usr/lib/x86_64-linux-gnu/ganesha/libfsalvfs.so',
'none',
'/etc/sudoers',
'/proc/net/tcp',
'/proc/net/udp',
'/etc/selinux/config',
'libdl.so.2',
'libc-',
'.so',
'cluster-config',
'recovery-signal',

Eureka Moment

Staring endlessly at this weird function I thought to myself: maybe I can look for code that is structured like this in all the dumps we obtained. We have all those block of mov byte ptr [rdi+?], '?':

So lets look for blocks of code that are highly dense with these opcodes:

import sys

with open(sys.argv[1], 'rb') as f:
    data = f.read()

STATE=None
for i in range(len(data) - 6):
    if ord(data[i]) == 0xc6 and ord(data[i + 1]) == 0x47:
        if STATE and (STATE[0] + STATE[1] + 0x40) >= i:
            STATE[1] = i - STATE[0]
            STATE[2] += 1
        else:
            if STATE and STATE[2] >= 20:
                print('Found region at 0x%x - 0x%x' % (STATE[0], STATE[0] + STATE[1]))
            STATE = [i, 4, 1] 

And I found them. Oh I did. Some adjustment even led to a version for ARM systems:

The GOlang thingy

I finally found the payload that was sent over to the GW machines. It had 2 stages: the first was the 8192 buffer loaded through the first covert channel. The kernel was modified to inject this buffer into the GOlang application and hook it. This will get fairly technical, but I enjoyed it and so will you:

First note that in the Golang stdlib an HTTP connection can be read through the net/http.(connReader).Read function. The calls are made through a io.Reader interface, so the calls are made through a virtual table, and the call locations cannot be statically identified.
the kernel inject begins by allocating a bunch of RWX memory immediately after the GOlang binary - let's call it the trampoline area, and it will include two types of generated trampoline functions,
Next the ELF symbol table was used to find the 'net/http.(*connReader).Read' symbol,
What we’ll call the 1st trampoline function (code below) is copied to the trampoline area, patching the area marked with HERE with the first 9 bytes of net/http.(\*connReader).Read
mprotect(net_http_connReader_read & ~0xfff, 8192, PROT_EXEC | PROT_READ | PROT_WRITE)
modified the beginning of net/http.(\*connReader).Read to a near jump into the trampoline - using 5 bytes of the 9 original used by 'move rcx, fs:….' that are the preamble to function.

First trampoline function

     pop     rax            
     pop     rcx
     push    rcx
     push    rax
     mov     r11, cs:qword_<relocated>
     mov     rdi, rcx
     call    qword ptr [r11+8]
     pop     rax
     pop     rcx
     push    rcx
     mov     rcx, fs:0FFFFFFFFFFFFFFF8h <---- HERE
     cmp     rsp, [rcx+10h]
     jmp     rax

When the trampoline is called (from the new near jump in the beginning of net/http.(*connReader).Read) it examines the stack to locate the return address, and checks if a second type of trampoline we'll refer to as the return trampoline has already been allocated for the return address for the function,
If not it allocates a new trampoline per call location of net/http.(*connReader).Read from the code below, replacing 123456789ABCDEFh with the absolute address of a function in the malware,
GOlang uses memory for all function argument passing, so immediately after the virtual function call to Read() there will always be a 5 byte mov reg, [rsp+?] to load Read()'s result into a register. This mov instruction is copied into the first db 5 dup(0) area,
those same 5 bytes are then replacing with a near jump to the 2nd trampoline

the 2nd db 5 dup(0) are filled with a relative near jmp back to the original code patch site.

    mov     rax, 123456789ABCDEFh
    mov     rdi, rsp
    call    rax
    db 5 dup(0)
    db 5 dup(0)      

This way eventually all the net/http.(*connReader).Read call sites are patched to call a function immediatly after net/http.(*connReader).Read virtual call returns. This lets the malicous code inspect the decoded HTTP packet.

On initialization the 1st stage malware also loads the hefty 2nd stage through the 2nd covert channel, and passes each buffer received from the patch on net/http.(*connReader).Read to it for inspection. The data collected is collected and compressed by the malware and stored back to the NFS server (the 2nd covert channel which bypasses read ACLs on NFS).

Before this case I did not think there was any nice way to hook random GO binaries, this technique is pretty cool.

Unfortunatly I cannot discuss what the 2nd payload actually as it will reveal stuff my employer isn't ready for yet.

How the kernel got patched? and why not the golang app?

The golang app is built inside the CI/CD network segment. This segment can only be accessed through monitored jump hosts with MFA. Each day, the CI/CD pipline clones the source code from the GIT server, builds it, and automatically tests it in a pre-production segment. Once tested it gets digitally signed and uploaded to the NFS server. The running app self updates, checking the digital signature beforehand.

The kernel, on the other hand, is manually built by the guy responsible for it on his own laptop. He then digitally signs it and stores it on a server where it is used by the CI/CD pipeline. Fortunatly for us a commented out line in a script in the CI/CD pipline (a line that was not commented out in the GIT!) did not delete old versions of the kernel and we know which versions were tampered with.

We noticed a 3 month gap about 5 month ago, and it corresponded with the guy moving the kernel build from a Linux laptop to a new Windows laptop with a VirtualBox VM in it for compiling the kernel. It looks as if it took the attackers three months to gain access back into the box and into the VM build.

What we have so far

We found a bunch of malware sitting in the network collecting PII information from incoming HTTPS connection after they are decoded in a GOlang app. The data is exfiltrated through the malware network and eventually is sent to the bad guys. We have more info but I am still working on it, expect another blog post in the future with more details, samples, etc’.

Q&A

Q: What was the initial access vector?

A: We have a pretty good idea, but I cannot publish it yet (RD and stuff). Stay tuned!
Q: Why didn't you upload anything to VT yet?

A: A few reasons:
- I need to make sure no client info is in the binaries - some of the binaries have hardcoded strings that cannot be shared
- All of the binaries I have have been reconstructed from memory dumps, so are not in their original form. Does anyone know how to upload partial dumps into VT?
Q: It there a security vulnerability in GO? in the Kernel?

A: Defenitly not! this is just an obnoxious attacker doing what obnoxious attacker do. I might even say the complexity of the stuff means they don’t have a 0day for this platform.
Q: What about YARA rules, C2 address, etc'?

A: Wait for it, there is a lot more coming!
Q: Why did you publish instead of collecting more?

A: To quote the client "I don't care who else they are attacking. I just want them off my lawn!", and he thinks publishing will prevent them from returning to THIS network. Hopefully what we publish next time will get them off other people’s lawns.
Q: Any Windows malware?

A: Definitly, including what we believe is an EDR bypass. Still working on it.
Q: Any zero days?

A: Maybe …
Q: Who are these bad guys you keep refering to?

A: No clue. Didn’t find anything similiar published. There is now sure way to make anything except unsubstantiated guesses, and I won’t do that.

To be continued.

Security of the Intel Graphics Stack - Part 2 - FW <-> GuC

2021-02-24T00:00:00-05:00

Today we'll continue our voyage into the graphics subsystem components.

The question we'll try to answer is what kind of communications occur between the GuC and the rest of the system. In this post we'll look at firmware components and next post at Windows components.

For a reminder what the GuC is, look at part1 post .

Part 1: The IntelGOP DXE driver
Part 2: From CSME
- Conclusion

Part 1: The IntelGOP DXE driver

The Intel Graphics Output Protocol (GOP) EFI DXE driver can be extracted in various versions from many UEFI capsules available through many vendors. For this post I redid my original analysis on a recent version from a CanonLake system.

The purpose of this exercise is to try and see whether the GOP driver communicates with the GuC over the PCIe bus (TL;dr: it doesn't)

The binary isn't to large - 84KB, so we can try to completely reverse engineer it. I used both IDA+HexRays and a dynamic analysis UEFI emulator I developed for just these cases. The emulator lets you run EFI DXE drivers in Windows simulating many UEFI services and allowing me to modify/inspect EFI interfaces, hook UEFI protocol structs, and even has some fuzzing capabilities.

Looking at the driver's entrypoint we see it stores the different service tables in globals and then jumps to the main() functions I called GopEntryPoint().

.text:0000000000001580 ; EFI_STATUS __fastcall ModuleEntryPoint(EFI_HANDLE ImageHandle, EFI_SYSTEM_TABLE *SystemTable)
.text:0000000000001580                 public _ModuleEntryPoint
.text:0000000000001580 _ModuleEntryPoint proc near             ; DATA XREF: HEADER:00000000000000E8↑o
.text:0000000000001580                 sub     rsp, 28h
.text:0000000000001584                 mov     r8, [rdx+60h]
.text:0000000000001588                 mov     rax, [rdx+58h]
.text:000000000000158C                 mov     cs:gIMAGE_HANDLE, rcx
.text:0000000000001593                 mov     cs:gBOOT_SERVICES, r8
.text:000000000000159A                 mov     cs:gRUNTIME_SERVICES, rax
.text:00000000000015A1                 mov     cs:gBOOT_SERVICES2, r8
.text:00000000000015A8                 mov     cs:gSYSTEM_TABLE2, rdx
.text:00000000000015AF                 call    GopEntryPoint
.text:00000000000015B4                 add     rsp, 28h
.text:00000000000015B8                 retn
.text:00000000000015B8 _ModuleEntryPoint endp

GopEntryPoint() first part is really boring, just setting up version information in global strings.

_int64 __fastcall GopEntryPoint(EFI_HANDLE img_handle_arg)
{
  EFI_HANDLE image_handle; // rbx
  CHAR16 *driver_desc_ptr; // rax
  __int64 img_handle; // r11
  __int64 result; // rax
  EFI_HANDLE Handle; // [rsp+50h] [rbp+18h]
  EFI_LOADED_IMAGE_PROTOCOL *Interface; // [rsp+58h] [rbp+20h]

  image_handle = img_handle_arg;
  v2 = atoi(L"0") == 1;
  driver_desc_ptr = gDriverDescription;
  v4 = 'I';
  byte_142A0 = v2;
  do
  {
    *driver_desc_ptr = v4;
    ++driver_desc_ptr;
    v4 = *(CHAR16 *)((char *)driver_desc_ptr + (char *)L"Intel(R) GOP Driver" - (char *)gDriverDescription);
  }
  while ( v4 );
  *driver_desc_ptr = 0;
  strcat(gDriverDescription, L" [");
  strcat(gDriverDescription, L"11");
  strcat(gDriverDescription, L".");
  strcat(gDriverDescription, L"0");
  strcat(gDriverDescription, L".");
  strcat(gDriverDescription, L"1014");
  strcat(gDriverDescription, L"]");
  gDriverState.ImgHandle = img_handle;
  v12 = &gDriverVersion;
  v13 = '1';
  do
  {
    *v12 = v13;
    ++v12;
    v13 = *(CHAR16 *)((char *)v12 + (char *)L"11" - (char *)&gDriverVersion);
  }
  while ( v13 );
  *v12 = 0;
  strcat(&gDriverVersion, L".");
  strcat(&gDriverVersion, L"0");
  strcat(&gDriverVersion, L".");
  strcat(&gDriverVersion, L"1014");
  gDriverState.ControllerName = (__int64)L"Intel(R) Graphics Controller";
  gDriverState.DriverVersion = v17;
  atoi(L"11");
  atoi(L"0");
  v18 = atoi(L"1014");

The second part does the actual work. First it looks for the EFI_LOADED_IMAGE_PROTOCOL to setup a the unload routine:

  gDRIVER_BINDING_PROTOCOL.Version = v18 + v19;
  result = gBOOT_SERVICES->OpenProtocol(
             image_handle,
             &EFI_LOADED_IMAGE_PROTOCOL_GUID,
             (void **)&Interface,
             image_handle,
             image_handle,
             2u);
  if ( result >= 0 )
  {
    Interface->Unload = (EFI_IMAGE_UNLOAD)UnloadImage;

And then install four protocol handlers, three of which I identified: one for driver binding and two for component name handling. The InstallMultipleProtocolInterfaces(..) can accept multiple protocols, each protocol has a GUID and the “virtual table” like structure used by UEFI. The final entry is NULL. Most UEFI protocol GUIDs are public (and appear in the EDK) so we can identify them easily and this identify the virtual table structures associated with them, for example for the UEFI binding protocol we have in DriverBinding.h:

#define EFI_DRIVER_BINDING_PROTOCOL_GUID \
	{0x18A031AB,0xB443,0x4D1A,0xA5,0xC0,0x0C,0x09,0x26,0x1E,0x9F,0x71}

GUID_VARIABLE_DECLARATION(gEfiDriverBindingProtocolGuid, EFI_DRIVER_BINDING_PROTOCOL_GUID);

typedef struct _EFI_DRIVER_BINDING_PROTOCOL EFI_DRIVER_BINDING_PROTOCOL;

typedef EFI_STATUS (EFIAPI *EFI_DRIVER_BINDING_PROTOCOL_SUPPORTED) (
	IN EFI_DRIVER_BINDING_PROTOCOL *This, 
	IN EFI_HANDLE ControllerHandle,
	IN EFI_DEVICE_PATH_PROTOCOL *RemainingDevicePath OPTIONAL
);

typedef EFI_STATUS (EFIAPI *EFI_DRIVER_BINDING_PROTOCOL_START) (
	IN EFI_DRIVER_BINDING_PROTOCOL *This,
	IN EFI_HANDLE ControllerHandle,
	IN EFI_DEVICE_PATH_PROTOCOL *RemainingDevicePath OPTIONAL
);

typedef EFI_STATUS (EFIAPI *EFI_DRIVER_BINDING_PROTOCOL_STOP) (
	IN EFI_DRIVER_BINDING_PROTOCOL *This,
	IN EFI_HANDLE ControllerHandle,
	IN UINTN NumberOfChildren,
	IN EFI_HANDLE *ChildHandleBuffer OPTIONAL
);

struct _EFI_DRIVER_BINDING_PROTOCOL {
	EFI_DRIVER_BINDING_PROTOCOL_SUPPORTED Supported;
	EFI_DRIVER_BINDING_PROTOCOL_START Start;
	EFI_DRIVER_BINDING_PROTOCOL_STOP Stop;
	UINT32 Version;
	EFI_HANDLE ImageHandle;
	EFI_HANDLE DriverBindingHandle;
};

This enables us to reverse the rest of GopEntryPoint:

    Handle = image_handle;
    gBOOT_SERVICES->InstallMultipleProtocolInterfaces(
      &Handle,
      &EFI_DRIVER_BINDING_PROTOCOL_GUID,
      &gDRIVER_BINDING_PROTOCOL,
      &EFI_COMPONENT_NAME2_PROTOCOL_GUID,
      &gCOMPONENT_NAME2_PROTOCOL,
      0i64);
    gDRIVER_BINDING_PROTOCOL.DriverBindingHandle = Handle;
    gDRIVER_BINDING_PROTOCOL.ImageHandle = image_handle;
    gBOOT_SERVICES->InstallMultipleProtocolInterfaces(
      &gDRIVER_BINDING_PROTOCOL.DriverBindingHandle,
      &UNKNOWN_PROTOCOL_GUID,
      &gDriverState.unknwon_proto,
      0i64);
    result = gBOOT_SERVICES->InstallMultipleProtocolInterfaces(
               &gDRIVER_BINDING_PROTOCOL.DriverBindingHandle,
               &GOP_COMPONENT_NAME2_PROTOCOL_GUID,
               &gGOP_COMPONENT_NAME2_PROTOCOL,
               0i64);
    if ( result >= 0 )
      qword_142B0 = (__int64)image_handle;
  }
  return result;
}

All the GUID values appear close to each other at the beginning of the binary, so we can take a shortcut and find all the GUIDs the driver uses:

.text:0000000000000240 EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID dd 9042A9DEh            ; Data1
.text:0000000000000240                                         ; DATA XREF: HEADER:00000000000000EC↑o
.text:0000000000000240                                         ; HEADER:00000000000001D4↑o ...
.text:0000000000000240                 dw 23DCh                ; Data2
.text:0000000000000240                 dw 4A38h                ; Data3
.text:0000000000000240                 db 96h, 0FBh, 7Ah, 0DEh, 0D0h, 80h, 51h, 6Ah; Data4
.text:0000000000000250 EFI_EDID_ACTIVE_PROTOCOL_GUID dd 0BD8C1056h           ; Data1
.text:0000000000000250                                         ; DATA XREF: InstallGraphicsProto+124↓o
.text:0000000000000250                                         ; uninstall2?+9B↓o ...
.text:0000000000000250                 dw 9F36h                ; Data2
.text:0000000000000250                 dw 44ECh                ; Data3
.text:0000000000000250                 db 92h, 0A8h, 0A6h, 33h, 7Fh, 81h, 79h, 86h; Data4
.text:0000000000000260 EFI_EDID_DISCOVERED_PROTOCOL_GUID dd 1C0C34F6h            ; Data1
.text:0000000000000260                                         ; DATA XREF: sub_1CA4+2A5↓o
.text:0000000000000260                                         ; InstallGraphicsProto+DF↓o ...
.text:0000000000000260                 dw 0D380h               ; Data2
.text:0000000000000260                 dw 41FAh                ; Data3
.text:0000000000000260                 db 0A0h, 49h, 8Ah, 0D0h, 6Ch, 1Ah, 66h, 0AAh; Data4
.text:0000000000000270 GOP_DISPLAY_BRIGHTNESS_PROTOCOL_GUID dd 6FF23F1Dh            ; Data1
.text:0000000000000270                                         ; DATA XREF: sub_1F78+B1↓o
.text:0000000000000270                                         ; uninstall2?+14B↓o ...
.text:0000000000000270                 dw 877Ch                ; Data2
.text:0000000000000270                 dw 4B1Bh                ; Data3
.text:0000000000000270                 db 93h, 0FCh, 0F1h, 42h, 0B2h, 0EEh, 0A6h, 0A7h; Data4
.text:0000000000000280 GOP_DISPLAY_BIST_PROTOCOL_GUID dd 0F51DD33Ah           ; Data1
.text:0000000000000280                                         ; DATA XREF: sub_1F78+75↓o
.text:0000000000000280                                         ; uninstall2?+F5↓o ...
.text:0000000000000280                 dw 0E57Fh               ; Data2
.text:0000000000000280                 dw 4020h                ; Data3
.text:0000000000000280                 db 0B4h, 66h, 0F4h, 0C1h, 71h, 0C6h, 0E4h, 0F7h; Data4
.text:0000000000000290 EFI_PCI_IO_PROTOCOL_GUID dd 4CF5B200h            ; Data1
.text:0000000000000290                                         ; DATA XREF: DriverBindingProtoSupported+CB↓o
.text:0000000000000290                                         ; DriverBindingProtoSupported+173↓o ...
.text:0000000000000290                 dw 68B8h                ; Data2
.text:0000000000000290                 dw 4CA5h                ; Data3
.text:0000000000000290                 db 9Eh, 0ECh, 0B2h, 3Eh, 3Fh, 50h, 2, 9Ah; Data4
.text:00000000000002A0 GOP_COMPONENT_NAME2_PROTOCOL_GUID dd 651B7EBDh            ; Data1
.text:00000000000002A0                                         ; DATA XREF: GopEntryPoint+22F↓o
.text:00000000000002A0                 dw 0CE13h               ; Data2
.text:00000000000002A0                 dw 41D0h                ; Data3
.text:00000000000002A0                 db 82h, 0E5h, 0A0h, 63h, 0ABh, 0BEh, 9Bh, 0B6h; Data4
.text:00000000000002B0 UNKNOWN_PROTOCOL_GUID dd 0DBCB2FCDh           ; Data1
.text:00000000000002B0                                         ; DATA XREF: UnloadImage+9A↓o
.text:00000000000002B0                                         ; GopEntryPoint+203↓o
.text:00000000000002B0                 dw 0E29Ah               ; Data2
.text:00000000000002B0                 dw 410Eh                ; Data3
.text:00000000000002B0                 db 9Dh, 0D9h, 0FAh, 9Dh, 5Fh, 0F4h, 0CDh, 0A7h; Data4
.text:00000000000002C0 MAYBE_AUX_PROTOCOL_GUID? dd 0C7D4703Bh           ; Data1
.text:00000000000002C0                                         ; DATA XREF: DriverBindingProtoStartImp+2A8↓o
.text:00000000000002C0                                         ; DriverBindingProtoStop+70↓o
.text:00000000000002C0                 dw 0F36h                ; Data2
.text:00000000000002C0                 dw 4E51h                ; Data3
.text:00000000000002C0                 db 0A9h, 83h, 5Eh, 61h, 0ACh, 0B8h, 68h, 3Ch; Data4
.text:00000000000002D0 EFI_DEVICE_PATH_PROTOCOL_GUID dd 9576E91h             ; Data1
.text:00000000000002D0                                         ; DATA XREF: DriverBindingProtoSupported+5F↓o
.text:00000000000002D0                                         ; DriverBindingProtoSupported+A2↓o ...
.text:00000000000002D0                 dw 6D3Fh                ; Data2
.text:00000000000002D0                 dw 11D2h                ; Data3
.text:00000000000002D0                 db 8Eh, 39h, 0, 0A0h, 0C9h, 69h, 72h, 3Bh; Data4
.text:00000000000002E0 ; EFI_GUID EFI_LOADED_IMAGE_PROTOCOL_GUID
.text:00000000000002E0 EFI_LOADED_IMAGE_PROTOCOL_GUID dd 5B1B31A1h            ; Data1
.text:00000000000002E0                                         ; DATA XREF: GopEntryPoint+169↓o
.text:00000000000002E0                 dw 9562h                ; Data2
.text:00000000000002E0                 dw 11D2h                ; Data3
.text:00000000000002E0                 db 8Eh, 3Fh, 0, 0A0h, 0C9h, 69h, 72h, 3Bh; Data4
.text:00000000000002F0 EFI_DRIVER_BINDING_PROTOCOL_GUID dd 18A031ABh            ; Data1
.text:00000000000002F0                                         ; DATA XREF: UnloadImage+BB↓o
.text:00000000000002F0                                         ; GopEntryPoint+1D2↓o
.text:00000000000002F0                 dw 0B443h               ; Data2
.text:00000000000002F0                 dw 4D1Ah                ; Data3
.text:00000000000002F0                 db 0A5h, 0C0h, 0Ch, 9, 26h, 1Eh, 9Fh, 71h; Data4
.text:0000000000000300 EFI_COMPONENT_NAME2_PROTOCOL_GUID dd 6A7A5CFFh            ; Data1
.text:0000000000000300                                         ; DATA XREF: UnloadImage+A1↓o
.text:0000000000000300                                         ; GopEntryPoint+1B8↓o
.text:0000000000000300                 dw 0E8D9h               ; Data2
.text:0000000000000300                 dw 4F70h                ; Data3
.text:0000000000000300                 db 0BAh, 0DAh, 75h, 0ABh, 30h, 25h, 0CEh, 14h; Data4

A few couldn't be identified. Another "fast forward" trick I can use is to find all locations protocols are installed or requested. If we look at how protocols are installed using gBOOT_SERVICES::InstallMultipleProtocolInterfaces:

.text:0000000000002938 FF 90 48 01 00 00                 call    qword ptr dword_148[rax]

We see the offset is pretty large, 0x148. We can just search for the wildcard "call qword ptr dword_148[reg]" and see if reg contains the global gBOOT_SERVICES. This way we can jump directly to the functions and identify what they do and name them:

Address	Function	Instruction
.text:000000000000188B	GopEntryPoint	                    FF 90 48 01 00 00                 call    [rax+EFI_BOOT_SERVICES.InstallMultipleProtocolInterfaces]
.text:00000000000018C3	GopEntryPoint	                    FF 90 48 01 00 00                 call    [rax+EFI_BOOT_SERVICES.InstallMultipleProtocolInterfaces]
.text:00000000000018E8	GopEntryPoint	                    FF 90 48 01 00 00                 call    [rax+EFI_BOOT_SERVICES.InstallMultipleProtocolInterfaces]
.text:0000000000001ECC	EnumConnectionsAndInstallEdidProto	FF 90 48 01 00 00                 call    [rax+EFI_BOOT_SERVICES.InstallMultipleProtocolInterfaces]
.text:0000000000001F50	EnumConnectionsAndInstallEdidProto	FF 90 48 01 00 00                 call    [rax+EFI_BOOT_SERVICES.InstallMultipleProtocolInterfaces]
.text:0000000000001FFA	InstallBrightnessProto	            FF 90 48 01 00 00                 call    [rax+EFI_BOOT_SERVICES.InstallMultipleProtocolInterfaces]
.text:0000000000002036	InstallBrightnessProto              FF 90 48 01 00 00                 call    [rax+EFI_BOOT_SERVICES.InstallMultipleProtocolInterfaces]
.text:000000000000221F	InstallGraphicsProto	            FF 90 48 01 00 00                 call    [rax+EFI_BOOT_SERVICES.InstallMultipleProtocolInterfaces]
.text:00000000000022A0	InstallGraphicsProto             	FF 90 48 01 00 00                 call    [rax+EFI_BOOT_SERVICES.InstallMultipleProtocolInterfaces]
.text:0000000000002938	DriverBindingProtoStartImp        	FF 90 48 01 00 00                 call    [rax+EFI_BOOT_SERVICES.InstallMultipleProtocolInterfaces]

This also gets as all the function tables for these protocols, and helps us understand the global state struct for the driver. Unlike C++, the UEFI function receive a This pointer that contains both data members and function pointers, for example for the GOP protocol:

...
typedef EFI_STATUS (EFIAPI *EFI_GRAPHICS_OUTPUT_PROTOCOL_BLT) (
    IN EFI_GRAPHICS_OUTPUT_PROTOCOL *This,
    IN EFI_GRAPHICS_OUTPUT_BLT_PIXEL *BltBuffer OPTIONAL,
    IN EFI_GRAPHICS_OUTPUT_BLT_OPERATION BltOperation,
    IN UINTN SourceX, IN UINTN SourceY,
    IN UINTN DestinationX, IN UINTN DestinationY,
    IN UINTN Width, IN UINTN Height,
    IN UINTN Delta OPTIONAL
);

typedef struct {
    UINT32 MaxMode;
    UINT32 Mode;
    EFI_GRAPHICS_OUTPUT_MODE_INFORMATION *Info;
    UINTN SizeOfInfo;
    EFI_PHYSICAL_ADDRESS FrameBufferBase;
    UINTN FrameBufferSize;
} EFI_GRAPHICS_OUTPUT_PROTOCOL_MODE;

struct _EFI_GRAPHICS_OUTPUT_PROTOCOL {
    EFI_GRAPHICS_OUTPUT_PROTOCOL_QUERY_MODE QueryMode;
    EFI_GRAPHICS_OUTPUT_PROTOCOL_SET_MODE SetMode;
    EFI_GRAPHICS_OUTPUT_PROTOCOL_BLT Blt;
    EFI_GRAPHICS_OUTPUT_PROTOCOL_MODE *Mode;
};

So the protocol structure has to be stored in some state structure. If the state structure is a singleton it can be stored as a global, but if we want multiple copies the driver allocates a state structure, places the protocol structure in a known offset within, and then can calculate the start of the structure from the This pointer provided to the protocol functions. We can use this information to try to piece together this global structre:

00000000 DriverState     struc ; (sizeof=0xE8, mappedto_92)
00000000                                         ; XREF: .text:gDriverState/r
00000000 language        dq ?                    ; offset
00000008 ImgHandle       dq ?                    ; XREF: GopEntryPoint+A9/w
00000010 field_10        dd ?
00000014 field_14        dd ?
00000018 graphics_proto  dq ?
00000020 field_20        dq ?                    ; XREF: GetDriverVersion+16/o
00000028 DriverVersion   dq ?                    ; XREF: GopEntryPoint+125/w
00000030 field_30        dq ?
00000038 active_proto_copy dq ?
00000040 field_40        dq ?                    ; XREF: GetControllerName+99/o
00000048 ControllerName  dq ?                    ; XREF: GopEntryPoint+11E/w
00000050 field_50        dq ?
00000058 field_58        dq ?
00000060 brightness_proto dq ?                   ; XREF: UnloadImage+8E/o
00000060                                         ; GopEntryPoint+1EE/o
00000068 name_proto      dq ?
00000070 bist_proto_orig GOP_DISPLAY_BIST_PROTOCOL_FUNC_TABLE ?
00000070                                         ; XREF: InstallBrightnessProto+50/o
00000080 bist_proto      GOP_DISPLAY_BIST_PROTOCOL ?
00000080                                         ; XREF: sub_44D8+21/o
00000080                                         ; sub_44D8+28/w ...
00000094 field_94        dd ?
00000098 field_98        dq ?                    ; XREF: sub_4900+24/o
00000098                                         ; sub_4900+2F/w ...
000000A0 field_A0        dq ?                    ; XREF: sub_4900+36/w
000000A0                                         ; sub_4900+319/r ...
000000A8 field_A8        dq ?                    ; XREF: sub_245C+14/r
000000A8                                         ; sub_245C+1B/o ...
000000B0 field_B0        dq ?                    ; XREF: sub_245C+86/r
000000B0                                         ; sub_259C+6C/r ...
000000B8 field_B8        dq ?                    ; XREF: sub_35A4+37A/o
000000B8                                         ; sub_35A4+384/w ...
000000C0 field_C0        dq ?                    ; XREF: sub_35A4+38B/w
000000C0                                         ; sub_35A4+3EF/r ...
000000C8 field_C8        dq ?
000000D0 field_D0        dq ?                    ; XREF: sub_35A4+420/o
000000D8 field_D8        dq ?
000000E0 field_E0        dq ?
000000E8 DriverState     ends

and so on.

It won't be too interesting to just dump more and more dissassembled functions here, as our goal is to find possible access to GuC. None of the functions I identified had any connection to the GuC, so next I looked at all accesses to PCI devices, as GuC accesses should be made using PCI. The devices are identified using EFI_DEVICE_PATH_PROTOCOL and accessed through EFI_PCI_IO_PROTOCOL_GUID.

DriverBindingProtoSupported+CB	`lea rdx, EFI_PCI_IO_PROTOCOL_GUID`
DriverBindingProtoSupported+173	`lea rdx, EFI_PCI_IO_PROTOCOL_GUID`
EnumConnectionsAndInstallEdidProto+259	`lea rdx, EFI_PCI_IO_PROTOCOL_GUID`
sub_245C+9C	`lea r8, EFI_PCI_IO_PROTOCOL_GUID`
sub_259C+33	`lea r8, EFI_PCI_IO_PROTOCOL_GUID`
sub_259C+81	`lea r8, EFI_PCI_IO_PROTOCOL_GUID`
DriverBindingProtoStartImp+44	`lea rdx, EFI_PCI_IO_PROTOCOL_GUID`
DriverBindingProtoStartImp+20C	`lea rdx, EFI_PCI_IO_PROTOCOL_GUID`
uninstall?+76	`lea rdx, EFI_PCI_IO_PROTOCOL_GUID`
uninstall?+220	`lea rdx, EFI_PCI_IO_PROTOCOL_GUID`
DriverBindingProtoStop+DD	`lea rdx, EFI_PCI_IO_PROTOCOL_GUID`
DriverBindingProtoStop+120	`lea rdx, EFI_PCI_IO_PROTOCOL_GUID`
sub_2EC0+158	`lea r8, EFI_PCI_IO_PROTOCOL_GUID`
GetControllerName+3A	`lea rdx, EFI_PCI_IO_PROTOCOL_GUID`
GetControllerName+59	`lea rdx, EFI_PCI_IO_PROTOCOL_GUID`
GetControllerName:loc_55C6	`lea r8, EFI_PCI_IO_PROTOCOL_GUID`

Some places are spurios, like:

.text:00000000000024F8                 lea     r8, EFI_PCI_IO_PROTOCOL_GUID
.text:00000000000024FF                 mov     rcx, rsi
.text:0000000000002502                 call    sub_5F04

Since sub_5F04 overrides r8 immediatly:

.text:0000000000005F04 sub_5F04        proc near               ; CODE XREF: sub_245C+A6↑p
.text:0000000000005F04                                         ; sub_259C+41↑p ...
.text:0000000000005F04
.text:0000000000005F04 count           = qword ptr -18h
.text:0000000000005F04 arg_0           = qword ptr  8
.text:0000000000005F04 proto_info      = qword ptr  20h
.text:0000000000005F04
.text:0000000000005F04                 mov     [rsp+arg_0], rbx
.text:0000000000005F09                 push    rdi
.text:0000000000005F0A                 sub     rsp, 30h
.text:0000000000005F0E                 mov     rax, cs:gBOOT_SERVICES
.text:0000000000005F15                 mov     rdi, rdx
.text:0000000000005F18                 lea     r9, [rsp+38h+count]
.text:0000000000005F1D                 lea     r8, [rsp+38h+proto_info]      ;; HERE!!

Long story short: no code in the GOP DXE driver communicates with the GuC.

Before moving on to CSME vs GuC, I was curious who exactly uses all these protocols, in the rest of the UEFI BIOS and Windows. I extracted the UEFI capsule and also mounted the Windows ISO and WIM files (dism /mount-image /imagefile:e:\sources\install.wim /index:1 /mountdir:c:\mnt\install /readonly), and then ran the following python script:

from struct import unpack
from os import walk
from mmap import mmap, ACCESS_READ
import os.path as path

GUIDS = (
((0xDE, 0xA9, 0x42, 0x90, 0xDC, 0x23, 0x38, 0x4A, 0x96, 0xFB, 0x7A, 0xDE, 0xD0, 0x80, 0x51, 0x6A), 'EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID'),
((0x56, 0x10, 0x8C, 0xBD, 0x36, 0x9F, 0xEC, 0x44, 0x92, 0xA8, 0xA6, 0x33, 0x7F, 0x81, 0x79, 0x86), 'EFI_EDID_ACTIVE_PROTOCOL_GUID'),
((0xF6, 0x34, 0x0C, 0x1C, 0x80, 0xD3, 0xFA, 0x41, 0xA0, 0x49, 0x8A, 0xD0, 0x6C, 0x1A, 0x66, 0xAA), 'EFI_EDID_DISCOVERED_PROTOCOL_GUID'),
((0x1D, 0x3F, 0xF2, 0x6F, 0x7C, 0x87, 0x1B, 0x4B, 0x93, 0xFC, 0xF1, 0x42, 0xB2, 0xEE, 0xA6, 0xA7), 'GOP_DISPLAY_BRIGHTNESS_PROTOCOL_GUID'),
((0x3A, 0xD3, 0x1D, 0xF5, 0x7F, 0xE5, 0x20, 0x40, 0xB4, 0x66, 0xF4, 0xC1, 0x71, 0xC6, 0xE4, 0xF7), 'GOP_DISPLAY_BIST_PROTOCOL_GUID'),
#((0x00, 0xB2, 0xF5, 0x4C, 0xB8, 0x68, 0xA5, 0x4C, 0x9E, 0xEC, 0xB2, 0x3E, 0x3F, 0x50, 0x02, 0x9A), 'EFI_PCI_IO_PROTOCOL_GUID'),
#((0xBD, 0x7E, 0x1B, 0x65, 0x13, 0xCE, 0xD0, 0x41, 0x82, 0xE5, 0xA0, 0x63, 0xAB, 0xBE, 0x9B, 0xB6), 'GOP_COMPONENT_NAME2_PROTOCOL_GUID'),
((0xCD, 0x2F, 0xCB, 0xDB, 0x9A, 0xE2, 0x0E, 0x41, 0x9D, 0xD9, 0xFA, 0x9D, 0x5F, 0xF4, 0xCD, 0xA7), 'UNKNOWN_PROTOCOL_GUID'),
((0x3B, 0x70, 0xD4, 0xC7, 0x36, 0x0F, 0x51, 0x4E, 0xA9, 0x83, 0x5E, 0x61, 0xAC, 0xB8, 0x68, 0x3C), 'MAYBE_AUX_PROTOCOL_GUID?'),
#((0x91, 0x6E, 0x57, 0x09, 0x3F, 0x6D, 0xD2, 0x11, 0x8E, 0x39, 0x00, 0xA0, 0xC9, 0x69, 0x72, 0x3B), 'EFI_DEVICE_PATH_PROTOCOL_GUID'),
#((0xA1, 0x31, 0x1B, 0x5B, 0x62, 0x95, 0xD2, 0x11, 0x8E, 0x3F, 0x00, 0xA0, 0xC9, 0x69, 0x72, 0x3B), 'EFI_LOADED_IMAGE_PROTOCOL_GUID'),
#((0xAB, 0x31, 0xA0, 0x18, 0x43, 0xB4, 0x1A, 0x4D, 0xA5, 0xC0, 0x0C, 0x09, 0x26, 0x1E, 0x9F, 0x71), 'EFI_DRIVER_BINDING_PROTOCOL_GUID'),
#((0xFF, 0x5C, 0x7A, 0x6A, 0xD9, 0xE8, 0x70, 0x4F, 0xBA, 0xDA, 0x75, 0xAB, 0x30, 0x25, 0xCE, 0x14), 'EFI_COMPONENT_NAME2_PROTOCOL_GUID')
)

guids = { bytes(k) : v for k, v in GUIDS }
first_dwords = set([unpack("<I", guid[0:4]) for guid in guids.keys()])

for root in ('c:\\mnt\\iso', 'c:\\mnt\\boot', 'c:\\mnt\\install', 'c:\\mnt\\uefi'):
    for dir, _, files in walk(root):
        for file in files:
            filename = dir + '\\' + file
            try:
                filelen = path.getsize(filename) & ~15
                if filelen == 0:
                    continue
                with open(filename, 'rb') as file:
                    with mmap(file.fileno(), filelen, access=ACCESS_READ) as mem:
                        for ofs in range(0, filelen, 16):
                            if unpack("<I", mem[ofs:ofs+4]) in first_dwords:
                                guid = mem[ofs:ofs+16]
                                try:
                                    name = guids[guid]
                                    print(f'{filename}\t{ofs:x}\t{name}')
                                except KeyError:
                                    pass
            except PermissionError:
                pass

The UEFI setup and legacy components use the GOP and the EDID components:

c:\mnt\uefi\\AMITSE.efi	400	EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID
c:\mnt\uefi\\Bds.efi	3d0	EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID
c:\mnt\uefi\\ConSplitter.efi	310	EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID
c:\mnt\uefi\\CsmVideo.efi	2c0	EFI_EDID_DISCOVERED_PROTOCOL_GUID
c:\mnt\uefi\\CsmVideo.efi	2d0	EFI_EDID_ACTIVE_PROTOCOL_GUID
c:\mnt\uefi\\CsmVideo.efi	320	EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID
c:\mnt\uefi\\GraphicsConsole.efi	2b0	EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID
c:\mnt\uefi\\Setup.efi	2e0	EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID
c:\mnt\uefi\\UefiPxeBcDxe.efi	490	EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID

In Windows we have only:

c:\mnt\boot\Windows\Boot\EFI\bootmgfw.efi       a1a0    EFI_EDID_ACTIVE_PROTOCOL_GUID
c:\mnt\boot\Windows\Boot\EFI\bootmgfw.efi       a220    EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID
c:\mnt\boot\Windows\System32\winload.efi        17e210  EFI_EDID_ACTIVE_PROTOCOL_GUID
c:\mnt\boot\Windows\System32\winload.efi        17e2a0  EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID
c:\mnt\boot\Windows\System32\winresume.efi      122c00  EFI_EDID_ACTIVE_PROTOCOL_GUID
c:\mnt\boot\Windows\System32\winresume.efi      122c80  EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID
c:\mnt\boot\Windows\System32\Boot\winload.efi   17e210  EFI_EDID_ACTIVE_PROTOCOL_GUID
c:\mnt\boot\Windows\System32\Boot\winload.efi   17e2a0  EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID
c:\mnt\boot\Windows\System32\Boot\winresume.efi 122bf0  EFI_EDID_ACTIVE_PROTOCOL_GUID
c:\mnt\boot\Windows\System32\Boot\winresume.efi 122c70  EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID
c:\mnt\install\Windows\Boot\EFI\bootmgfw.efi    a1a0    EFI_EDID_ACTIVE_PROTOCOL_GUID
c:\mnt\install\Windows\Boot\EFI\bootmgfw.efi    a220    EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID
c:\mnt\install\Windows\System32\SecConfig.efi   110b80  EFI_EDID_ACTIVE_PROTOCOL_GUID
c:\mnt\install\Windows\System32\SecConfig.efi   110c00  EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID
c:\mnt\install\Windows\System32\winload.efi     17e210  EFI_EDID_ACTIVE_PROTOCOL_GUID
c:\mnt\install\Windows\System32\winload.efi     17e2a0  EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID
c:\mnt\install\Windows\System32\winresume.efi   122c00  EFI_EDID_ACTIVE_PROTOCOL_GUID
c:\mnt\install\Windows\System32\winresume.efi   122c80  EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID
c:\mnt\install\Windows\System32\Boot\winload.efi        17e210  EFI_EDID_ACTIVE_PROTOCOL_GUID
c:\mnt\install\Windows\System32\Boot\winload.efi        17e2a0  EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID
c:\mnt\install\Windows\System32\Boot\winresume.efi      122bf0  EFI_EDID_ACTIVE_PROTOCOL_GUID
c:\mnt\install\Windows\System32\Boot\winresume.efi      122c70  EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID
c:\mnt\iso\bootx64.efi a1a0    EFI_EDID_ACTIVE_PROTOCOL_GUID
c:\mnt\iso\bootx64.efi a220    EFI_GRAPHICS_OUTPUT_PROTOCOL_GUID

So basically most of the GOP DXE driver functions go unused and can be considered bloat …

Are EFI_GRAPHICS_OUTPUT_PROTOCOL and EFI_EDID_ACTIVE_PROTOCOL_GUID possible vectors for exploitation from UEFI -> Windows? Assume for example a DXE driver has a bug that can be exploited using specialized hardware, and you gain execution in the UEFI firmware during boot. Can these protocols be used as an attack surface to attack SecureBoot Windows?

As seen before, EFI_GRAPHICS_OUTPUT_PROTOCOL has a driver controlled Mode member

struct _EFI_GRAPHICS_OUTPUT_PROTOCOL {
    EFI_GRAPHICS_OUTPUT_PROTOCOL_QUERY_MODE QueryMode;
    EFI_GRAPHICS_OUTPUT_PROTOCOL_SET_MODE SetMode;
    EFI_GRAPHICS_OUTPUT_PROTOCOL_BLT Blt;
    EFI_GRAPHICS_OUTPUT_PROTOCOL_MODE *Mode;
};

In turn EFI_GRAPHICS_OUTPUT_PROTOCOL_MODE is defined as:

typedef struct {
    UINT32 MaxMode;
    UINT32 Mode;
    EFI_GRAPHICS_OUTPUT_MODE_INFORMATION *Info;
    UINTN SizeOfInfo;
    EFI_PHYSICAL_ADDRESS FrameBufferBase;
    UINTN FrameBufferSize;
} EFI_GRAPHICS_OUTPUT_PROTOCOL_MODE;

These structure are used in several functions inside the console library shared by all the relevant Windows components. The two main functions are ConsoleEfiGopOpen and ConsoleEfiGopEnable:

__int64 __fastcall ConsoleEfiGopOpen(CONSOLE_DATA *this)
{
  ...
  if ( EfiOpenProtocol(this->efi_handle, (__int64)&EfiGraphicsOutputProtocol, &gop_protocol) >= 0 )
  {
    status = EfiGopGetCurrentMode(gop_protocol, &mode, &mode_info);
    if ( status >= 0 )
    {
      orig_mode = mode;
      new_mode = mode;
      
      ... check if mode is allowed, if not get allowed mode ...
      
      // fill state with mode data
      is_rgb = mode_info.PixelFormat == PixelBlueGreenRedReserved8BitPerColor;
      this_1->gop_protocol = gop_protocol;
      this_1->new_mode = new_mode;
      this_1->orig_mode = orig_mode;
      if ( is_rgb )
        bits_per_pixel = 32;
      else if ( mode_info.PixelFormat == PixelBitMask )
        bits_per_pixel = 24;      
      else {
        status = STATUS_UNSUCCESSFUL;
        goto exit_handler;
      }
      this_1->orig_horiz_res = mode_info.HorizontalResolution;
      this_1->orig_vert_res = mode_info.VerticalResolution;
      pixels_per_scan_line = mode_info.PixelsPerScanLine;
      this_1->orig_bits_per_pixel = bits_per_pixel;
      result = 0i64;
      this_1->orig_pixels_per_scan_line = pixels_per_scan_line;
      return result;
      
    }
exit_handler:
    EfiCloseProtocol(this_1->efi_handle, &EfiGraphicsOutputProtocol);
    return (unsigned int)status;
  }
  return 0xC00000BB;
}

EfiGopGetCurrentMode() in turn uses MmArchTranslateVirtualAddress to get physical addresses for the output:

int __fastcall EfiGopGetCurrentMode(EFI_GRAPHICS_OUTPUT_PROTOCOL *gop, unsigned int *mode, EFI_GRAPHICS_OUTPUT_MODE_INFORMATION *info)
{
  ...
  info_phys_addr = info;
  mode_phys_addr = mode;
  gop_phys_addr = gop;
  context_mode = *gCurrentExecutionContext;
  if ( *gCurrentExecutionContext != ExecutionContextFirmware )
  {
    if ( gop )
      status = MmArchTranslateVirtualAddress(gop, (unsigned __int64 *)&phys_addr, 0i64, 0i64);
    else
      status = 0;
    if ( !status )
      return STATUS_UNSUCCESSFUL;
    gop_phys_addr = phys_addr;
    is_mapped = mode_phys_addr ? MmArchTranslateVirtualAddress(
                                   mode_phys_addr,
                                   (unsigned __int64 *)&phys_addr,
                                   0i64,
                                   0i64) : 0;
    if ( !is_mapped )
      return STATUS_UNSUCCESSFUL;
    mode_phys_addr = (unsigned int *)phys_addr;
    is_mapped_2 = info_phys_addr ? MmArchTranslateVirtualAddress(
                                     info_phys_addr,
                                     (unsigned __int64 *)&phys_addr,
                                     0i64,
                                     0i64) : 0;
    if ( !is_mapped_2 )
      return STATUS_UNSUCCESSFUL;
    info_phys_addr = (EFI_GRAPHICS_OUTPUT_MODE_INFORMATION *)phys_addr;
    BlpArchSwitchContext(ExecutionContextFirmware);
  }
  *mode_phys_addr = gop_phys_addr->Mode->Mode;
  mode_info = gop_phys_addr->Mode->Info;
  *(_OWORD *)&info_phys_addr->Version = *(_OWORD *)&mode_info->Version;
  info_phys_addr->PixelInformation = mode_info->PixelInformation;
  info_phys_addr->PixelsPerScanLine = mode_info->PixelsPerScanLine;
  if ( context_mode != ExecutionContextFirmware )
    BlpArchSwitchContext(context_mode);
  return v3;
}

The most we can get from this is an arbitary read from physical memory by Windows.

Lets look at ConsoleEfiGopEnable:

unsigned int __fastcall ConsoleEfiGopEnable(CONSOLE_DATA *this)
{
  ...
  status = EfiGopGetCurrentMode(this->gop_protocol, &old_mode, &mode_info);
  if ( status < 0 )
    return status;
  new_mode_1 = old_mode;
  if ( old_mode != new_mode )
  {  
    status = EfiGopSetMode(this_1->gop_protocol, new_mode);
    if ( status >= 0 )
    {
      BlDisplayInvalidateOemBitmap();
      EfiGopGetCurrentMode(this_1->gop_protocol, &mode, &mode_info);
      new_mode_1 = old_mode;
    }
  }
  
    if ( mode_info.PixelFormat == PixelBlueGreenRedReserved8BitPerColor )
        bits_per_pixel = 32;
    else if ( mode_info.PixelFormat == PixelBitMask )
        bits_per_pixel = 24;
    else { ...; return STATUS_UNSUCCESSFUL; }
    
    EfiGopGetFrameBuffer(this_1->gop_protocol, &frame_buffer_base, &frame_buffer_size);
    if ( BlMmMapPhysicalAddressEx(&frame_buffer, frame_buffer_base, frame_buffer_size, 8u, 0) >= 0
      || (status = BlMmMapPhysicalAddressEx(&frame_buffer, frame_buffer_base, frame_buffer_size, 1u, 0), status >= 0) )
    {
      this_1->frame_buffer = (void *)frame_buffer_1;
      this_1->frame_buffer_size = frame_buffer_size;
      this_1->bits_per_pixel = bits_per_pixel;
      this_1->horiz_res = mode_info.HorizontalResolution;
      ... contonue filling this_1 with mode_info ...
      return result;
    }
  }
  return STATUS_UNSUCCESSFUL;

Here Windows map the physical address supplied by GOP->FrameBuffer (retrieved in EfiGopGetFrameBuffer) into Windows. We can control FrameBuffer so we might be able to arbitarily map any physical memory as the frame buffer.

How does that help us? If for example the OEM logo (specified in the 'BGRT' ACPI table) is copied to the FrameBuffer, we can write data under our control to a physical address under our control - after the bootmgr has already been verified as part of the Secure Boot process.

But this is tangental to this post so we’ll examine this vector in a future post.

Part 2: From CSME

Now lets turn to the question wether CSME accesses the GuC and vice-versa.

The CSME is really big, so an exhastive disassembly like we did for the GOP is less relevant. So where might the CSME engine need to communicate with the GuC?

One place that comes into mind is the PAVP - Protected Audio Video Path. This is the component that protects protected HD content from being copied. The protection is implemented by creating a secure pipeline from the media components in the Windows kernel, through the GFX driver, and all the way to the display. The CSME is used to protect the pipeline including certs, keys and much more.

We can start with the CSME HECI (Host Embedded Controller Interface) driver on Windows and find the relevant HECI messages. One group of interesting messages I found was for the LSPCON component. LSPCON stands for Level Shifter and Protocol Converter, which is used for HDR signalling over HDMI.

No hard work means no fish, so we go on a fishing expedition and finally manage to extract the PAVP component from an old CSME15 build. Its about 300KB in size, so still quite big.

Reversing this I went down a deep rabbit hole. I finally discovered a function I named PAVP_init_heci, that is called from main and initializes the HECI communication module in PAVP and registers an interface with three functions:

handle async messages - PAVP_handle_async_message
HECI connection request - PAVP_connect
HECI disconnect request - PAVP_disconnect (all the names are mine)

PAVP_heci_handle_async_message() handles different types of messages like widevine, asmf, PlayReady and so on. We are interested in CPHS - Intel Content Protection HECI Service, a function I named PAVP_process_cphs_message(). Digging deeper we eventually reach the LSPCON command handler:

.text:0010775B ; int __cdecl LSPCON_command_handler(PavpCtx *ctx, void *heci_msg, int heci_msg_len, int max_out_len, int *out_len)
.text:0010775B LSPCON_command_handler proc near        ; CODE XREF: PAVP_heci_command_handler+8D↑p
.text:0010775B
.text:0010775B var_14          = dword ptr -14h
.text:0010775B msg_len         = dword ptr -10h
.text:0010775B ctx             = dword ptr  8
.text:0010775B heci_msg        = dword ptr  0Ch
.text:0010775B heci_msg_len    = dword ptr  10h
.text:0010775B max_out_len     = dword ptr  14h
.text:0010775B out_len         = dword ptr  18h
.text:0010775B
.text:0010775B cmd = ebx
.text:0010775B                 push    ebp
.text:0010775C                 mov     ebp, esp
.text:0010775E                 push    edi
.text:0010775F                 push    esi
.text:00107760                 push    cmd
.text:00107761                 sub     esp, 8
.text:00107764                 mov     eax, [ebp+heci_msg_len]
.text:00107767                 mov     ecx, [ebp+ctx]
.text:0010776A                 mov     [ebp+msg_len], eax
.text:0010776D                 mov     eax, [ebp+max_out_len]
.text:00107770                 mov     cmd, [ebp+heci_msg]
.text:00107773                 mov     [ebp+var_14], eax
.text:00107776                 mov     esi, [ebp+out_len]
.text:00107779                 test    ecx, ecx
.text:0010777B                 jz      short err_cmd_not_in_range
.text:0010777D                 cmp     [ecx+PavpCtx.Lspcon], 0
.text:00107781                 jz      short err_cmd_not_in_range
.text:00107783                 test    cmd, cmd
.text:00107785                 setz    dl
.text:00107788                 test    esi, esi
.text:0010778A                 setz    al
.text:0010778D                 or      dl, al
.text:0010778F                 jnz     short err_cmd_not_in_range
.text:00107791                 cmp     [ebp+msg_len], 0Fh ; cmd_len <= sizeof(LSPCON_heci_command_header_t)
.text:00107795                 ja      short is_cmd_id_in_heci_range

It begins by verifying the command buffer is big enough to fit the LSPCON HECI command header:

00000000 LSPCON_heci_command_header_t struc ; (sizeof=0x10, mappedto_125)
00000000                                         ; XREF: LSPCON_HECICMD_PLAYBACK_DONE_IN/r
00000000                                         ; LSPCON_HECICMD_PLAYBACK_DONE_OUT/r ...
00000000 version         dd ?
00000004 cmdid           dd ?                    ; XREF: LSPCON_command_handler:is_cmd_id_in_heci_range/r
00000008 status          dd ?
0000000C size            dd ?                    ; XREF: LSPCON_command_handler+6B/w
0000000C                                         ; LSPCON_command_handler+91/w ...
00000010 LSPCON_heci_command_header_t ends

Next it checks the command is one of the 7 LSPCON HECI commands and retreives appropriate handler from a global handler list:

.text:001077B8 is_cmd_id_in_heci_range:                ; CODE XREF: LSPCON_command_handler+3A↑j
.text:001077B8                 mov     edi, [cmd+LSPCON_heci_command_header_t.cmdid]
.text:001077BB                 lea     eax, [edi-0E000h] ; is 0xE000 < id < 0xE008
.text:001077C1                 cmp     eax, 7
.text:001077C4                 jbe     short get_handler
                               ...
.text:001077D4 get_handler:                            ; CODE XREF: LSPCON_command_handler+69↑j
.text:001077D4                 mov     edx, dword ptr ds:gLSPCONCmdHandlerTable[eax*8] ; gLSPCONCmdHandlerTable.HandleFunc
.text:001077DB                 test    edx, edx        ; EDX contains handler
.text:001077DD                 jnz     short check_cmd_data

The global list looks something like:

gLSPCONCmdHandlerTable[] = {
      { 0 },
      { LSPCON_get_status,             sizeof(LSPCON_HECICMD_GET_LSPCON_STATUS_IN),    sizeof(LSPCON_HECICMD_GET_LSPCON_STATUS_OUT)},
      { LSPCON_set_dev_cert,           sizeof(LSPCON_HECICMD_SET_LSPCON_CERT_IN),      sizeof(LSPCON_HECICMD_SET_LSPCON_CERT_OUT)},
      { LSPCON_init_session,           sizeof(LSPCON_HECICMD_INIT_SESSION_IN),         sizeof(LSPCON_HECICMD_INIT_SESSION_OUT)},
      { LSPCON_init_limits,            sizeof(LSPCON_HECICMD_INIT_LIMITS_IN),          sizeof(LSPCON_HECICMD_INIT_LIMITS_OUT)},
      { LSPCON_playback_done,          sizeof(LSPCON_HECICMD_PLAYBACK_DONE_IN),        sizeof(LSPCON_HECICMD_PLAYBACK_DONE_OUT)},
      { LSPCON_ack,                    sizeof(LSPCON_HECICMD_MSG_ACK_IN),              sizeof(LSPCON_HECICMD_MSG_ACK_OUT)},
      { LSPCON_get_topology,           sizeof(LSPCON_HECICMD_GET_TOPOLOGY_IN),         sizeof(LSPCON_HECICMD_GET_TOPOLOGY_OUT)},
  };

After verifying the size of the input and output structs the actual command handle is called.

.text:001077FD
.text:001077FD check_cmd_data:                         ; CODE XREF: LSPCON_command_handler+82↑j
.text:001077FD                 movzx   edi, word ptr ds:unk_82364[eax*8] ; gLSPCONCmdHandlerTable.InputSize
.text:00107805                 cmp     edi, [ebp+msg_len]
.text:00107808                 ja      short sizes_error
.text:0010780A                 movzx   eax, word ptr ds:unk_82366[eax*8] ; gLSPCONCmdHandlerTable.OutputSize
.text:00107812                 cmp     eax, [ebp+var_14]
.text:00107815                 ja      short sizes_error
                               ...
.text:00107830
.text:00107830 loc_107830:                             ; CODE XREF: LSPCON_command_handler+C5↑j
.text:00107830                 push    cmd
.text:00107831                 push    ecx
                               ...
.text:00107846                 call    edx             ; Call Command Handler!

Reveresing all the command handlers we find something interesting in the most unexpected one (thus the last I REd): LSPCON_playback_done(). It took me a while to even understand its releated to the GuC, and I’ll explain later how it does so.

What does LSPCON_playback_done do? It checks whether HDCP restrictions should remain in place after a playback is complete.

The function begins by verifying the input parameter (LSPCON_HECICMD_PLAYBACK_DONE_IN) is valid:

.text:00107C6B ; int __cdecl LSPCON_playback_done(PavpCtx *ctx, void *msg)
.text:00107C6B LSPCON_playback_done proc near
.text:00107C6B
.text:00107C6B cur_hdcp_requirements= dword ptr -18h
.text:00107C6B count_active_sessions= dword ptr -14h
.text:00107C6B var_10          = dword ptr -10h
.text:00107C6B ctx             = dword ptr  8
.text:00107C6B msg             = dword ptr  0Ch
.text:00107C6B
.text:00107C6B ctx_ptr = edi
.text:00107C6B                 push    ebp
.text:00107C6C                 mov     ebp, esp
.text:00107C6E                 push    ctx_ptr
.text:00107C6F                 push    esi
.text:00107C70                 push    ebx
.text:00107C71                 sub     esp, 0Ch
.text:00107C74                 mov     [ebp+count_active_sessions], 0
.text:00107C7B                 mov     esi, [ebp+msg]
.text:00107C7E                 mov     eax, ds:stack_cookie_ptr
.text:00107C83                 mov     [ebp+var_10], eax
.text:00107C86                 xor     eax, eax
.text:00107C88                 mov     ctx_ptr, [ebp+ctx]
.text:00107C8B                 test    esi, esi
.text:00107C8D                 jnz     short check_valid_header
                               ...
.text:00107C99 check_valid_header:                     ; CODE XREF: LSPCON_playback_done+22↑j
.text:00107C99                 mov     [esi+LSPCON_HECICMD_PLAYBACK_DONE_IN.header.size], 0
.text:00107CA0                 test    ctx_ptr, ctx_ptr
.text:00107CA2                 jz      short invalid_parameter
.text:00107CA4                 cmp     [ctx_ptr+PavpCtx.Lspcon], 0
.text:00107CA8                 jz      short invalid_parameter

And now comes the interesting part:

.text:00107CAA                 lea     eax, [ebp+count_active_sessions]
.text:00107CAD                 push    eax             ; num_active_sessions
.text:00107CAE                 push    0               ; type
.text:00107CB0                 push    ctx_ptr         ; ctx
.text:00107CB1                 call    GUC_get_active_sessions ; 
.text:00107CB6                 add     esp, 0Ch
.text:00107CB9                 mov     ebx, eax
.text:00107CBB                 test    eax, eax
.text:00107CBD                 jz      short got_active_sessions

If there are any remaining active sessions the code continues to check what level of HDCP protection they require and set protection to that level if it is lower then the current level, I won’t go into that disassembly as its not really interesting.

Why do I think GUC_get_active_sessions is actually related to GuC and why did I name it that? Lets continue by examining this function. Its just a wrapper around a function I called GUC_send_message that sends message no. 6,

.text:0010452C ; int __cdecl GUC_get_active_sessions(PavpCtx *ctx, int type, unsigned int *num_active_sessions)
.text:0010452C GUC_get_active_sessions proc near       ; CODE XREF: LSPCON_playback_done+46↓p
.text:0010452C
.text:0010452C guc2csme        = GUC2CSME_MSG ptr -18h
.text:0010452C csme2guc        = CSME2GUC_MSG ptr -10h
.text:0010452C ctx             = dword ptr  8
.text:0010452C type            = dword ptr  0Ch
.text:0010452C num_active_sessions= dword ptr  10h
.text:0010452C
.text:0010452C ctx_ptr = esi
                               ...
.text:0010455B type_ok:
.text:0010455B                 mov     dword ptr [ebp+csme2guc.command], GUC_MSG_GET_ACTIVE_SESSIONS ; =6
.text:00104562                 mov     [ebp+csme2guc.data1], al
.text:00104565                 lea     eax, [ebp+guc2csme.value]
.text:00104568                 mov     [ebp+guc2csme.value], 0
.text:0010456F                 push    eax             ; guc2csme
.text:00104570                 lea     eax, [ebp+csme2guc]
.text:00104573                 push    eax             ; csme2guc
.text:00104574                 push    ctx_ptr         ; ctx
.text:00104575                 call    GUC_send_message

GUC_send_message() gets two parameters in addition to the PAVP context: a CSME2GUC structure and a GUC2CSME structure. How does it work? It tries to send the message several times in a loop, each time waiting for a short timeout. The first iteration of the loop also wakes the GuC by enabling it through managment functions (if it isn’t already enabled), and sending a special wake message using a function I named GUC_send_VDM().

.text:001041FF ; int __cdecl GUC_send_message(PavpCtx *ctx, CSME2GUC_MSG *csme2guc, GUC2CSME_MSG *guc2csme)
.text:001041FF GUC_send_message proc near              ; CODE XREF: GUC_get_active_sessions+49↓p
.text:001041FF                                         ; sub_1045C5+3F↓p
.text:001041FF
.text:001041FF ctx             = dword ptr  8
.text:001041FF csme2guc        = dword ptr  0Ch
.text:001041FF guc2csme        = dword ptr  10h
.text:001041FF
.text:001041FF attempt = esi
.text:001041FF ctx_ptr = ebx
.text:001041FF                 push    ebp
.text:00104200                 mov     ebp, esp
.text:00104202                 push    edi
.text:00104203                 push    attempt
.text:00104204                 xor     attempt, attempt
.text:00104206                 push    ctx_ptr
.text:00104207                 mov     ctx_ptr, [ebp+ctx]
.text:0010420A
.text:0010420A send_loop:                              ; CODE XREF: GUC_send_message+A3↓j
.text:0010420A                 inc     attempt
.text:0010420B                 cmp     attempt, 1
.text:0010420E                 jnz     short send_wake_msg_loop
.text:00104210
.text:00104210 first_attempt:
.text:00104210                 push    ctx_ptr
.text:00104211                 call    GUC_disable_power_gate?
.text:00104216                 mov     edi, eax
.text:00104218                 pop     eax
.text:00104219                 test    edi, edi
.text:0010421B                 jnz     loc_1042A8
.text:00104221
.text:00104221 send_wake_msg_loop:                     ; CODE XREF: GUC_send_message+F↑j
.text:00104221                                         ; GUC_send_message+4E↓j
.text:00104221                 push    VDM_CSME_TO_GUC_WAKE_REQ
.text:00104223                 push    0               ; msg
.text:00104225                 push    ctx_ptr
.text:00104226                 call    GUC_send_VDM    ; VDM == Vendor Defined Message?
.text:0010422B                 add     esp, 0Ch
.text:0010422E                 mov     edi, eax
.text:00104230                 test    eax, eax
.text:00104232                 jnz     msg_error
.text:00104238                 push    [ebp+guc2csme]
.text:0010423B                 push    GUC_IS_AWAKE
.text:0010423D                 push    ctx_ptr
.text:0010423E                 call    GUC_wait_for_message ; wait for GUC is awake message
.text:00104243                 add     esp, 0Ch
.text:00104246                 mov     edi, eax
.text:00104248                 cmp     eax, PAVP_STATUS_TRY_AGAIN
.text:0010424D                 jz      short send_wake_msg_loop
.text:0010424F                 cmp     eax, PAVP_STATUS_TIMEOUT
.text:00104254                 jnz     short got_awake_msg
.text:00104256
.text:00104256 timeout:                                ; CODE XREF: GUC_send_message+92↓j
.text:00104256                                         ; GUC_send_message+C9↓j
.text:00104256                 mov     edi, PAVP_STATUS_TIMEOUT
.text:0010425B                 jmp     short loc_104297
.text:0010425D ; ---------------------------------------------------------------------------
.text:0010425D

Once the GuC awake message was received the actually GuC message is send, again with GUC_send_VDM().

.text:0010425D got_awake_msg:                          ; CODE XREF: GUC_send_message+55↑j
.text:0010425D                 test    eax, eax
.text:0010425F                 jnz     short loc_1042A8
.text:00104261                 mov     eax, [ebp+csme2guc]
.text:00104264                 push    VDM_FROM_CSME
.text:00104266                 push    dword ptr [eax+CSME2GUC_MSG.command]
.text:00104268                 push    ctx_ptr
.text:00104269                 call    GUC_send_VDM
.text:0010426E                 add     esp, 0Ch
.text:00104271                 mov     edi, eax
.text:00104273                 test    eax, eax
.text:00104275                 jnz     short loc_1042A8
.text:00104277                 push    [ebp+guc2csme]
.text:0010427A                 mov     eax, [ebp+csme2guc]
.text:0010427D                 movzx   eax, [eax+CSME2GUC_MSG.command]
.text:00104280                 push    eax
.text:00104281                 push    ctx_ptr
.text:00104282                 call    GUC_wait_for_message

Its then waits for the return message GUC_wait_for_message(). Now you have to say - Wise guy, how do you know this is actually releated to GuC? What is this VDM stuff? Did Ded Moroz drop them in your cabin?

VDMs are Vendor Defined Messages, a way to send custom messages to devices over a PCI bus. They are sent through IOCTLs to the VDM driver in CSME. The IOCTL gets data through a message:

00000000 IOCTL_VDM_WRITE struc ; (sizeof=0x12, mappedto_145)
00000000 addr_offset     dd ?
00000004 data            dd ?          ; This is a bitfield per the spec
00000008 info            VDM_TX ?
00000012 IOCTL_VDM_WRITE ends

00000000 VDM_TX          struc ; (sizeof=0xA, mappedto_142)
00000000                                         ; XREF: GucCtx/r
00000000                                         ; IOCTL_VDM_WRITE/r
00000000 msg             dd ?                    ; XREF: setup_guc_vdm+F/r
00000004 pci_req_id      dw ?                    ; XREF: setup_guc_vdm+12/w
00000006 tag             dw ?
00000008 pci_tgt_id      dw ?                    ; XREF: setup_guc_vdm+1C/w
0000000A VDM_TX          ends

Here you have the first hint of how I connected all this to the GuC. Lets just get the VDM function out of the way:

.text:0014889D VDM_write       proc near               ; CODE XREF: sub_1028DB+CE↑p
.text:0014889D                                         ; GUC_send_VDM+4F↑p ...
.text:0014889D
.text:0014889D var_40          = byte ptr -40h
.text:0014889D vdm_ioctl       = IOCTL_VDM_WRITE ptr -3Ch
.text:0014889D var_10          = dword ptr -10h
.text:0014889D fd              = dword ptr  8
.text:0014889D addr_info       = dword ptr  0Ch
.text:0014889D addr_offset     = dword ptr  10h
.text:0014889D data            = dword ptr  14h
.text:0014889D
.text:0014889D                 push    ebp
.text:0014889E                 mov     ebp, esp
.text:001488A0                 push    edi
.text:001488A1                 push    esi
.text:001488A2                 push    ebx
.text:001488A3                 sub     esp, 34h
.text:001488A6                 mov     ebx, [ebp+fd]
.text:001488A9                 mov     eax, ds:stack_cookie_ptr
.text:001488AE                 mov     [ebp+var_10], eax
.text:001488B1                 xor     eax, eax
.text:001488B3                 mov     edi, [ebp+addr_info]
.text:001488B6                 test    ebx, ebx
.text:001488B8                 js      short invalid_parameter
.text:001488BA                 test    edi, edi
.text:001488BC                 jz      short invalid_parameter
.text:001488BE                 lea     esi, [ebp+vdm_ioctl]
.text:001488C1
.text:001488C1 build_ioctl_data:
.text:001488C1                 push    44 ; sizeof(vdm_ioctl)
.text:001488C3                 push    0
.text:001488C5                 push    esi
.text:001488C6                 call    near ptr memset
.text:001488CB                 mov     eax, [ebp+addr_offset]
.text:001488CE                 mov     [ebp+vdm_ioctl.addr_offset], eax
.text:001488D1                 mov     eax, [ebp+data]
.text:001488D4                 mov     [ebp+vdm_ioctl.data], eax
.text:001488D7                 lea     eax, [ebp+vdm_ioctl.info]
.text:001488DA                 push    0Ah             ; sizeof(TX info)
.text:001488DC                 push    edi
.text:001488DD                 push    0Ah
.text:001488DF                 push    eax
.text:001488E0                 call    near ptr memcpy_s
.text:001488E5                 lea     eax, [ebp+var_40]
.text:001488E8                 push    eax
.text:001488E9                 push    44
.text:001488EB                 push    esi
.text:001488EC                 push    44
.text:001488EE                 push    esi
.text:001488EF                 push    2        ; IOCTL write
.text:001488F1                 push    ebx
.text:001488F2                 call    near ptr ioctl_s

The IOCTL is sent to a file handle. Where is it set? We now go back to the PAVP init code and look for all places where file handles are init. There we find to functions I am pretty sure initialize the GuC and the Graphics Key Manager (GKM), thus I appropriatly named them GUC_init() and GKM_init() (I keep reminding you I named these functions as I have no clue what is their realy name, these are my guesses).

As usual, the function begins by checking it's input argument:

.text:001043C3 GUC_init        proc near               ; CODE XREF: pavp_init+259↑p
.text:001043C3
.text:001043C3 ctx             = dword ptr  8
.text:001043C3
.text:001043C3 ctx_ptr = ebx
.text:001043C3                 push    ebp
.text:001043C4                 mov     ebp, esp
.text:001043C6                 push    esi
.text:001043C7                 push    ctx_ptr
.text:001043C8                 mov     esi, 1005h
.text:001043CD                 mov     ctx_ptr, [ebp+ctx]
.text:001043D0                 test    ctx_ptr, ctx_ptr
.text:001043D2                 jz      invalid_paramter
.text:001043D8                 cmp     [ctx_ptr+PavpCtx.guc_ctx], 0
.text:001043DC                 jnz     invalid_paramter

Next it allocates a context for GuC operations:

.text:001043E2                 push    90 ; sizeof(GucContext
.text:001043E4                 push    1
.text:001043E6                 call    near ptr calloc ; allocate GucContext (0x5A bytes)
.text:001043EB                 mov     [ctx_ptr+PavpCtx.guc_ctx], eax
.text:001043EE                 test    eax, eax
.text:001043F0                 pop     esi
.text:001043F1                 pop     edx
.text:001043F2                 jnz     short alloc_ok  ; start with no FD

The struct itself:

00000000 GucCtx          struc ; (sizeof=0x5A, mappedto_140)
00000000 vdm_file_descriptor dd ?                ; XREF: GUC_init:alloc_ok/w
00000000                                         ; GUC_init+5C/w ...
00000004 pg_timer        Timer ?
00000028 watchdog        Timer ?                 ; XREF: GUC_command_handler+8C/o
0000004C vdm             VDM_TX ?                ; XREF: GUC_init:loc_104441/o
00000056 state           dd ?                    ; XREF: GUC_pg_timer_routine+39/w
00000056                                         ; GUC_init+127/w
0000005A GucCtx          ends

It first checks if a file descriptor has already been setup by the Graphics Key Manager, and if so uses the same file descriptor - apparently they share the same VDM channel. Otherwise a new FD is setup in setup_guc_vdm(). The rest of the code initializes two timers - one related to some kind of watchdog and the other to power managment.

.text:0010440C
.text:0010440C alloc_ok:                               ; CODE XREF: GUC_init+2F↑j
.text:0010440C                 mov     [eax+GucCtx.vdm_file_descriptor], 0FFFFFFFFh ; start with no FD
.text:00104412                 mov     eax, [ctx_ptr+PavpCtx.graphic_key_mgr]
.text:00104415                 test    eax, eax
.text:00104417                 jz      short no_gkm
.text:00104419                 mov     edx, [ctx_ptr+PavpCtx.guc_ctx]
.text:0010441C                 mov     eax, [eax+GkmCtx.vdm_file_descriptor]
.text:0010441F                 mov     [edx+GucCtx.vdm_file_descriptor], eax
.text:00104421
.text:00104421 no_gkm:                                 ; CODE XREF: GUC_init+54↑j
.text:00104421                 mov     eax, [ctx_ptr+PavpCtx.guc_ctx]
.text:00104424                 cmp     [eax+GucCtx.vdm_file_descriptor], 0
.text:00104427                 jns     short loc_104441
.text:00104429                 push    4B00FDh
.text:0010442E                 mov     esi, 100Eh
.text:00104433                 push    2
.text:00104435                 call    near ptr log_printf_0
.text:0010443A                 pop     eax
.text:0010443B                 pop     edx
.text:0010443C                 jmp     invalid_paramter
.text:00104441 ; ---------------------------------------------------------------------------
.text:00104441
.text:00104441 loc_104441:                             ; CODE XREF: GUC_init+64↑j
.text:00104441                 add     eax, GucCtx.vdm
.text:00104444                 push    eax
.text:00104445                 call    setup_guc_vdm
.text:0010444A                 mov     esi, eax

And this is the part we have been waiting for:

.text:00102810 setup_guc_vdm   proc near               ; CODE XREF: GKM_init+2D↓p
.text:00102810                                         ; GUC_init+82↓p
.text:00102810
.text:00102810 vdm             = dword ptr  8
.text:00102810
.text:00102810 vdm_ptr = edx
.text:00102810                 push    ebp
.text:00102811                 mov     eax, 1005h
.text:00102816                 mov     ebp, esp
.text:00102818                 mov     vdm_ptr, [ebp+vdm]
.text:0010281B                 test    vdm_ptr, vdm_ptr
.text:0010281D                 jz      short loc_10284A
.text:0010281F                 mov     al, byte ptr [vdm_ptr+(VDM_TX.msg+3)]
.text:00102822                 mov     dword ptr [vdm_ptr+VDM_TX.pci_req_id], 0B0h ; CSME: bus: 0, device: 22, function 0
.text:00102829                 or      eax, 7
.text:0010282C                 mov     [vdm_ptr+VDM_TX.pci_tgt_id], 10h ; GUC: buf: 0, device: 2, function 0
.text:00102832                 and     eax, 0FFFFFF8Fh
.text:00102835                 mov     byte ptr [vdm_ptr], 0D3h
.text:00102838                 mov     [vdm_ptr+3], al
.text:0010283B                 mov     al, [vdm_ptr+2]
.text:0010283E                 or      byte ptr [vdm_ptr+1], 0Fh
.text:00102842                 and     eax, 0FFFFFF80h
.text:00102845                 mov     [vdm_ptr+2], al
.text:00102848                 xor     eax, eax
.text:0010284A
.text:0010284A loc_10284A:                             ; CODE XREF: setup_guc_vdm+D↑j
.text:0010284A                 pop     ebp
.text:0010284B                 retn
.text:0010284B setup_guc_vdm   endp

Here we have the internal bus IDs for the GuC and CSME.

Results are retrieved using GUC_wait_for_message() - it uses select() to wait on the VDM file handle and parses the message. Something interesting I found out it that messages are not initiated only by the CSME - the GuC can initiate messages to the CSME and the CSME responds. GUC_wait_for_message() uses a handler table with 11 entries, but 4 are NULL.

For example, one message I decoded gets some production information for the chip:

.text:00103EDA GUC_api_get_production_info proc near
.text:00103EDA
.text:00103EDA var_14          = byte ptr -14h
.text:00103EDA var_13          = byte ptr -13h
.text:00103EDA var_E           = byte ptr -0Eh
.text:00103EDA var_D           = byte ptr -0Dh
.text:00103EDA var_C           = dword ptr -0Ch
.text:00103EDA ctx             = dword ptr  8
.text:00103EDA
.text:00103EDA ctx_ptr = esi
.text:00103EDA                 push    ebp
.text:00103EDB                 mov     ebp, esp
.text:00103EDD                 push    ctx_ptr
.text:00103EDE                 push    ebx
.text:00103EDF                 sub     esp, 0Ch
.text:00103EE2                 mov     [ebp+var_14], 0
.text:00103EE6                 mov     ctx_ptr, [ebp+ctx]
.text:00103EE9                 mov     eax, ds:stack_cookie_ptr
.text:00103EEE                 mov     [ebp+var_C], eax
.text:00103EF1                 xor     eax, eax
.text:00103EF3                 push    ctx_ptr
.text:00103EF4                 call    GUC_enable_power_gate
.text:00103EF9                 lea     eax, [ebp+var_14]
.text:00103EFC                 push    eax
.text:00103EFD                 call    test_byte_12h_from_snowball_rbe_sku
.text:00103F02                 pop     ecx
.text:00103F03                 test    eax, eax
.text:00103F05                 pop     ebx
.text:00103F06                 mov     ebx, 109h
.text:00103F0B                 jnz     short loc_103F48
.text:00103F0D                 mov     ebx, 9
.text:00103F12                 cmp     [ebp+var_14], 0
.text:00103F16                 jnz     short loc_103F48
.text:00103F18                 lea     eax, [ebp+var_13]
.text:00103F1B                 mov     ebx, 109h
.text:00103F20                 push    eax
.text:00103F21                 call    get_7_bytes_from_snowball_rbe_sku
.text:00103F26                 pop     edx
.text:00103F27                 test    eax, eax
.text:00103F29                 jnz     short loc_103F48
.text:00103F2B                 mov     bl, [ebp+var_E]
.text:00103F2E                 mov     al, [ebp+var_D]
.text:00103F31                 shr     bl, 2           ; actuall data from CPUs looks like production year & week
.text:00103F34                 and     eax, 0Fh
.text:00103F37                 shl     eax, 9
.text:00103F3A                 and     ebx, 3Fh
.text:00103F3D                 shl     ebx, 0Dh
.text:00103F40                 or      ebx, 109h
.text:00103F46                 or      ebx, eax
.text:00103F48
.text:00103F48 loc_103F48:                             ; CODE XREF: GUC_api_get_production_info+31↑j
.text:00103F48                                         ; GUC_api_get_production_info+3C↑j ...
.text:00103F48                 push    ctx_ptr
.text:00103F49                 call    GUC_enable_power_gate
.text:00103F4E                 push    2
.text:00103F50                 push    ebx
.text:00103F51                 push    ctx_ptr
.text:00103F52                 call    GUC_send_VDM
.text:00103F57                 mov     edx, [ebp+var_C]
.text:00103F5A                 xor     edx, ds:stack_cookie_ptr
.text:00103F60                 jz      short loc_103F67
.text:00103F62                 call    near ptr __stkchk
.text:00103F67
.text:00103F67 loc_103F67:                             ; CODE XREF: GUC_api_get_production_info+86↑j
.text:00103F67                 lea     esp, [ebp-8]
.text:00103F6A                 pop     ebx
.text:00103F6B                 pop     ctx_ptr
.text:00103F6C                 pop     ebp
.text:00103F6D                 retn
.text:00103F6D GUC_api_get_production_info endp

Why do I think this is related to production information? Because it reads data from a file called "/snowball/rbe_sku" (Intel’s name!). I don’t have any idea what Snowball means, RBE usualy means ROM Boot Extenion, so it reads data from the ROM? The actuall data from a few processors appears to be correlated to production year and work week for the CPU.

.text:00148AF7 test_byte_12h_from_snowball_rbe_sku proc near
.text:00148AF7                                         ; CODE XREF: pavp_init+10A↑p
.text:00148AF7                                         ; GUC_api_get_production_info+23↑p ...
.text:00148AF7
.text:00148AF7 buffer          = byte ptr -24h
.text:00148AF7 stack_cookie    = dword ptr -8
.text:00148AF7 var_4           = dword ptr -4
.text:00148AF7 out_byte_12h    = dword ptr  8
.text:00148AF7
.text:00148AF7                 push    ebp
.text:00148AF8                 mov     ebp, esp
.text:00148AFA                 push    ebx
.text:00148AFB                 sub     esp, 20h
.text:00148AFE                 mov     eax, ds:stack_cookie_ptr
.text:00148B03                 mov     [ebp+stack_cookie], eax
.text:00148B06                 xor     eax, eax
.text:00148B08                 lea     eax, [ebp+buffer]
.text:00148B0B                 push    1Ch
.text:00148B0D                 mov     ebx, [ebp+out_byte_12h]
.text:00148B10                 push    eax
.text:00148B11                 push    offset aSnowballRbeSku_0 ; "/snowball/rbe_sku"
.text:00148B16                 call    read_file_completely
.text:00148B1B                 add     esp, 0Ch
.text:00148B1E                 test    eax, eax
.text:00148B20                 jnz     short loc_148B2A
.text:00148B22                 mov     dl, [ebp+buffer+12h]
.text:00148B25                 and     edx, 1
.text:00148B28                 mov     [ebx], dl
.text:00148B2A
.text:00148B2A loc_148B2A:                             ; CODE XREF: test_byte_12h_from_snowball_rbe_sku+29↑j
.text:00148B2A                 mov     ecx, [ebp+stack_cookie]
.text:00148B2D                 xor     ecx, ds:stack_cookie_ptr
.text:00148B33                 jz      short loc_148B3A
.text:00148B35                 call    near ptr __stkchk
.text:00148B3A
.text:00148B3A loc_148B3A:                             ; CODE XREF: test_byte_12h_from_snowball_rbe_sku+3C↑j
.text:00148B3A                 mov     ebx, [ebp+var_4]
.text:00148B3D                 leave
.text:00148B3E                 retn
.text:00148B3E test_byte_12h_from_snowball_rbe_sku endp

.text:00148A54 read_file_completely proc near          ; CODE XREF: get_7_bytes_from_snowball_rbe_sku+21↓p
.text:00148A54                                         ; test_byte_12h_from_snowball_rbe_sku+1F↓p ...
.text:00148A54
.text:00148A54 filename        = dword ptr  8
.text:00148A54 buffer          = dword ptr  0Ch
.text:00148A54 byte_count      = dword ptr  10h
.text:00148A54
.text:00148A54                 push    ebp
.text:00148A55                 mov     ebp, esp
.text:00148A57                 push    edi
.text:00148A58                 push    esi
.text:00148A59                 push    ebx
.text:00148A5A                 push    0
.text:00148A5C count = esi
.text:00148A5C                 mov     count, [ebp+byte_count]
.text:00148A5F
.text:00148A5F open_file:
.text:00148A5F                 push    [ebp+filename]
.text:00148A62                 call    near ptr open
.text:00148A67 file_handle = ebx
.text:00148A67                 mov     file_handle, eax
.text:00148A69                 pop     eax
.text:00148A6A                 test    file_handle, file_handle
.text:00148A6C                 pop     edx
.text:00148A6D                 mov     eax, 222
.text:00148A72                 js      short loc_148A98
.text:00148A74
.text:00148A74 read_file:
.text:00148A74                 push    count
.text:00148A75                 push    [ebp+buffer]
.text:00148A78                 push    file_handle
.text:00148A79                 call    near ptr read
.text:00148A7E
.text:00148A7E close_file:
.text:00148A7E                 push    file_handle
.text:00148A7F                 mov     edi, eax
.text:00148A81                 call    near ptr close
.text:00148A86                 add     esp, 10h
.text:00148A89                 test    edi, edi
.text:00148A8B                 js      short loc_148A93
.text:00148A8D                 xor     eax, eax
.text:00148A8F                 cmp     edi, count
.text:00148A91                 jz      short loc_148A98
.text:00148A93
.text:00148A93 loc_148A93:                             ; CODE XREF: read_file_completely+37↑j
.text:00148A93                 mov     eax, 99
.text:00148A98
.text:00148A98 loc_148A98:                             ; CODE XREF: read_file_completely+1E↑j
.text:00148A98                                         ; read_file_completely+3D↑j
.text:00148A98                 lea     esp, [ebp-0Ch]
.text:00148A9B                 pop     ebx
.text:00148A9C                 pop     esi
.text:00148A9D                 pop     edi
.text:00148A9E                 pop     ebp
.text:00148A9F                 retn
.text:00148A9F read_file_completely endp

Conclusion

I am still actively working on this to see what attack surfaces there are from GuC->CSME and CSME->GuC, but it looks like Intel did a really good job checking bounds and arguments. The Graphics Key Manager is next in the queue, it look like the surface there is more promising.

There is also a lot more to decode in PAVP, I only decoded a small part of the context structure:

PavpCtx         struc ; (sizeof=0x80, mappedto_123)
00000000 field_0         dd ?
00000004 field_4         dd ?
00000008 heci_client     dd ?
0000000C server_ctx      dd ?
00000010 graphic_key_mgr dd ?                    ; XREF: GUC_init+4F/r
00000014 vkm             dd ?
00000018 guc_ctx         dd ?                    ; XREF: GUC_pg_timer_routine+32/r
00000018                                         ; GUC_disable_power_gate?+1E/r ...
0000001C Lspcon          dd ?                    ; XREF: LSPCON_command_handler+22/r
0000001C                                         ; LSPCON_playback_done+39/r ...
00000020 field_20        dd ?
00000024 timer_ctx       dd ?                    ; XREF: GUC_disable_power_gate?+56/r
00000024                                         ; GUC_command_handler+90/r ... ; struct offset (PavpPortConfig)
00000028 field_28        dd ?
0000002C port_cfg        PavpPortConfig ?
00000044 field_44        dd ?
00000048 field_48        dd ?
0000004C field_4C        dd ?
00000050 field_50        dd ?
00000054 handlers        dd ?                    ; XREF: GUC_command_handler+29/r
00000058 field_58        dd ?
0000005C field_5C        dd ?
00000060 field_60        dd ?
00000064 field_64        dd ?
00000068 field_68        dd ?
0000006C field_6C        dd ?
00000070 field_70        dd ?
00000074 field_74        dd ?
00000078 field_78        dd ?
0000007C field_7C        dd ?
00000080 PavpCtx         ends

Enough for today, especially as my day job has warmed up a bit in the last three weeks - more on that later! I promise it will be very interesting (but not hardware related).

Security of the Intel Graphics Stack - Part 1 - Introduction

2021-02-10T00:00:00-05:00

General Architecture
Core Graphics
The GuC
Host Graphics Architecture
Conclusion

I promised I’ll post stuff about low level hardware issues, and here is my second post on the subject, the first part in a series about the Intel graphics stack.

This post series will be a summary of about a decade of unpublished research I am trying to organize and share. Not all of it is current, as newer hardware is harder to inspect and reverse, but I think much of the research is relevant.

The first post below is a quick introduction to the different components on the hardware and software side we’ll need to dive into security issues in the next post.

General Architecture

Processor graphics - The graphics unit that is part of the processor itself. Has had many codenames over the years, HD Graphics, UHD Graphics, Iris, Gen9, Gen11, Intel Xe and so on. Even the ‘Gen’ name has double meaning - both generation and ‘Graphics ENgine’. In UEFI code it is sometimes refered to at the IGD - Integrated Graphics Device.
The GuC - an embedded i486 core that supports graphics scheduling, power management and firmware attestation.
UEFI and OS Drivers

Core Graphics

As discussed in the introduction to the SecureBoot post, the Intel CPU has four major component groups - the CPU cores, the L3 (or LLC) cache slices, the ‘Uncore’ or ‘System Agent’ parts, all connected through a ring bus inside the die.

The graphics process is made up from several slices and an unslice (like uncore) area that includes common components. Each slice is divided into subslices and a slice common area. The subslices are made up of several Execution Units (EUs), and Texture unit and a L1 Cache/Memory. The common area includes the L3 cache and the dataport. The limit to the number of slices is the interconnect between them and the unslice. There is always only a single unslice. In the unslice we can find the connection to the ring bus, aptly named the GT interface (GTI), the Command Streamer is reads commands from the system memory and into the graphics processor, the fixed function pipline (FF pipeline), and the thread dispatcher \& spawner that lunch shader programs and GPGPU (General Purpose Computing) programs onto the EUs. The FF pipeline deals with fixed functions such as vertice operations (called the Geometry Pipe), and other dedicated hardware such as video transcoding.

Different SKUs have different combinations of these. For example:

Skylake GT2: 1 slice of 3 subslices of 8 EUs (1x3x8)
Skylake GT3: 2x3x8
Skylake GT4: 3x3x8

The graphics engine is also connect straight to the IOSF (Intel On Chip Fabric internal bus, see the secureboot post bus, through a controller called Gunit. Gunit is connect to both the primary and secondary IOSF and exports functions for communicating with the graphics engine and implementing IOMMU support for graphics memory and unified memory.

All of this is connected to the display IO interconnect and output to DisplayPort and HDMI outputs.

2D Graphics Pipeline

The 2D graphics engine is a standalone IP block in the unslice area, and has its own command streamer, registers and cache. It has 256 different operation codes, for example:

3D Graphics Pipeline

The fixed function pipeline in the unslice implements the DirectX 11 redndering pipeline stages: Vertex Fetch -> Vertex Shader -> Hull Shader -> Tessellator -> Domain Shader -> Geometry Shader -> Clipper -> Windower -> Z Ordering, -> Pixel Shader - >Pixel Output. Some of these functions are self contained, but many are implemented using by running shader programs on the EUs in the slices. EUs can send certain operations back into dedicated hardware units.

The Execution Units (EUs)

The EUs are in-order mulithreaded SIMD processing cores. Each execution thread is dispatched has its own 128 register space and executed programs called “kernels”. All instructions are 8 channels wide, e.g. operate on 8 registers at a time (or 16 half registers). Its supports arithmetic, logical and control flow instructions on floats and ints. Registers are addressed by address. The EU thread dispatcher implements priorities based on age, i.e. oldest is highest priority, and whether the trhead is blocked waiting on instruction fetches, register dependencies etc’. C

The GuC

The GuC is a small embedded core that supports graphics scheduling, power management and firmware attestation. It is implemented in an i486DX4 CPU (also called P24C and Minute IA), although it seems that since broadwell it has been extended to the Pentium (i586) ISA. It runs a small microkernel call μOS. The GuC μOS runs only kernel level tasks (even though μOS supports μApps). The firmware is written in C with not stdlib. In the GuC we can find supporting blocks: ROM memory, 8KB L1 on core cache, 64KB/128KB/256KB (Broadwell/Skylake/CannonLake) of SRAM memory which is used for code+data+cache and a 8KB stack. It also has power management, DMA engine, etc’. Communication to the GuC is done through memory-mapped IO and bidirectional interrupts.

The GuC offers a light-weight mechanism for dispatch work the host submits to the GPU. This means the GPU driver does not need to handle dispatch and job queuing, making it much faster. The user mode driver (UMD) can communicate with the GuC directly when required and bypass the need to context switch the main CPU into kernel mode. The kernel mode driver (KMD) uses the GuC as a gateway for job submission as well. This simplifies the Kernel and provides a single point where all jobs are submitted. Communication between the UMD and the GuC is done through shared memory queues.

Why is the GuC interesting? Because I think it can communicate with the CSME, CPU and GPU and everything over the IOSF, and if it has bugs it can be used to gain very privileged access to the system and memory.

Boot ROM and GuC firmware

At system startup GuC is held at reset state until the UEFI firmware initializes the shared memory region for the GPU. Inside the shared region a special subregion call WOPCM is set aside fur GuC (and HuC) firmware. It then releases the GuC from reset and it in turn starts executing a small non-modifiable Boot ROM (16/32KB in size) that initializes the basic GuC hardware, and waits for an interrupt signalling the firmware has been copied to the WOPCM region. The GuC firmware is an opaque blob supplied by Intel as part of the GPU KMD, which copies it to the shared memory region (GGTT) and signals the Boot ROM with an interrupt. The bootrom verifies the firmware with a digital signature using a SHA256 hash + PKCSv2.1 RSA signature, and if the test passes copies it to SRAM and starts executing.

The GUC firmware can be extracted from the graphics driver and reversed. Screenshot of IDA open on the kabylake GuC:

The GuC also attest the firmware for the video decoder unit, called HuC. The HuC is an HEVC/H.265 decoded implement in hardware.

The μOS kernel

The μOS kernel runs in 32-bit protected mode, with no paging and old-style segments model (CS, DS, etc’). All code run in ring0. The OS handles HW/SW exceptions and crashes, and supplies debugging and logging services.

Interrupts are handled through the local APIC - I found interrupts coming from the IOMMU, power management, display interfaces, the GPU and the CPU.

It runs a single process - which initializes the system and then waits for interrupts/events in a loop.

Communication with the OS

Commands are dispatched through a ring buffer work queue. Each work item has a header followed by a command. Once a command is posted the CPU notifies the GuC using a “doorbell” interrupt.

The Windows kernel mode driver supports GuC debugging by setting a registry key:

\\REGISTRY\MACHINE\SOFTWARE\Intel\KMD\GuC\\
	GuCEnableUkLogging=1
    
\\REGISTRY\MACHINE\SOFTWARE\Intel\KMD\GuC\\
    GuCLoggingVerbositySelect=0/1/2/3 (low, medium, high, max)

Host Graphics Architecture

So far we only discussed hardware. The software part of the graphics stack is divided into three levels: UEFI DXE, kernel mode and user mode.

UEFI

Traditionally VGA support was implemented with a legacy Video VBIOS as an PCI option ROM. In UEFI VBIOS was modified into a DXE driver call the Graphics Output Protocol (GOP), which support basic display for the UEFI setup menu and for the OS bootloader. The GOP is supplied by Intel to the UEFI vendor. The GOP supplies two basic functions:

Changing the graphics mode - resolution, pixel depth, etc’
Getting the physical address of the framebuffer

The Windows boot-loader uses the GOP to setup a memory mapped video framebuffer before entering VBS, and after the hypervisor and SK are loaded the access by winload is only through the framebuffer without invoking the GOP. Windows also uses the GOP for disabling blue screens.

Windows

On Windows, Intel supplies a fairly large graphics driver that implements both the user mode driver (UMD) and kernel mode driver (UMD). Applications using Direct3D communicate through the D3D runtime to the DXGI abstraction interface (in dxgkrnl.sys), which in turn communicated with the KMD. The KMD treats 2D Blt and 3D operations through different pipelines and dispatches the operations to the GPU.

The GPU driver is riddled with telemetry, but I haven’t figured out yet how much of it is sent automatically to Intel, altough crashes are sent through OCA - Online Crash Analysis.

Basic Memory Management

A very important job of the graphics drivers (both KMD and UMD) is memory management (GMM). The Graphics Memory space is the virtual memory allocated to the GPU, and is translated using the system pages tables to the physical RAM. The memory contains stuff lime geometry data, textures, etc’. The GPU hardware used Graphics Page Tables (GTTs) to decode virtual addresses supplied by the software graphics memory space into hardware. The use of MMUs and page tables on both ends (sw \& hw) has three main benefits: virtualization, per-process isolated graphics memory and non-contiguous physical memory for better utilization.

The GTTs come in two variants:

Global GTT - a single one level table mapping directly into system pages. It is managed by the HW and configured in UEFI. The UEFI DXE driver maps the GTT into memory and initializes it. It is also called Graphics Stolen Memory (GSM) and Unified Memory Architecture (UMA), not to be confused with CSME’s UMA.
Per-process GTT (PPGTT). This has changed significantly in the Broadwell graphics engine, so we’ll discuss only the new architecture. Modern PPGTT is basically a mirror of the CPU’s paging model with 4 paging levels.

The GMM part of the KMD handles and tracks graphics allocations, manages the GTTs, caching coherence, stolen memory allocation and something I won’t go into right now called swizzling. The GMM is essential for performance as it allows memory to be setup by the CPU and then accessed by the GPU directly without copying from system memory to GPU memory.

Its important to note that in modern system the whole system memory can be used for graphics. The driver reports fictious “dedicated” video memory probably to fix old games.

Security-wise, the graphis driver needs to make sure user process can gain access only to memory allocated to that process, and is cleared before transferring the memory to a different process.

SVM Mode

The Intel GPU have added support for another organic memory model, the OpenCL SVM model. In SVM mode the GPU and CPU share the exact same page table, so data structures can be shared AS-IS between both, including embedded pointers and such. Five levels of SVM are supported.

Coarse grained - CPU \& GPU have different buffers
Fine grained - CPU \& GPU can share memory buffer
Fine grained system - CPU \& GPU share entire system memory

+-----------------+------------------------------------------------------------------------------+
|                 | Type                                                                         |
+-----------------+-----------------------+--------------------------------+---------------------+
|                 |  Coarse-graind-buffer | Fine-grained buffer            | Fine-grained system |
+-----------------+                       +-----------------+--------------+                     |
| Type            |                       | without atomics | with atomics |                     |
+-----------------+-----------------------+-----------------+--------------+---------------------+
| Shared          | V                     | V               |        V     | V                   |
| virtual         |                       |                 |              |                     |
| address         |                       |                 |              |                     |
| space           |                       |                 |              |                     |
+-----------------+-----------------------+-----------------+--------------+---------------------+
| No need for     |                       | V               |        V     | V                   |
| explicit        |                       |                 |              |                     |
| mapping by host |                       |                 |              |                     |
+-----------------+-----------------------+-----------------+--------------+---------------------+
| Fine-           |                       | V               |        V     | V                   |
| grained         |                       |                 |              |                     |
| coherency       |                       |                 |              |                     |
+-----------------+-----------------------+-----------------+--------------+---------------------+
| Fine-           |                       |                 |        V     | V                   |
| grained         |                       |                 |              |                     |
| synchorinzation |                       |                 |              |                     |
+-----------------+-----------------------+-----------------+--------------+---------------------+
| Implicit use    |                       |                 |              | V                   |
| of memory       |                       |                 |              |                     |
| from CPU        |                       |                 |              |                     |
| malloc() from   |                       |                 |              |                     |
| GPU and entire  |                       |                 |              |                     |
| CPU address     |                       |                 |              |                     |
| space           |                       |                 |              |                     |
+-----------------+-----------------------+-----------------+--------------+---------------------+

Cache Coherence

Both the CPUs and GPUs have a complex memory hierarchy involving many caches. For example:

CPU: L1 Cache -> L2 Cache -------------\ 
                                       |------> *System LLC Cache -> eDRAM -> RAM 
GPU: Transient Cache -> GPU L3 Cache --/

GPU memory accesses do not pass through the CPU core’s L1+L2 caches, so the GPU implements snooping to maintain memory-cache coherency. The GPU basically sniffs the traffic on the CPU L1/L2 caches, and invalidates its own cache (I think this is relevant only to BigCore CPUs, and on Atom this is optional and very costly). The GPU’s transient caches are not snoopable by the CPU and must be explicitly flushed. The GPU L3 Cache is snoopable by the CPU on some Intel platforms.

Boot process

At boot, the operating system and kernel mode drive will detect and query the display devices, initialize a default display topology. After boot up, display config request will be sent to KMD and KMD in turn will configure the GEN display hardwires There are also use cases of display hot-plug during runtime, handled by OS user and kernel mode modules/drivers.

Once the driver is loaded it DirectX initializes it from DxgkDdiStartDevice() which eventually leads to a function that setups the render table per architecture:

void setup_render_function_table(HW_DEVICE_EXTENSION *pHwDevExt)
{
    KM_RENDER_CONTEXT   *render_context;

    ...

    switch(get_render_core(pHwDevExt))
    {
    ...
        case GEN3_FAMILY:
            ...
        case GEN4_FAMILY:
            ...
            ...
        case GEN8_FAMILY:
            render_context->FuncTable.PresentBlt                     = func_Gen6PresentBlt;
            render_context->FuncTable.PresentFlip                    = func_Gen6PresentFlip;
            render_context->FuncTable.RenderBegin                    = func_Gen6RenderBegin;
            render_context->FuncTable.Render                         = func_Gen7Render;
            render_context->FuncTable.RenderEnd                      = func_Gen6RenderEnd;
            render_context->FuncTable.GDIRender                      = func_Gen6GDIRender;
            render_context->FuncTable.BuildPagingBuffer              = func_Gen7BuildPagingBuffer;
            render_context->FuncTable.SubmitCommand                  = func_Gen8SubmitCommand;
            render_context->FuncTable.PreemptCommand                 = func_Gen6PreemptCommand;
            render_context->FuncTable.QueryCurrentFenceIRQL          = func_Gen6QueryCurrentFenceIRQL;
            render_context->FuncTable.IdleHw                         = func_Gen6IdleHw;
            render_context->FuncTable.StopHw                         = func_Gen6StopHw;
            render_context->FuncTable.ResumeHw                       = func_Gen6ResumeHw;
            render_context->FuncTable.GetMDLToGttSize                = func_GetMdlToUpdateGTTCmdSize;
            render_context->FuncTable.UpdateMDLToGtt                 = func_MDLToGttUpdateGttCmd;
            render_context->FuncTable.GetMDLToGttSizeOnePage         = func_GetMdlToUpdateGTTCmdSizeOnePage;
            render_context->FuncTable.UpdateMDLToGttOnePage          = func_UpdateOneGttEntry;
            ...

OCA

OCA is a mechanism that lets drive store device data and send it through windows update back to the driver vendor. There are two cases of failures:

Windows thinks there is a problem and the driver needs to be reloaded (TDR). Windows calls DxgkDdiCollectDbgInfo(), a mechanism that lets drive store device data and send it through windows update back to the driver vendor. The Intel GPU driver can add more then 1MB of data through DxgkDdiCollectDbgInfo().
In case of a blue screen (bugcheck), KmBugcheckSecondaryDumpDataCallback() is called and the driver passes data to it. After both function the data is converted into an OCA blob using CreateOCAXXXDivision, and it is later uploaded to Microsoft and from there to Intel. The Intel OCA blob contains lots of system and driver information, including what appears to be an Intel specific unique identifier assigned by the driver to the machnine and can be used for tracking.

Conclusion

In this post we learned the basic components of the graphics stack. In the next post on the graphics stack we’ll start looking into security implications.

Analysis of SSH keys found in the wild

2021-02-08T00:00:00-05:00

In 2018 I was contracted to help a large organization with a very distributed and remote structure. One of the things that I found was that the organization does not have a strict policy regarding the creation, storage and lifecycle of SSH keys.

I decided to look into this issue in general, so in Feb 2019 wrote a crawler that looked for SSH keys around the web - public repos, s3 bucket with bad permissions, data dumps from companies and so on.

From this I got 4807 keys. Next I wrote a small python script that tried the SSH keys - just autenticate and close the connection, without opening any channels as to not actually access the target systems which would be illegal.

I managed to authenticate into 221 hosts, 5 were FreeBSD, 1 was MacOS, 3 were Linux on ARM64, and the rest were Linux x64. This means I have 221 working keys found on the web and no way to notify their owners they should change their keys.

General interesting statistics:

Of the 4807 keys 966 were malformed and 1036 were encrypted (20%). Of the 1036 encrypted I could break 88 passwords using dictionaries and an additional 41 passwords using John-the-ripper on a 3-year old 8-core Xeon workstation after a month of brute-forcing.

Sizes (all were SHA256):

root@DESKTOP-MR4OQPJ:~/keys# for i in id_rsa* ; do ssh-keygen -l -f $i; done | sed 's/:.*//' | sort | uniq -c | sort -n -k 2
1023 SHA256
1024 SHA256
2047 SHA256
2048 SHA256
3000 SHA256
4048 SHA256
4096 SHA256
8192 SHA256
16384 SHA256

I don’‘t get the wird sizes: 1023-bit, 2047-bit, 3000-bit, and 4048-bit. Anyone have an idea?

Encryption type:

root@DESKTOP-MR4OQPJ:~/enc# grep -h DEK-Info id_rsa* | sed 's/,.*//' | sort | uniq -c
DEK-Info: AES-128-CBC
DEK-Info: AES-256-CBC
DEK-Info: DES-EDE3-CBC

Why still use DES keys?

for keys that I could not break:

DEK-Info: AES-128-CBC
DEK-Info: AES-256-CBC
DEK-Info: DES-EDE3-CBC

Distributions (in 2019, from uname)
87 were Ubuntu
38 were RHEL/Centos 6
25 were RHEL/Centos 7
7 were Amazon
5 were RHEL/Centos 5
2 were Debian
2 were CoreOS
1 was Gentoo
1 was Fedore32
2 were armv7l
1 was armv5tel
the rest I could not identify from uname -a
Most common kernels (in 2019, from uname)
44 were Linux 2.6.x
39 were Linux 4.4.x
28 were Linux 4.15.x
35 were Linux 3.10.x
15 were Linux 3.13.x
13 were Linux 4.9.x

Last week (after two years!) I reran the test against the 221 working keys and 179 still work. To make sure these are not honepots I added to the testing script a checked for the length of the remote .bash_history file, and none seem to be honeypots.

Abusing Sybase for lateral movement

2021-02-07T00:00:00-05:00

A few years ago I was asked to help on a red-team exercise in a company doing hardware R&D.

The company had a very strict password policy, and every computer had a randomized local adminsitrator account password and local SMB server disabled.

We managed to gain access to one developer but got stuck there. We did find one thing though: many of the developers had Sybase Adaptive SQL server installed on their systems as it was bundled by default with LabVIEW and Siemens Step 7, both in use by the target.

I installed LabVIEW and tried accessing it through the Adaptive SQL client. Looking through the connect dialog I notice something interesting: one of the options was "Start and connect to a database on another computer":

When selecting this option you need to specify the DB filename. I tried specifying an SMB server and could and pressed "Connnect". Amazingly, the target computer connected back over an SMB null session to the share I specified. I setup a Samba server that allows anonymous access and placed a DB file I crafted with credentials I specified during creation. This time I managed to connect and execute SQL statments against my server. What was more interesting, the account permissions and roles were set by the DB file and not by the host, so I could setup in advance in my DB to have an administrator role and then I could execute "xp_cmdshell" on the remote host.

We tried this in the field using ssh port forwarding back home on 445 and got access to most developer computers.

This was quiet a few years ago, but looking over the CVE DB for Sybase I don't see any issue that sounds like that, so I guess if you encounter Step7 or LabVIEW during a pentest you now know what to do …

In-depth dive into the security features of the Intel/Windows platform secure boot process

2021-02-04T00:00:00-05:00

Introduction and System Architecture
Overview
Early power on
Bring-Up (BUP)
CPU initialization
UEFI initialization
- Loading the boot loader
Windows Boot
Other OSs
More Protections

This blog post is an in-depth dive into the security features of the Intel/Windows platform boot process. In this post I'll explain the startup process through security focused lenses, next post we'll dive into several known attacks and how they were handled by Intel and Microsoft. My wish is to explain to technology professionals not deep into platform security why Microsoft's SecureCore is so important and necessary.

Introduction and System Architecture

We must first begin with a brief introduction to the hardware platform. Skip this if you have read the awsome material available on the web about the Intel architecture, I'll try to briefly summarize it here.

The Intel platform is based on one or two chips. Small systems have one, the desktop and server ones are separated to a CPU complex and a PCH complex (PCH = Platform Controller Hub).

The CPU complex deals with computation. It holds the "processor" cores, e.g. Sunny Cove that implement the ISA, as well as cross core caches like the L3 cache, and more controllers that are grouped together as "the system agent" or the "uncore". The uncore contains the memory controller and display, e.g. GPU and display controller.

The PCH handles all other IO, including access to the firmware through SPI or eSPI, wifi, LAN, USB, HD audio, SMBus, thunderbolt and etc'. The PCH also hosts several embedded processors, like the PMC, the Power Management Controller.

An additional part of the PCH is a very important player in our story, the CSME, or Converged Security & Management Engine, a i486 IP block (also called Minute IA). CSME is responsibly for much of the security model of Intel processors as well as many of the manageability features of the platform. The CSME block has its own dedicated ~1.5mb of SRAM memory and 128KB of ROM, as well as a dedicated IOMMU, called the A-Unit (that even has its own acode microcode) located in the CSME's uncore', thats allows access from ME to the main memory, as well as DMA to/from the main memory and using the main memory as an encrypted paging area ("virtual memory"). The CSME engine runs a customized version of the Minix3 microkernel, also recent versions have changed it beyond recognition adding many security features.

Buses

Lets use this post to also introduce the main interconnects in the system. The main externally facing interconnect bus is PCI-E, a fast bust that can reach 64GBps in its latest incarnations. A second external bus is the LPC, or Low Pin Count bus, a slow bus for connecting devices such as SPI flash, the TPM (explained below), and old peripherals such as PS/2 touchpads.

Internally the platform is based around the IOSF, or Intel On-chip System Fabric, which is a pumped up version of PCI-E that supports many additional security and addressing features. For addressing IOSF adds SourceID and DestID fields that contain the source and destination of any IOSF transaction, extending PCI-E Bus-Device-Function (BDF) addressing to enable routing over bridges. IOSF also extends addressing by adding support for multiple address root namespaces, currently defining three: RS0 for host memory space, RS1 for CSME memory space, and RS2 for the Innovation-Engine (IE), another embedded controller currently present only on server chipsets. There are two IOSF busses in the PCH - the Primary Fabric and the Sideband Fabric. The Primary Fabric is high speed, connecting the CPU to the PCH (through a protocol call DMI), as well as high speed devices such as Gigbait Ethernet, WiFi and eSPI. The Sideband Fabric is used to connect the CSME to low-speed devices, including the PMC (Power Management Controller), the RNG generator, GPIO pins, USB, SMBus, and even debugging interfaces such as JTAG.

More Components

Another interesting component is the ITH, or Intel Trace Hub, which is codenamed North Peak (NPK). The ITH can trace different internal hardware component (VIA - Visualization of Internal Signals, ODLA - On-chip logic analyzer, SoCHAP - SOC performance counters, IPT - Intel Process Trace, AET - Intel Architecture Trace), and external component like CSME, the UEFI firmware, and you can even connect it to ETW. This telemetry eventually finds its way to Intel in various methods.

The TPM is designed to provide a tamper proof environment to enforce system security through hardware. It implements in hardware many essential functions: sha1 & sha256 hashing algorithms, many crypto and key derivation functions, measurment registers call the Platform Configuration Registers (PCRs), a secret key - Endorsment Key - used to derive all other keys, and non-volatile storage slots for storing keys and hashes. Discrete TPM chips (i.e. those that are a separate chip on the mainboard or SOC and connected through the LPC) are call dTPMs, or can be implemented in the CSME module's firmware and called fTPMs.

The TPM's PCR are initialized to zero when the platform boots and are filled up with measurements through the boot process. PCRs 0-15 are intended for "static" use - they reset when the platform boots; They are supposed to give the OS loader a view of the platform initialization state. PCRs 17-22 are for "dynamic" use - they get reset on each secure launch (GETSEC[SENTER]); They are supposed to be used by the attestation sofware that checks if the OS is trusted.

The Flash Chip

SPI flash has 5 major regions: the Descriptor regions, the CSME region, the Gigabit Ethernet Region, the Platform Data Region, and the UEFI region. In the image below you can see an example of how the flash is organized.

Later versions added more regions:

These regions are categorized as fault tolerant (FTPs) and non fault tolerant partitions (NFTPs). Fault tolerant partitions are critical for boot, and verified during early boot (like the RBE, the CSME ROM Boot extensions will discuss in a few paragraphs). If verification fails - the system does not boot. Examples of non fault tolerant partitions are the Integrated Sensor Hub (or ISH) firmware.

SPI flash protection is applied at multiple levels: On the flash chip itself, in the SPI flash controller (in the PCH), in UEFI code and in CSME code.

The SPI controller maps the entire flash to memory at a fixed address, so reads/writes are usually done simply by reading/writing memory. The SPI controller translates this to flash-specific commands issued on the SPI bus, using a table of flash-specific commands stored in the flash descriptor region. This is called "Hardware Sequencing", meaning the SPI controller issues the actual SPI commands When hardware sequencing is in use, the SPI controller enforces several flash protections based on the masters region table in the flash (but can be overriden using a hardware PIN).

The SPI controller also implements a FLOCKDN flag. FLOCKDN is a write-once bit that, when set, disables use of software sequencing and modification of the PR registers until the next reset. The CSME sets this in the Bring-UP process (bup_storage_lock_spi_configuration(), see below). This happens when the UEFI notifies it that it is at the end of POST. In addition to the region access control table, the SPI controller also has an option to globally protect up to five regions in the flash from write access by the host using five registers, called Protected Registers (PRs), which are intended for the UEFI firmware to protect itself from modification while the OS is running.

It is also possible to issue direct flash commands using "Software Sequencing" by writing to the OPTYPE/OPMENU registers, since this can be used circumvent the SPI-enforced protections, software sequencing is usually disabled after POST using the FLOCKDN bit.

How is the flash updated?

UEFI region is updated through an UEFI capsule, This update happens during POST, before PRs and FLOCKDN is set, therefore, the BIOS region is still accessible to UEFI code.

Many OEMS have then own UEFI anti-tamper protections. For example, HP has SureStart on laptops and workstations, and Dell has TrustedDevice SafeBIOS. SafeBIOS copies bad firmware images to the EFI system partition, and the Dell Trusted Device software on Windows sends their hashes plus the hash of the UEFI firmware currently in memory to a Dell cloud server (*.delltrusteddevicesecurity.com) to check against a list of "authorized" hashes. Server platforms have similiar protections, including iLO for HP and iDRAC in Dell. The CSME region can usually be updated only from within the CSME. However, for more complicated upgrades CSME can temporarily unlock the ME region for host read & write.

Overview

In the next sections we'll look over all the stages of boot.

Early power on

Boot starts the PMC, the Power Management Controller. In modern Intel systems the PMC is an ARC core and its the first controller to execute code once electricity is applied to the system. We'll talk more about PMC in a later post as its quiet interesting and has its own microcode and firmware, and event generates telemetry over the IOSF SB bus (which we'll talk about in a moment).

While the PMC does its init, the rest of the system is held at bay at a RESET state.

The next part to start running is the CSME. Recall from the first post in the series, CSME, or Converged Security and Managment Engine is a MinuteIA (i486 CPU IP block) embedded in the Platform Controller Hub (PCH). The CSME begins running from its own embedded 128KB ROM - the CSME-ROM. This ROM is protected with a hardware fuse that is burned by Intel during production. When started the CSME ROM starts like a regular 486 processor BIOS - in the reset vector in real mode. Its first order of business is to enable protected mode. Next it checks if the system is configured in ROM bypass mode to assist debugging, if so maps the ROMB partition in SPI and starts executing from there - a mode call ROM bypass mode which we might dig into later. Next the CSME's SRAM is initialized and a page table is created mapping SRAM and ROM and then paging is enabled. Once basic initialization is out of the way CSME can switch to C code that does some more complex initialization: initiating the IOMMU (AUnit), the IACP and hardware crypto keys which are calculated from fixed values in hardware. Finally, the DMA engine is used to read the next stage, called the Rom Boot Extension, or RBE, from the system firmware flash through SPI, and verifies it against the cryptographic keys prepared earlier. CSME ROM uses a special table, the Firmware Interface Table, or FIT, a table of pointers to specific regions in the flash and is itself stored in a fixed flash address.

The RBE's job is to load the CSME OS kernel and verify it cryptographically. This process is optimized by using a mechanism called ICV, or Integrity-Check Values, hardware cached verified hashes - as long as the CSME kernel has the same hash it does not require crypto verification. Another check performed by the RBE is an anti-rollback check, making sure that once the CSME has been upgraded to a new version it cannot be downgraded back to the original version. Before starting the main CSME kernel the RBE loads pre-OS modules. An example pre-OS module is IDLM, which can be used to load debug-signed firmware on a production platform.

The kernel starts by enabling several platform security features: SMEP, Supervisor Mode Access Prevention, prevents exploits from running mapped kernel memory from ring3, and DEP, Data Execution Prevention, which prevents exploits from running code from stack regions. It also generates per-process syscall table permissions, aswell as ACL and IPC permissions.

Bring-Up (BUP)

Once everything is ready the kernel loads the Process Manager which executed "IBL processes", which includes Bring-Up (BUP) and the Loader. The BUP loads virtual file system, or VFS server, parses the init script of the FTPR partition and loads all IBL modules listed there. This includes: the Event Dispatcher Server (eventdisp) - service that allows publishing, registering and acknowledging receipt of named events (sort of DBUS), the Bus Driver (busdrv) - a driver that permits other drivers to access devices on the CSME's internal bus, the RTC driver (prtc), the Crypto/DMA driver (crypto) - provices access to services offered by the OCS hardware (SKS, DMA engines), the Storage driver (storage) - which provides access to the MFS filesystem, the Fuse driver (fpf) and finally the Loader Server (loadmgr).

As seen in the image below, this is the stage where the CPU finally begins execution.

CPU initialization

Once the CSME is ready it releases the main CPU from the RESET state. The main CPU loads microcode from the FIT table and sets it up (after CSME verified the uCode cryptographically) . I won't go into details about microcode, also called uCode, here as I have a full post planned on microcode later. Whats important to know is that microcode does not only include the "implementation" of the instruction set architecture (ISA), but also many routines for intilization, reset, paging, MSRs and much mich more. As part of CPU initialization it loads another module from the FIT, the Authenticated Code Module (ACM). The ACM implements BootGuard, a security feature to check cryptographically verify the UEFI signature before it is loaded (once called "AnchorCove"). This begins the Static Root Of Trust Model (SRTM), where CSME ROM verifies the CSME, which verifies the microcode, which verifies the ACM, which verifies the UEFI firmware, which verifies the operating system. This is done by chaining their hashes and storing them in the TPM. The ACM also initializes TXT, the Dynamic Root of Trust Model (DRTM) which we will detail in a few paragraphs.

UEFI initialization

Once the CPU completes initialization, the Initial Boot Block (IBB) of the UEFI firmware is executed. The startup ACM authenticates parts of the FIT and the IBB using the OEM key burned into the fuses, authenticates it and measures it into PCR0 in the TPM. PCR0 is also referred to as the CRTM (Core Root of Trust Measurement)

The first stage of IBB is SEC which is responsible for very early platform initialisation, and loading the UEFI secure boot databases from non-volatile (NV) storage (these keys have various names such as PK, KEK, DB, DBX). Next comes PEI core, or "main" module of the Pre EFI initialization. It loads several modules (PEIMs) that initialiaze basic hardware such as memory, PCI-E, USB, basic graphics, basic power managment and more. Some of this code is implemented by the UEFI vendors or OEMs, and some come from Intel in "FSPs", Firmware Support Packages, which perform "Silicon Initialization". Common UEFI firmwears can have as many as a 100 PIE modules.

The UEFI spec does not covers signature/authentication checks in PEI phases. Thats why Intel needed BootGuard to do the bootstrapping: At power-on, BootGuard measures the IBB ranges which include PEI.

Following PEI the Driver Execution Environment is loaded by a security PEI module which verifies their integrity cryptographically beforehand. DXE is responsible for setting up all the rest of the hardware and software execution environment in preparation for OS loading. It also setups System Management Mode (which we'll talk about soon), sensors and monitoring, boot services, real-time clocks and more. A modern UEFI firmware can have as much as 200 different DXE drivers installed.

Many OEMs use BootGuard to authenticate DXE as well by configuring the IBBs to include the entire PEI volume in the flash (PEI Core + PEI modules) and the DXE Core. Secure Boot is used to verify each PEI/DXE image that is loaded before executing it. These images are measured and extended into the TPM's PCR0 as well.

The DXE environment initializes two important tables: the EFI Runtime services table and the EFI Boot Service Table. Boot Services are used by the operating system only during boot and discarded thereafter. These include memory allocation services and services to access DXE drivers like storage, networking and display. Runtime services are kept in memory for use by the operating system whenever required, and include routines for getting and setting the value of EFI variables, clock manipulation, hardware configuration, firmware capsule updates and more.

Finally the UEFI firmware measures the platform (e.g. chipset) security configuration (NV variables) into PCR1 and then locks them by calling a function in the ACM.

Loading the boot loader

The final driver to be loaded by DXE is the Bood Device Selection module or BDS. BDS scans its stored configuration, comparing it with the currently available hardware and decides on a boot device. This gets executed in legacy boot and non secureboot systems. In SecureBoot mode another DXE component called the SecureBootDXE is loaded to authenticate the OS boot loader. The cryptographic key used is stored in DXE and verified as part of BootGuard. SecureBootDXE also compares the boot loader agains a signed list of blacklisted or whitelisted loaders.

Windows Boot

Now we are ready for Transient System Load (TSL), most of DXE gets discarded and the OS bootloader is loaded. The bootloader (called the IPL) is measured into PCR4 and control is transfered to it. For Windows this is bootmgrfw.efi, the Windows Boot Manager. It first initialzes security policies, handles sleep states like hibernation, and finally uses EFI boot services to load the Windos loader, winload.efi.

Winload

Winload initializes the system's page tables in preparation for loading the kernel, loads the system registry hive, loads the Kernel and the Hardware Abstraction Layer (HAL DLL) and early boot drivers. They are all authenticated cryptographically, and their measurement are stored into the TPM. Once thats done, it uses UEFI memory services to initialze the IOMMU. Once everything is loaded into its correct place in memory, the EFI boot service are discarded.

HVCI

When HVCI, or HyperVisor protected Code Integrity is enabled a different process occurs. Winload does not load the kernel, instead loading the Hypervisor loader (hvload.efi), which in turn loads the hypervisor (hvix64.exe), and sets up a protected virtual machine called VTL1 - Virtual Trust Level 1. It then loads the Secure Kernel (SK) into VTL1, and then setups VTL0, the untrusted level for the normal kernel. Now winload.efi is resumed within VTL0 and continues to boot the system within VTL0. The secure kernel continues running in the background providing security features like authentication as well as memory protection services for VTL0.

Its important to note that the hypervisor and secure kernel do not trust UEFI, and do not initiate any UEFI calls while running. Any future UEFI runtime service calls will be executed from within the VTL0 virtual machine thus protected from harming the hypervisor and secure kernel.

The regular OS kernel boot then continues in VTL0. Malicous UEFI and driver code cannot affect the hypervisor or the secure kernel. Malicious drivers can and will continue to attack user mode code in VTL0, but they must be signed by Microsoft and thus can be analyzed before being approved or blocked quickly if a bug/exploit is found.

Dynamic Root of Trust Model (DRTM)

The whole security model presented so far is based on a chain of verifications. But what happens if that chain is broken by a bug? UEFI implementations have many security bugs, and those will affect the security of the whole system. To alleviate this issue Intel and Microsoft developed the Dynamic Root of Trust Model (DRTM), available since Windows 10 18H2. In DRTM, winload starts a new load verification chain using an Intel security feature called TXT. TXT measures critical parts of the OS during OS loading. The process is initiated by the OS executing a special instruction - GETSEC[SENTER], implemented in microcode, which results in the loading, authentication and execution of an ACM called the Secure Init ACM (SINIT ACM). The ACM can be on the flash, or can be supplied by the OS with the GETSEC instruction.

The GETSEC-SENTER microcode flow clears PCR17-23, does an initial measurement into PCR17 that includes the SINIT ACM and the parameters of the GETSEC instruction and executes the SINIT ACM. SINIT measures additional secure-launch related stuff into PCR17 which includes the STM (if present), digest of Intel Early TXT code and matching elements of the Launch Control Policy (LCP). The LCP checks the platform is in a known-good state by checking PCRs 0-7, and that the OS is in a known-good state by checking PCRs 18-19. Next SINIT measures authorities involved up to now into PCR18 (the measurement is of the authority (e.g. the signer/key) and not the data to allow for upgrades).

The OS now continues to load and use the PCRs for attestation telemetry.

SecureBoot + DRTM + BitLocker (Windows uses PCRs 7 and 11 for Secure Boot based BitLocker) make sure the system is almost impervious to attacks.

The Windows secure boot process is implemented in an executable call tcblaunch.exe, TCB - Trusted Compute Base. This is the executable the SINIT ACM measures and launches. The reason tcblaunch.exe was inevented is that data generated from within tcblaunch is considered secure, while data generated from winload can be tainted. A funny artifact of the MLE launch process is caused by the fact that it is 32-bit, but tcblaunch.exe is 64-bit. Microsoft hacked this by providing a 32-bit mlestartup.'exe binary inside the MSDOS header region of the MZ/PE file.

UEFI Memory Attributes Table

As stated before, Windows wants to run the UEFI runtime services in VTL0. By default the OS cannot lock these memory pages to be W^X (only write or only execute, not both) because many old UEFI systems still mix code and data. Microsoft solves this by introducing a new UEFI table, the UEFI Memory Attributes Table (MAT), which specifies if the runtime service should execute from VTL0 (by marking the memory region as EFI_MEMORY_RO|EFI_MEMORY_XP), or must run with RWX protections. Since this is a gaping whole, the UEFI runtime's parameters are santized using a VTL code - and this is enabled only for a restricted subset of runtime calls).

Other OSs

[IMAGE]

Some Linux distrubutions use Intel TBOOT implementation for DTRM launch. VMware ESXi support DRTM and TXT from version 6.7U1 using a customized version of TBOOT, and attastation information is managed through VSphere.

[ IMAGE]

More Protections

IOMMU and DMA protections

DMA is a platform feature that allows hardware to write directly to main memory bypassing the CPU. This greatly enhances performance, but comes with a security cost: hardware can overwrite UEFI or OS memory after it has been measured and authenticated. This means malicous hardware can attack the OS after boot and tamper with it.

To solve this problem the memory managment controller of the platform was extended to protect IO, and called the IOMMU. Intel calls this technology VT-d, and it implements address paging with permissions for DMA. The IOMMU allows the OS and its drivers to setup the memory regions devices are allowed to write to. Another protection mechanism in IOMMU used by the UEFI firmware and later the OS is Protected Memory Regions, or PMRs. These define regions that can only be accessed from the OS on the CPU and never by devices through DMA. The IOMMU must be enabled very quickly early in boot to protect from malicous on-board firmware attacking before the OS loads.

To ensure the mechanism for setting up the PMRs is not tampered with it too is measured, including the IOMMU ACPI table, the APIC table, the RAM structure definition, and DMA protection information.

Windows uses the IOMMU and PMRs to protect itself since Windows 10 18H2, and calls this feature Kernel DMA Protection. The Kernel DMA protection prevents DMA to VTL1, hypervisor and VTL0's kernel regions. Microsoft also allows special implement

There is an undocumented feature in the kernel used by Graphics/DirectX to allow sharing the kernel's virtual memory address space with the graphics card (Device-TLB, ExShareAddressSpaceWithDevice()).

Secure Devices

Microsoft allows some device to be isolated from VTL0 and used only from code in VTL1 to protect sensitive information used for logon, like the face recognition camera and fingerprint sensors. Secure devices discovered using ACPI table "SDEV" (SDEV_SECURE_RESOURCE_ID_ENTRY, SDEV_SECURE_RESOURCE_MEMORY_ENTRY).

Secure devices can be either pure-ACPI devices or PCI devices. Both can be targets for DMA requests

It seems the drivers for secure devices are actually VTL1 user-mode processes that call basic functions in IUMBASE to communicate with the device (DMA, read/write PCI configuration space, do memory-mapped IO), for example: GetDmaEnabler / DmaMapMemory / SetDmaTargetProperties / MapSecureIo / UnmapSecureIo

SMM

SMM, or the System Managment Mode, is a special mode invoked to handle various hardware and software interrupts, and is implemented as part of the UEFI firmware. For example, SMM can simulate a PS/2 keyboard by handling keyboard interrupts and translating them into USB read/write. When a legacy application performs an IO IN/OUT operation on a PS/2 port, the SMI handler registered for that port is executed, transfers the system into SMM mode, runs the DXE USB keyboard driver, and then returns the result transparently. SMM is also used for security features by allowing certain actions to occur only from SMM. The caveat of SMM is that it has full access to the system, and operates in "ring -2", even higher then VTL-1 and the hypervisor. It has been used for attacks for many years (look in google for NSA's SOUFFLETROUGH).

Intel & Microsoft have developed three technologies to protect the OS from SMM: IRBR, STM, PPAM.

IRBR, or Intel Runtime BIOS Resilience, runs the SMI handler in protected mode with paging enabled, with a page table set up to only map SMRAM, as well as CPU protection to prevent changes to the paging table in SMM mode.

STM - SMM Transfer Monitor, means that most of the SMI handler virtualized, with only a small part called the STM serving as its hypervisor. I don't think is actually implemented in UEFI.

PPAM - also called Nifty Rock or Devil's Gate Rock, tries to fill the gap between IRBR and STM by prepending an Intel entry-point to the SMI handler. Intel supplies a signed module called PPAM that can measure certain attributes of the SMI handler and report them to the OS. The OS can then make a policy decision on how to proceed. All SMI handler must also be registered in a table called the WSMT table. The firmware's WSMT tables declares to the OS that the firmware guarantees three things: FixedCommBuffers - a guarantee that the SMM will vaildate that the input/output buffers of the operation, CommBufferNestedPtrProtection that extends this guarantee to any pointers within input/output structures, and SystemResourceProtection that indicated that the SMI handler will not reconfigure the hardware.

Memory Reset protections

After a warm boot or even a fast cold boot some secrets (keys) might remain in memory. Intel provides security for these secrets using special TXT Secrets registers.

Starting a blog at this time

2021-02-01T00:00:00-05:00

Is this a good time to start a new cyber security blog?

I have been working in cybersecurity for quite some time, but have always been afraid of writing publicly about my work: afraid of being publicly rediculed for my work, afraid of my english proficiency and afraid in general.

When I finally got the curage to start the North Korean blog thing happened, litterally a half hour before starting, throwing me way down.

But finally I decided to go anyway.

I’ll be posting a lot of my backlog in the next few weeks, including: the Intel PC boot process, intel uCode stuff, expereicnes as a blue teamer.

Lets hope someone somewhere ever reads these words.

Too-doo-loo, Igor