Exadata Security: the joy of PXE and UEFI: Secure boot violation
Today I had the honour to join my colleague Freek D’Hooge (find him here: @dhoogfr )in the datacenter. Task for today, reimage an 1/8th X7 Exadata using PXE. Andy (Colvin) warned me already that, when trying to reimage them, they are now using UEFI – boot. Knowing this in advance, this would be an #exatastic day.
As usual, Andy was right. Trying to boot one of the cells on the pxe “the old method” resulted in:
“Secure boot violation: Invalid signature detected. Check Secure boot policy in setup”
So … we now knew we had some work to do.
Step 1, find your sole source of truth … my oracle support. This time, close but no cigar. The MOS-note “How to setup a PXE Boot Server to Re-Image an Exadata Compute Node (Doc ID 1577323.1)” was not complete. It is lacking the information on how to do a PXE boot with an UEFI system.
It is not difficult, at least once you know it.
A very valuable source of information, is actually the online documentation on how to setup a PXE server for a Oracle Linux 7 system. You can find that one here: https://docs.oracle.com/cd/E52668_01/E54695/html/ol7-install-pxe-dhcp-tftp.html
Because it’s a bit long, I’ll highlight the steps we had to take to convert our legacy PXE server, which we normally use and is built using the mos-note guidelines, into one who can support the UEFI boot.
Packages
First of all, you need some more required packages: BOOTX64.efi, grubx64.efi, shim.efi
and also, there are some optional ones as well: MokManager.efi
These rpms can be found in the online yum repo from oracle, or in the ISO files from oracle linux.
|
1 2 3 |
# cd /tempdir # rpm2cpio grub2-efi-version.rpm | cpio -idmv # rpm2cpio shim-version.rpm | cpio -idmv |
Of course, in the datacenter you don’t have internet access. No problem for that, these files are in the PXE – images you have downloaded from edelivery as well.
The grubx64, BOOTX64.efi and MokManager.efi are located in the nfsimg-18.xxxx.tar file.
You can get them out like this:
|
1 2 3 |
# tar xf nfsimg-18.<fill in your version>-cell.tar EFI # cd EFI/BOOT # cp BOOTX64.efi grubx64.efi MokManager.efi grub.cfg /tftpboot/efi |
the shim is a little trickier, you get it from the cellbits:
|
1 2 3 4 5 |
# tar xf nfsimg-<fill in your version>-cell.tar cellbits/commonos.tbz # cd cellbits # tar xf commonos.tbz shim-0.9-2.0.6.2.el6.x86_64.rpm # rpm2cpio shim-0.9-2.0.6.2.el6.x86_64.rpm | cpio -idmv # cp ./boot/efi/EFI/redhat/shim.efi /tftpboot/efi/ |
that’s all for the extra package requirements.
dhcpd.conf
Yes, this file needs some attention as well.
In the general section on top following info must be added:
|
1 2 3 |
set vendorclass = option vendor-class-identifier; option pxe-system-type code 93 = unsigned integer 16; set pxetype = option pxe-system-type; |
And in the subnet section you need an if-clause:
|
1 2 3 4 5 6 7 |
# Enable UEFI Netboot if substring(vendorclass, 0, 9)="PXEClient" { if pxetype=00:06 or pxetype=00:07 { filename "shim.efi"; } else { filename "pxelinux.0"; } |
take into account, that this is the way on my system. This is a relative path. Just keep that in mind it can be different for you.
Grub-files
In a normal pxe linux boot you need to create the files in pxelinux.cfg/01-<mac address lower case and – instead of : >
Well, this is a little different here as well. The name of the file it’s grub.cfg-01-“MAC ADDRESS with – separator” and in my case, it was also expecting an – at the end.
So we have two components, components and storage cells.
Compute nodes
They get this as content for the 01-<mac address> file for the compute node:
|
1 2 3 4 5 6 7 8 9 10 11 |
set default 0 set timeout=3 menuentry 'Compute Node' { echo "Loading vmlinux" linuxefi efi/vmlinux-nfs-<fill in the correct version for this file>-compute stit dhcp pxe boot-from=uefi factory reboot-on-success notests=diskgroup sk=<change to the pxe servers ip>:/tftpboot/ preconf=<change to the pxe servers ip>:/tftpboot/customerconfigs/preconf.csv console=ttyS0,115200n8 echo "Loading initrd" initrdefi efi/initrd-nfs-<fill in the correct version for this file>-compute.img echo "Booting install kernel" } |
Storage cell nodes
They get this as content for the 01-<mac address> file for the storage cell node:
|
1 2 3 4 5 6 7 8 9 10 11 |
set default 0 set timeout=10 menuentry 'Storage cell' { echo "Loading vmlinux" linuxefi efi/vmlinux-nfs-<fill in the correct version for this file>-cell stit dhcp pxe boot-from=uefi factory reboot-on-success notests=diskgroup sk=<change to the pxe servers ip>:/u01/exadata_nfs preconf=<change to the pxe servers ip>:/tftpboot/customerconfigs/preconf.csv console=ttyS0,115200n8 echo "Loading initrd" initrdefi efi/initrd-nfs-<fill in the correct version for this file>-cell.img echo "Booting install kernel" } |
Software location
As this is a first version, I will definitely change it to clean it up, the system expects the files in /tftboot. So move the files vmlinux-nfs* and initrd-nfs-* to /tftpboot/ and make sure to respect the proper permissions.
Restart services
As the configuration from the dhcp server has been changed, this service must be restarted. This can be done using
|
1 |
# service dhcpd restart |
And basically, that’s it. No other changes are needed. Set the bootdevice from the cell/compute to PXE and restart the components. They will be happy to boot from your PXE server.
Disclaimer: this works for me, and this is not intended as a copy/paste procedure. Also, make sure that you are comfortable with knowing what you are doing or changing. If in doubt about anything, seek some help and if necessary ask Oracle or an installation partner to assist you in this.
As always, questions, remarks? find me on twitter @vanpupi