Is this a good place for VM/hypervisor project discussion?

If so I've got a bunch of good ramblings...errrr, discussion for the forum.

Back story: I have lots of scanners, lots of receivers and quite a few ham rigs which are computer controllable. Much of the software which supports this stuff doesn't play well with contemporary Windows, so you have to run it in virtual machines.

Easy for the operator with a few receivers and one computer. I have at least 50 plus 12 discrete PCs but want to condense this down to one to three big systems.

Should anyone want to hear the rest of the story (as Paul Harvey used to say) I'll fill this thread in and hopefully generate some interest.

Continuing on:

The big challenge for users with a complex radio layout is getting enough I/O and enough visualization to control and display everything at once. In the main shack I have 9 32" monitors, 4 24" monitors, two 17" KVM displays and another display that will ultimately tie in to a monitoring system plus another port on whichever server is active. That equates to a lot of video cards. Combine that with the I/O requirements and things get complicated.

I'd previously solved this by building discrete systems, favoring signage-type mini PCs where possible and larger desktop cube systems if additional I/O cards or lots of drive space is needed. While this approach scales decently it still lacks provisions for storing warm copies or backups of VMs. You need a NAS or centralized repository, and THAT can also fail if the controller fails.

The donation of a server by a nephew led me to poke around quite a bit and flesh out some concepts which can be extended to workstations or desktops with enough processing power and RAM to support multiple concurrent VMs. Next, I'll cover hypervisor types and the problem with some popular software packages. Then look at some ways to get everything playing nicely.

A word about hypervisors:

There are two distinct types and each has it pluses - and minuses.

Bare metal hypervisors (such as xcp-ng) are the go-to where speed is required, but the one thing they don't do well is to assign discrete resources to individual VMs. Case in point: A PCIe serial adapter with 8 ports, or a video card with 4 display outputs. You typically have to pass the entire card through to a given VM, making it unavailable to other VMs. (If we had 16 PCIe slots in a system this wouldn't be a big deal - but most have less than 8. So we have to carefully consider what to run and how to run it on such a system.)

The next type - Type 2 - are OS-hosted. QEMU, VirtualBox, VM Player and similar fit this description. Benefits include being able to assign running VMs to individual system displays so if you have three instances of FreeDOS/Probe and three instances of WinXP/Bonito, Spectrum Commander and whatnot all get their own dedicated display - assuming your video cards can support this many monitors. The drawbacks of a Type 2: Many limit the number and type of resources which can be assigned to any given VM and almost all of them don't allow PCI pass-through. So forget about using that neato capture card in a VM, right?

Last, the challenge of many popular radio-control software packages - and then into the meat and potatoes of the project:

The PC I'm composing this thread on has a Vin11 Pro VM running Spectrum Commander to control a couple scanners in my upstairs office. I have several more connected to the computer but the display will get...busy...with that many panels if I fire up additional radios. Additionally, you can't launch multiple instances of the program on the same OS. This holds true for a number of other programs, with ProScan being a notable exception. Then there's the issue of CPU and resource loading. I have 16GB and 2 CPUs assigned to the Win11 VM and that's doable but if I fired up another VM for the other scanners the system comes to a crawl.

What's needed is a server or high core count workstation with a lot of RAM, PCIe slots and disk space to handle concurrent VMs of any appreciable volume. Following is something of a collection of op notes and how-tos when building such a configuration.

Taking stock of the server, and some general rules of thumb:

I was fortunate to get a system (Dell R810) with 512GB RAM and all four CPU sockets populated. Of course I wanted as many cores as I possibly could fit in the unit so off to eBay I went. Ten 10-core v3 Xeons were shipped for the princely sum of $30. By the way, they cost $5k EACH when new. It now has 40 cores, 80 threads and all run at 2.4GHz. Slower than lots of newer systems but a lot cheaper for a proof-of-concept build.

The next item was updating the various firmwares including BIOS, iDRAC (Dell's Remote Access Controller) and swapping out the existing H310 RAID card for a higher performance one. This system supports an H700 so one was obtained along with cables. Drives were upgrated from 1TB 7200RPM SATA to 1.8TB 10k RPM SAS types - even though we don't want to boot from RAID.

Two things to keep in mind with servers and workstations of this vintage: Many do not support NVMe boot and many only recognize one NVMe drive at a time, installed via PCIe adapter. Which isn't going to work if lots of fast drives are needed. Lastly, the optical SATA drive used in these systems isn't a "true" SATA drive in there is no +12v supplied on the drive cable. Let's tackle the initial boot problem first then on to more fast storage.