Hi chod and welcome to our forum!
I’d say they’re fairly similar. VirtualBox is completely free so that’s a great benefit and VMware has a free version (VMware Player) and a paid version (VMware Workstation). The free version is a bit cut in features in comparison to the paid version. Generally, VMware is the more professional product though, that is getting used by many companies so bug fixes etc. will happen there often. The biggest problem is IMO that the free version doesn’t support snapshots (correct me guys if I’m wrong), meaning you cannot save a clean state of the virtual machine and then reset it to that state once the scammer messed everything up and you want to start a new bait. If you find maybe a cheap key for VMware Workstation on ebay or so, that’s a good choice.
I’m using VMware personally and created some pre-made and disguised virtual machines for scambaiting so if you want to directly start scambaiting, check this out: