Here's the configuration that ultimately worked, with no fiddling of any configuration files or changes to my symfony code of any kind.

The relevant part of the Path to phpunit.phar is: <your-project-directory>/bin/.phpunit/phpunit-6.5/phpunit
You'll receive something along the lines of
Fatal error: Class 'Symfony\Bundle\FrameworkBundle\Test\KernelTestCase' not found
when running your unit tests if you haven't checked the 'Default configuration file' box.