Page 1 of 1

Unit testing bugs and issues

PostPosted: November 29th, 2017, 8:13 pm
by emsmith
We have solutions with multiple PHP Projects (internal libraries, etc)

But you can have only one active phpunit.xml file active at a time for unit testing. This is suboptimal, since the different projects have isolated test suites with their own requirements, vendor directories, bootloaders... Each project should be able to have it's phpunit configuration file set and used for those tests. So you can't run all, or even run a set in a solution without going back and changing the active phpunit.xml config file. This is annoying.

Also devsense requires their own specialized phpunit.xml files, they don't recognize perfectly valid phpunit xml syntax

[11/29/2017 3:09:03 PM Warning] Cannot read the configuration file 'C:\xxxx\tests\phpunit.xml'.Please make sure the file has the correct format.

Code: Select all
<phpunit
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/4.5/phpunit.xsd"
         bootstrap="../vendor/autoload.php"
         colors="always"
         verbose="true"
         beStrickAboutCoversAnnotation="true"
         beStrictAboutOutputDuringTests="true"
         beStrictAboutTestsThatDoNotTestAnything="true">

  <testsuites>
    <testsuite name="Unit Tests">
      <directory>./Unit</directory>
    </testsuite>
    <testsuite name="Integration Tests">
      <directory>./Integration</directory>
    </testsuite>
  </testsuites>
</phpunit>



Also if the bracketed namespace syntax is used in a test file

http://php.net/manual/en/language.names ... ltiple.php

It throws a wonderful error

[11/29/2017 3:00:07 PM Error] Internal error has occured with following information: System.ArgumentNullException: Value cannot be null.
Parameter name: source
at Microsoft.VisualStudio.TestPlatform.ObjectModel.ValidateArg.NotNullOrEmpty(String arg, String parameterName)
at Microsoft.VisualStudio.TestPlatform.ObjectModel.TestCase..ctor(String fullyQualifiedName, Uri executorUri, String source)
at PHP.VisualStudio.TestAdapter.TestExplorer.PhpUnitOutputParser.HandleOutputLine()
at PHP.VisualStudio.TestAdapter.TestExplorer.PhpUnitOutputParser.Parse()
at PHP.VisualStudio.TestAdapter.TestExplorer.PhpTestExecutor.RunTestsInternal(IEnumerable`1 tests, IRunContext runContext, IFrameworkHandle frameworkHandle, Boolean filter)
at PHP.VisualStudio.TestAdapter.TestExplorer.PhpTestExecutor.RunTests(IEnumerable`1 tests, IRunContext runContext, IFrameworkHandle frameworkHandle, Boolean filter)
[11/29/2017 3:00:07 PM Error] An exception occurred while invoking executor 'executor://php-tools/': Value cannot be null.
Parameter name: source


Although I'd love to use the integrated unit testing features, instead I'm back to powershell scripting to run tests on a regular basis

Re: Unit testing bugs and issues

PostPosted: December 4th, 2017, 4:37 pm
by Miloslav Beno
Hello,

Thank you very much for great feedback!

Your suggestion to allowe phpunit.xml per project makes perfect sense. We will do it. I will inform you about the progress in this thread.

About the issues you've described:

- Cannot read the configuration file...

We are too strict here and verifing phpunit.xml with the xsd file. Unfortunatelly phpunit doesn't follow their scheme so it really doesn't make any sense for us to verify it. e.g. colors is defined as boolean in the scheme, but values can be "never", "auto" or "always" according to the manual.

We will fix that

- Bracketed namespace syntax is used in a test file

Could you please send us a file that causes this?

Thank you so much,