Testing TYPO3’s Core - Part VIII: Future

It’s not that easy to pinpoint precisely how much time and effort is required in total for developing and maintaining software. But it’s definitely the latter - namely keeping the software up-to-date - which is the bigger bulk of the two. Our process of developing all of the workflow patterns included today’s TYPO3’s review process is a wild and intensive story in itself.

Integrating the various different testing levels and running them continuously not only improved the codebase, it also made us better programmers and improved the stability of TYPO3 as a powerhouse content management system.

In the future we aim to achieve more of the same. Join me now, for taking a glimpse into our freshly polished crystal ball. Let’s take a peek at what we strive to achieve in the future.

Procedures like executing unit tests are well-known to most core contributors nowadays, yet there are more fancy parts of the system which aren’t and this certainly needs changing. In addition, these areas are hardly documented at the moment. There are a few wiki pages about these subjects but they don’t really cut it. We badly need much better documentation to help contributors write, execute and maintain tests within TYPO3’s core. This has to improve ... pretty darn quick!

This one’s easy. We can always do with more tests. As patch reviews are part of our workflow process, everyone contributing towards TYPO3 keeps an eye on tests and requests additional ones if feasible. Yet the core codebase does still have rough edges with huge monster files leading to hard-to-test areas and these areas need to be restructured. Refactoring will lead to higher code coverage, stabilized code, better re-use capabilities, separated concerns and improved readability. In short: all of the good results that better testing lead to!

When establishing the test infrastructure, we started off with unit tests. We can now look back on quite some years of experience and we’ve learnt a lot through the fundamental method of problem solving, namely trial and error. As a result, we now have various well-functioning tests in the core. But things do change in the course of time and there are areas that could be handled differently nowadays. These parts should be improved. Some unit tests can just be removed (which always hurts somehow) and others can be changed from unit to functional tests (and thus become much more useful).

With the younger testing areas like the acceptance tests, there is also room to add some more click-through tests to the backend. This area is a “playground of its own sort”. It’s lots of fun to hack on and it’s a good area for new contributors to work on. Highly appreciated!

Alas, some of our tests are still rather slow. The main reason is that TYPO3’s core still “pollutes” the global state and this can’t be reset in a simple way. This affects the functional tests in particular. As things are now, we do process forking to work around this issue which leads to additional costs. The “state issue” is not only ugly for the functional tests, it’s also a nightmare for extension developers. For instance, as it is now, creating a proper website frontend link from within a command line or backend context is a tedious task. The system does not provide a good solution for that, yet.

This is one of the changes the core contributors aim at solving. Creating a fully encapsulated state that can be properly frozen, reset and recreated would be an awesome solution. It would not only increase the current functional test speed, it could also lead to a much better PSR-7 integration and would resolve lots of issues extension developers face every day. A big win in all areas, if done right.

It seems vendors have finally started looking at better “headless” support in their browsers. A headless variant of Chrome has recently been released. We’re keeping a close eye on it to see whether it’s stable enough for us to be integrated into our workflows. Firefox seems to work on something similar. Both is great news. Everything that happens in this area will simplify the setup and thus raise developer acceptance.

There’s a further issue we’ve been aware of for quite a while already, namely that the installation setup of the TYPO3 system clearly needs improvement. This has to run more smoothly and also has be more user-friendly. If the installation of the project is not successful right from the beginning, new users are overwhelmed and discouraged. And some just leave, which harms TYPO3’s reputation, of course. This part is not tested often by contributors in their daily work. It’s sometimes only shortly before new versions are to be released that the core team realizes that parts of the installation routine are broken. This hurts, and it hurts bad. It also puts a lot of extra pressure onto the core team at a point in time when they have a lot on their plates anyway as delivery deadlines with regards to LTS releases are always guaranteed and met. It’s important that things run smoothly in the final phase. Therefore, we recently added a basic acceptance test coverage of the initial TYPO3 installation process which will be further improved as it would benefit from additional tests.

With the rewrite of the TYPO3 core towards the database API doctrine, the project is able to support more and more data management systems (=DBMS), such as MySQL, postgreSQL, and mssql, to name but a few. Establishing support for another database basically means to first get the functional tests green on this platform. Once that’s done, the backend and installation process will work pretty much okay and just a few further manual tests are needed to verify that the system is healthy on a given DBMS platform.

At the moment, the team is working on Microsoft SQL server support. Although the current state is still far from good, we have meanwhile reached the “green bar feeling” of the functional tests. There are still issues and not the entire test suite goes green. Reaching that goal will make TYPO3 run much more smoothly on the Microsoft Azure cloud. And the tests will never go red after they’ve been added to the mainline tests. Promised.

Another DBMS will need work afterwards, we’re pretty sure about that ...

While we’re at it, Windows is still a headache when it comes to hosting TYPO3 in a good way. It works, but it requires some additional knowledge that is sparsely documented. Executing parts of the testing stack on the Microsoft web server IIS and further native components could help to better support this platform which is becoming more and more important.

TYPO3 core testing has come a long way. Compared to other open source systems - and yes, compared to many closed source systems, too - we’re in a pretty good shape. We’ve given you a glimpse behind the scenes with this series about testing TYPO3 and have also shown what testing can really mean to a project. We hope you had some fun reading all of this and can benefit from the knowledge, procedures and ideas we’ve outlined.

We may continue this series sometime further along the line. For instance, one topic still missing is how extension authors can benefit from the test infrastructure. This is possible and extensions like news show how it can be done. If you come across any topics you’d like to know more about then don’t hesitate to contact us.

Until then: May the test be with you!

You can find answers to frequently asked technical questions, instructional clips and all sorts of helpful videos on our TYPO3 YouTube channel. Come along and join us there. If you find this useful, please share the link on your network.