Code Sprint in Hamburg - Knocking TYPO3's Localization Into Shape

If you’re a developer, code sprints are the place to go. They’re a get-together of dedicated coders and a great opportunity for meeting like-minded people, for learning, and for making new friends. Lots of work gets done, knowledge is shared, and the days get wrapped up together sometime late at night, which doesn’t leave all too much time for sleeping. All I can say is that it’s a price worth paying! 

Earlier this month, 16 TYPO3 developers met up in Hamburg for a code sprint. The main focus was to make TYPO3’s localization more efficient–one of the major steps towards the next TYPO3 LTS version v9, scheduled to be released in October 2018.

It’s the process by which a product or application is adapted to meet language requirements. It includes translation, and goes beyond it. If you’re doing business beyond your home country, localization provides a key to international success.

TYPO3 CMS has 15+ years of experience in handling multilingual sites.

TYPO3 has the structure it takes to develop complex websites with a wide range of content in various languages whilst remaining user-friendly. Since the beginning, TYPO3 has had a powerful yet simple localization audit system integrated into the core, that helps in translating and managing global content as it is created. This makes TYPO3 CMS a good player in the global marketplace. You can launch your product in as many languages as you need with one single installation.

The basic structure of handling languages is pretty straightforward. The default language gets defined, and all translations are subsets of this parent record. When a record is requested in a language other than the default language, the default language record gets fetched first, and is then overlaid with the language specific record. All language related information is gathered and processed in this way.

In TYPO3, this structure evolved in the course of time. And over the years, other layers were integrated into the system that hindered the overlay process for these records. A clear call for action! Cleaning this up was marked as top priority by the core team, the TYPO3 roadmap got defined accordingly, and the code sprint in Hamburg was scheduled for September 7-10, 2017.

16 developers came to Hamburg to work together at this event. We were kindly hosted by +sitegeist media solutions GmbH and had the pleasure of using their truly beautiful conference room.

The main focus of the TYPO3 code sprint in Hamburg: making a good localization even better!

1. Localization: Removed table
   Due to various hard coded parts in the core, the feature ‘translations in own table’ only ever worked for pages. Therefore, the long-term plan is to remove the whole feature from the core.
   In preparation for this major step, the TCA setting had been removed a while ago.
   Further steps in this direction were to now:
   > install an Upgrade Wizard (to migrate records from pages_language_overlay into pages) and
   > install a Database Wrapper (to handle queries with target pages_language_overlay and rewrite them to fetch the data from pages).
   There are still a few open ends here, but when completed, this will serve as a backwards compatibility layer for extensions.
   Besides removing the table, the translated records won’t be overlay records anymore. They’ll be fully synchronized records that can be used without their parent record. This is in accordance with the latest development throughout the core.

2. Localization: Extbase
   Extbase got special attention. The process was streamlined with the defined core behaviour in order to make it easier to develop extensions and is now ready for v9.

3. Localization: Frontend rendering
   Removing the overlay processing for translated record processing was also one of our tasks in Hamburg. We were able to merge most of the regarding patches, there are only a few that still need attention.

4. Ongoing improvement: backend UX
   The improvement of TYPO3’s backend User Experience (UX) is an important part of the current roadmap. For example, patches in the area of icon processing have been merged to the core.

5. Ongoing improvement: install tool
   The install tool in TYPO3’s backend needs restructuring and we worked hard on this. A few patches are still in need of a review and once these have been merged into the system, this task will be completed. We aim at doing this as soon as possible.

6. ExtJS removal
   This task has been on the to do list for quite a while now and two major LTS versions have been released in the meantime. Now–finally–we are almost finished with replacing the last usage sector, which is the backend page tree.

The first day was for saying our hellos, setting up our gear, and distributing the work at hand. We did this during the morning, with breakfast and coffee, in the nice sitting area in our host’s conference room. Round midday, when everybody had arrived, we started gathering interest groups and put up our agenda. We were able to get our hands on our keyboards in less than an hour, and the first patches were pushed to the review system not long afterwards.

A second code sprint was taking place at the same time at Tritum GmbH. There, a team was working on giving ext:form a new blast of features and improvements. Both teams helped each other out in terms of reviews.

As usual, we had pizza delivered on site for dinner and a big thank you goes to TeamWFP and Benji Kott for sponsoring this meal. Due to the fact that some of the participants had been on the road as early as 4am, we finished off the day relatively early, at 11pm.

It was almost 10am before we were ready to go. But then everybody was deeply concentrated and coding away to their heart’s content. Reviewing patches had to wait. The topics were the same as the day before, and our sprint board started showing that we were progressing nicely.

During the day, nothing noteworthy happened (which is actually a good thing), except that we noticed a higher demand for sweets. The good news? The supermarket was just around the corner. The bad news? Well, we all know where (or better: what) snacking leads to. :-)

Friday evening was our night out. We went to a restaurant nearby and enjoyed some hours of eating and talking. At some point we had to decide what to do with the rest of the evening. Some of us went downtown to do some sightseeing, rain notwithstanding. And others went back to the office to do more reviewing and merging.

Altogether, it was a day with loads of merges, and they were by far not all no-brainers! For example, now the tight coupling of the log- and history table that used to provide the rollback function for records is a thing of the past. Yay! Now there is a log, and there is a history, each nicely in their own place and with their own functionality and responsibility.

The revamped install tool also received further attention. There’re still quite a few smaller issues that need to be solved, and more popped up as we proceeded.

Some preparation patches also made it. They will serve as a basis for refactoring and restyling e.g. the file list module. Stay tuned for the changes to come, they will be visible and enhance the user experience.

One of the larger chores also came quite some way: the removal of ExtJS.

Finally, at 1:30am in the morning, we grabbed a bit of sleep.

This day started off pretty quiet, and again rather late. Breakfast was on, so was coffee. And this gave us the head start we needed to launch into another demanding day of coding.

Taking on loose ends was the topic of the day, in order to get as much finished as possible, before leaving the next day.

For me, Saturday’s highlight was the patch that streamlines extbases language handling with the behaviour provided by TypoScript. Here, tests weren’t quite finished, but once this makes it into mainline, integrators will no longer suffer from the inconsistent overlay processing.

The removal of the translation table feature also made good progress. This feature never ever worked for any other table than pages. Thanks to this “special” treatment, we have a whole lot of conditions everywhere in the core. Once this feature has been removed completely, we will remove all those conditions related to the database table resolving. It will enhance readability and maintainability of the most important classes in the core, and will therefore also come with a new boost in performance.

As for dinner: all the sayings about developers running on pizza and coffee are true! And in house delivery gave us extra time for coding. Fortunately +sitegeist’s kitchen is big enough to fit us all. So, yes, pizza managed to lure us away from our laptops for a short while and we enjoyed both the meal and each other’s company. Connecting with each other and learning about plans and dreams is as much a part of code sprints as the actual coding work itself. We form bonds and friendships that carry us much longer than the actual event lasts.

This is the unique spirit of TYPO3, where we feel inspired to share - both pizza and knowledge.

The last day of this highly efficient event. When I first wrote these lines, which was around 9am that morning, we already had 41 tickets marked “done” on our sprint board! And together with the team of coders in Jena, we had been able to merge altogether 89 patches since Thursday–a pretty decent result for three days!

Interested in numbers and colorful diagrams? We tracked all of our activity here: https://forger.typo3.org/.

For most of us, the day was shorter than the ones before. We had to travel home, as a new working week was beginning the next day. In the end, we managed to merge altogether more than 100 patches into the TYPO3 branches. Also, the next steps were planned and we all took “to dos” home with us.

Events like this wouldn’t be possible without the generous support of the TYPO3 Association, TYPO3’s Community and sponsors.

We would like to thank:

• our host +sitegeist media solutions GmbH for providing such a great location
• the employers of all 16 participants, who enabled people to take some time off work in order to travel to Hamburg
• TeamWFP for sponsoring our pizza
• the community for liking our social media activity and cheering us on :-)
• all 16 participants, who gave their time and their effort ... and their sleep!