Open Source Project

As previously mentioned in Isabelle’s Post, onOffice migrated from PHP Version 7.2 to 8.1.

The problem: https://www.php.net/manual/en/migration80.incompatible.php

Comparison

However, this list is not exhaustive! Apart from the operators listed, there are:

  • more type-unsafe operators, such as >, <, >=, <=, <=>, !=
  • functions which perform type-unsafe comparisons in a certain (default) case, such as in_array(), array_keys(), array_search()
  • and the good ol’ switch/case statement.

Our first reaction:reaction.gif

The question we asked ourselves: how do we develop in a way that is compatible with PHP 7 and 8 at the same time?
Because of the huge number of affected users and services, in both a stable and a beta environment, the switch can’t happen all at once.

Our codebase is several MILLION LoC big and ad-hoc refactoring the type-unsafe comparisons into type-safe ones is out of the question! The risk of breaking anything is way too big.
But how do we uncover potential changes in behavior between the two major versions? How can you ever be sure that everything acts the same way after your change? Expecting type-unsafe operators to behave the same way even after breaking changes is unreasonable, and just hoping that nothing changes and relying on luck is not a sound strategy for business-critical code. We needed some kind of detection mechanism for changes in behavior to protect our customers — we don’t want to bring the real estate industry to a halt just because we decide to update our PHP version!

We talked to Stefan Priebsch of thePHP.cc and came to the conclusion that we should create a polyfill that mimics the old behavior of PHP 7. When we asked what the consultation would cost, he said the time was free of charge as a gift to the PHP community, if we open source our solution. And here we are.

But hold on, it doesn’t stop with the polyfill! Instead of replacing all type-unsafe operators and leaving it there to rot, we created wrappers that notify us if the operands WOULD lead to a difference in PHP 8 if the operator hadn’t been replaced beforehand. Once we got a notification, we replaced such occurrences with their equivalent method from our collection of polyfills. Each occurrence we didn’t get a notification for could safely be reverted into the original type-unsafe operator. This way we made sure to not change the behavior and not stick to our polyfills forever, at least where it was safe to be removed.

Kudos to our colleague Stefan Becker for the great first prototype of the implementation. The site https://www.exakat.io/en/string-to-number-comparison-with-php-8-x/ has also been a great inspiration.

Technical Overview

Let’s start by explaining the eq functions of Phase1 and Phase2, respectively.

In class Phase1:

public static function eq($a, $b): bool
{
   $native = ($a == $b);
   $computed = StringToNumberComparison::eq($a, $b);

   self::validate($a, $b, $computed, $native);

   return $native;
}

And here is Phase2::eq():

public static function eq($a, $b): bool
{
   $native = ($a == $b);
   $computed = StringToNumberComparison::eq($a, $b);

   self::validate($a, $b, $computed, $native);

   return $computed;
}

You might have noticed the call to StringToNumberComparison::eq(). This class StringToNumberComparison contains the actual code that mimics the behavior of type-unsafe operators the way we know it from PHP 7: any type is allowed as an argument for each of the two parameters and will be compared as if you’d used == in PHP 7. This is the $computed value.

The difference between the two is in the detail: Phase1::eq returns the $native value (the actual) and Phase2 returns $computed. That’s why you’d want to run Phase1 in PHP 7 and Phase2 in PHP 8. If any differences occur, the validate function makes sure we find out! In this case, it will call the report method, which is basically just an error-logger, or, if run in a development environment, throws an exception.

The function c_eq() is shown below

<?php

if (7 === PHP_MAJOR_VERSION) {
    function c_eq($a, $b): bool
    {
        return Phase1::eq($a, $b);
    }
} else {
    function c_eq($a, $b): bool
    {
        return Phase2::eq($a, $b);
    }
}

Some of the replacements for operators include c_eq, c_ne, c_gt, c_gte, c_lt, c_lte, c_spaceship.
These were used as building blocks for polyfills of actual PHP functions c_inArray and c_arraySearch, c_arrayKeys, which are polyfills for in_array(), array_search() and array_keys()if their third parameter (strict check for equality) is unset or set to false.

Since switch/case statements behave as if the switch condition was compared to each case in an unsafe manner, our rector rules turn these into a check with our new c_eq() function.
Take this switch/case as an example:

$input = ' 1e2 ';

switch ($input) {
    case '1e2':
      echo 'a numeric string';
      break;
    case 100:
      echo 'a number';
      break;
    default:
      echo 'something else';
}

// PHP 7 would emit "a number"
// PHP 8 would emit "a numeric string"

After refactoring, the above switch/case statement looks like this:

$input = ' 1e2 ';

switch (true) {
    case c_eq($input, '1e2'):
      echo 'a numeric string';
      break;
    case c_eq($input, 100):
      echo 'a number';
      break;
    default:
      echo 'something else';
}

// this way, both PHP 7 and 8 emit "a number"

How do we know we didn’t make any mistake in our implementation of the polyfills?

We had to replace the affected PHP operators with our compatibility functions first! But how do you replace thousands of operators with replacement functions without burning out?

Rector to the rescue!

Rector eliminates the burden of updating every single line of code for (specific kinds of) systematic refactoring. It’ll parse the codebase into an abstract syntax representation of the input files, apply certain rules to that syntax tree and rebuild it into .php files. This works astonishingly well, for search-and-replace changes that are perhaps too complicated to trust to regex, but are expressable as code.

And that is exactly what we did – we implemented rules that look for a certain PHP node (e.g. the spaceship operator, <=>) in the parsed PHP code and replace it with a different one. That way it was quite simple to replace some operators with their replacement functions. We did the same thing with the affected array functions in_array(), array_keys() and array_search(), as well as the more complex switch/case. Of course, we also implement Rector rules to undo the changes!

As an example, here is the rule that replaces <=> with the c_spaceship() function with the correct arguments:

use PhpParser\Node;
use PhpParser\Node\Arg;
use PhpParser\Node\Expr\BinaryOp\Spaceship;
use PhpParser\Node\Expr\FuncCall;
use PhpParser\Node\Name;
use Rector\Core\Rector\AbstractRector;

class StringComparisonSpaceship extends AbstractRector
{
   // ...
   
    public function getNodeTypes(): array
    {
        return [
            Spaceship::class,
        ];
    }

    public function refactor(Node $node): FuncCall
    {
        return new FuncCall(new Name('c_spaceship'), [new Arg($node->left), new Arg($node->right)]);
    }
}

It’s amazing how easily we can update an entire repository.

As a next step, we thought it was very important to have the newly-refactored codebase run in as many unit-test and integration test cycles as possible to uncover bugs in our implementation before it gets shipped to our production systems. As a reminder: Phase 1 ensures that the new polyfills behave the same way as their native PHP equivalent. We even uncovered one bug in our StringToNumberComparison implementation that way!

Rector comes with a rule to convert switch-statements into match-expressions, but because these aren’t necessarily type safe, this could lead to a change in behavior. Our artisanal Rector rules for the migration are all type safe and we use a PhpStan rule to ensure that type-unsafe cases aren’t built into new code by mistake. This means, we can change them into match-expressions later without any worries.

140k Tests

This whole thing only worked so well because of our test — we covered as many specific cases as possible in a Snapshot-Test for the StringToNumberComparison class. This was possible through running a cross-comparison in several stages:

The expectations file was saved to a file for easy lookup later, while running the PHP 8 tests.
With the combination of both operands and the operator, the StringToNumberComparison test can get the expectation (generated in PHP 7) and compare it to the result during the run in PHP 8.

PHPUnit 8.5.41 by Sebastian Bergmann and contributors.
.                                                                   1 / 1 (100%)
Time: 1.28 seconds, Memory: 78.29 MB
OK (1 test, 143145 assertions)

A whole milestone for one development team, less than 2 seconds for the test system.

Conclusion

“Mom, can we get a match expression?”
“We have a match expression at home!”
The match expression at home: 

switch (true) {
    case $character === 'Spider Man':
       return 'Peter Parker';
}

At onOffice, type-unsafe checks are forbidden in new code. We’d like to point out that the pattern above was the only allowed way to implement a switch/case statement during the migration period, so that they could all easily be replaced with match-expressions as soon as our production systems were switched to PHP 8.

These tools have facilitated the migration of PHP from 7 to 8 enormously for us. And even though PHP 8.4 is out now, we believe there are still a lot of PHP projects out there hesitating to migrate away from PHP 7 because of the fear of breaking anything. We’re hoping to take that fear away from you, so you can enjoy the benefits of PHP 8 soon!

Links

This text was written jointly by Olea Plum and Jakob Jungmann.