One of the things I hate in the PHP Build System is that Makefile‘s it generates cannot handle the cases when a header file changes. In a perfect world, any change to a header file would cause all files depending on it to be rebuilt.

Here is an example to demonstrate what I mean:

PHP_ARG_ENABLE(test, whether to enable test extension [ --enable-test  Enable test extension])
if test "$PHP_TEST" = "yes"; then
    AC_DEFINE([HAVE_TEST], [1], [""])
    PHP_NEW_EXTENSION([test], [test.c], $ext_shared,, [-Wall])
    PHP_SUBST(TEST_SHARED_LIBADD)
fi

#ifndef PHP_TEST_H
#define PHP_TEST_H

#endif

#include "php_test.h"

Now if we run phpize && ./configure && make, this code will build successfully (yet the shared library cannot be used as a PHP extension, but it does not really matters):

/bin/bash /tmp/test/libtool --mode=compile cc -Wall -I. -I/tmp/test -DPHP_ATOM_INC -I/tmp/test/include -I/tmp/test/main -I/tmp/test -I/usr/include/php/20160303 -I/usr/include/php/20160303/main -I/usr/include/php/20160303/TSRM -I/usr/include/php/20160303/Zend -I/usr/include/php/20160303/ext -I/usr/include/php/20160303/ext/date/lib  -DHAVE_CONFIG_H  -g -O2   -c /tmp/test/test.c -o test.lo
libtool: compile:  cc -Wall -I. -I/tmp/test -DPHP_ATOM_INC -I/tmp/test/include -I/tmp/test/main -I/tmp/test -I/usr/include/php/20160303 -I/usr/include/php/20160303/main -I/usr/include/php/20160303/TSRM -I/usr/include/php/20160303/Zend -I/usr/include/php/20160303/ext -I/usr/include/php/20160303/ext/date/lib -DHAVE_CONFIG_H -g -O2 -c /tmp/test/test.c  -fPIC -DPIC -o .libs/test.o
/bin/bash /tmp/test/libtool --mode=link cc -DPHP_ATOM_INC -I/tmp/test/include -I/tmp/test/main -I/tmp/test -I/usr/include/php/20160303 -I/usr/include/php/20160303/main -I/usr/include/php/20160303/TSRM -I/usr/include/php/20160303/Zend -I/usr/include/php/20160303/ext -I/usr/include/php/20160303/ext/date/lib  -DHAVE_CONFIG_H  -g -O2   -o test.la -export-dynamic -avoid-version -prefer-pic -module -rpath /tmp/test/modules  test.lo
libtool: link: cc -shared  -fPIC -DPIC  .libs/test.o    -g -O2   -Wl,-soname -Wl,test.so -o .libs/test.so
libtool: link: ( cd ".libs" && rm -f "test.la" && ln -s "../test.la" "test.la" )
/bin/bash /tmp/test/libtool --mode=install cp ./test.la /tmp/test/modules

OK, now let us change something in php_test.h:

#ifndef PHP_TEST_H
#define PHP_TEST_H

#ifdef HAVE_CONFIG_H
#include "config.h"
#endif

#endif

Now run make:

$ make

Build complete.
Don't forget to run 'make test'.

You can see that our changes to php_config.h went undetected.

What implications does this have for a developer?

Consider you have a structure used by several functions of your extension. One day you decide that char is too narrow for a field, and you decide to change it to int. Or you want to reorder the fields in the structure to optimize its size. You modify the definition in the header file and run make. In a perfect world you would expect that all source files depending on that header file would be rebuilt. But with the PHP build system this is not the case. And if this goes unnoticed, your extension can suddenly crash or produce weird results. If you don’t rebuild it from scratch, you may end up spending many time debugging the issue and wondering WTF is going on.

A possible solution is to rebuild everything once you change a header.

Another solution is to add some code to Makefile to generate dependencies automatically.

GCC and clang can generate dependencies automatically, and we can rely upon this functionality.

First of all, we need to get the list of the source files. PHP Build System uses shared_objects_test (where test is the name of the extension; the name of the extension is stored in the PHP_PECL_EXTENSION Makefile variable) variable with the list of .lo files (libtool Objects); each .lo file corresponds to a source file. Dependencies of a source file will be put in the file with the same name but with a different extension (.d).

SOURCE_DEPS = $(patsubst %.lo,%.d,$(shared_objects_$(PHP_PECL_EXTENSION)))

Then we need a rule to generate dependency files from source files:

%.d : %.c
	$(CC) -MM -MP -MF"$@" -MT"$(@:%.d=%.lo)" -MT"$@" $(COMMON_FLAGS) "$<"

%.d : %.cpp
	$(CXX) -MM -MP -MF"$@" -MT"$(@:%.d=%.lo)" -MT"$@" $(COMMON_FLAGS) "$<"
  • -MM tells the preprocessor to output a rule suitable for make describing the dependencies of the main source file; note that -MM does not mention header files that are found in system header directories, nor header files that are included, directly or indirectly, from such a header. This is to reduce the size of the dependency file, but should you need the full list of dependencies, you can use -M instead.
  • -MP instructs the preprocessor to add a phony target for each dependency other than the main file, causing each to depend on nothing. This deals with the situation when you delete header files.
  • -MF specifies a file to write the dependencies to.
  • -MT changes the target of the rule emitted by dependency generation. libtool puts .o files to .libs directory, but writes a .lo file next to the source file. The first -MT says that the target should be the .lo file instead of .o (both .lo and .d have the same name and differ only with extension). The second -MT adds the generated .d file to the list of dependencies: if the list of dependencies for a file changes, the file needs to be rebuilt.

Next, we need to tell make to use the generated dependencies:

ifeq (,$(findstring clean,$(MAKECMDGOALS)))
-include $(SOURCE_DEPS)
endif

If make target does not have clean (i.e., make clean, make distclean, etc), we include the generated dependency files, but do not fail if no files exist (e.g., the very first run).

Finally, we need a rule to clean up all .d files. For that we can use either the PHP approach (delete all *.d files globally) or just remove .d files we know of. Both approaches have their pros and cons: PHP’s one could remove someone else’s file (say, you have a vendor directory with third party files), removal of known .d files may leave .d files from other builds (for example, you have multiple branches in your repository; if one of the branches has more sources files, then when you switch to another branch and run make before make clean, .d files for those extra files will not be removed).

clean: clean-deps

clean-deps:
	find . -name \*.d | xargs rm -f

# OR, if you want to exclude some directories:
#clean-deps:
#	find . -type f -name \*.d -not -path './vendor/*' -not -path './tests/*' | xargs rm -f

# OR:
#clean-deps:
#	-rm -f $(SOURCE_DEPS)

However, this would be a bad solution if you needed to edit your Makefile manually every time you run configure.

We save the code above to Makefile.frag.deps:

SOURCE_DEPS = $(patsubst %.lo,%.d,$(shared_objects_$(PHP_PECL_EXTENSION)))

clean: clean-deps

clean-deps:
	find . -type f -name \*.d -not -path './vendor/*' -not -path './tests/*' | xargs rm -f

%.d : %.c
	$(CC) -MM -MP -MF"$@" -MT"$(@:%.d=%.lo)" -MT"$@" $(COMMON_FLAGS) "$<"

%.d : %.cpp
	$(CXX) -MM -MP -MF"$@" -MT"$(@:%.d=%.lo)" -MT"$@" $(COMMON_FLAGS) "$<"

ifeq (,$(findstring clean,$(MAKECMDGOALS)))
-include $(SOURCE_DEPS)
endif

and add the following code to our config.m4 (right after PHP_SUBST):

    if test "$GCC" = "yes"; then
        PHP_ADD_MAKEFILE_FRAGMENT([Makefile.frag.deps])
    fi

if test "$GCC" = "yes" is used to detect a GCC-compatible compiler (clang will also have GCC=yes), and we use this check to filter out compilers which don’t understand -M options for dependency generation. PHP_ADD_MAKEFILE_FRAGMENT adds our Makefile.frag.deps into the resulting Makefile.

Now we need to run phpize && ./configure && make -B and enjoy the proper dependency handling.

Update: here is another interesting article, but I have not tried those rules. Si fractum non sit, noli id reficere, after all 🙂

PHP Extensions: How to Make make Rebuild Dependencies Correctly
Tagged on:         

Leave a Reply

Your email address will not be published. Required fields are marked *