Paul M. Jones

Don't listen to the crowd, they say "jump."

Package Development Standards: "pds/skeleton" Now Open For Review!

The new Package Development Standards initiative is proud to present its first publication, pds/skeleton (and the related research) for public review. If you are a package author, you are invited to post your comments and criticisms of the publication as issues on the relevant Github repository.

The pds/skeleton publication describes a set of standard top-level PHP package directories and files. If you are an author of more than three packages on Packagist, chances are you already follow the standard! That's because PDS initiative researches the PHP package ecosystem to recognize commonly adopted development practices. (See this list of over than 47,000 packages; if your package is there, it has been tentatively noted as already compliant.)

The public review period will last at least 2 weeks, perhaps longer. If there are no substantial errors revealed during that time, the standard will advance to "beta" status for adoption, and then to "stable" if no further errors are discovered in use.

Meanwhile, if you review the publication and determine that your package follows it, you should add pds/skeleton to your Composer "require-dev" block, because that will help the initiative track formal adoption rates.


The Iron Law of (Open Source) Bureaucracy

Something to watch out for in open-source projects: Pournelle's "Iron Law of Bureaucracy." In the organization around your project, you will find two kinds of people:

  1. Those who concentrate on the goals of the organization. These are generally coders, devops, and infrastructure folks.
  2. Those who concentrate on the organization itself. These are generally managers of people within the organization, and controllers of how & when the organization communicates within itself and outside itself.

Pournelle's Iron Law asserts that group 2 will always end up in control of the organization.

Police your membership accordingly.


PHP file_get_contents() HTTPS/SSL error on Mac OS

Problem on Mac OS Sierra (10.12.x): Using file_get_contents() for an https resource fails with error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed.

This solution appears to be legit, although I am not primarily a security guy, so real security folks should chime in if they seem something untoward:

% sudo mkdir -p /usr/local/libressl/etc/ssl/certs
% sudo curl -o /usr/local/libressl/etc/ssl/cert.pem https://curl.haxx.se/ca/cacert.pem

This creates the default_cert_file directories and imports the latest certificate store from curl.haxx.se.

Source: PHP on macOS Sierra can’t access SSL data – andrewyager.com


Independent Packages and Subtree Splits

You’ll sometimes see a PHP package hosted in a Github repository with the heading or subtitle “[READ ONLY] Subtree Split”. This indicates that the package is actually copied from another codebase (usually a framework) and is not intended to be worked on separately from that other codebase.

As such, a “subtree split” is not necessarily a sign of an independent package. Using a subtree split to publish a package says to me that the authors’ concentration is on the framework-as-a-whole, not on the package in-and-of- itself.

For example, Symfony does subtree splits for its components, as does Laravel for its Illuminate components. Those packages from the frameworks are not developed independently; they only move forward as part of the whole framework.

In these cases, you often end up with composer.json in the framework origin directories, which is not something I generally expect. Further, the framework subdirectories may have their own src/tests/docs/etc. directories. They are there so that the subtree split can have them available at their own top level, but in the origin framework, it is again something I find unexpected.

I say: if you’re going to advertise independent packages, actually write them independently. Let them be their own thing. Aura has done it that way since its beginning, and Zend Framework converted to that approach in version 3. Then you can compose the truly independent packages into a framework, instead of subtree-splitting your framework into pseudo-independent packages that are still bound to the origin framework development and release process.


PHP ssh2.sftp opendir/readdir fix

This bug https://bugs.php.net/bug.php?id=73597 related to the PECL ssh2 extension bit us yesterday, so this post is a public service announcement that will (hopefully) save you from writing your own workaround like I almost did.

Problem: PHP 5.6.28 (and apparently 7.0.13) introduced a security fix to URL parsing, that caused the string interpolation of the $sftp resource handle to no-longer be recognized as a valid URL. In turn, that causes opendir(), readdir(), etc. to fail when you use an $sftp resource in the path string, after an upgrade to one of those PHP versions.

Solution: Instead of using "ssh2.sftp://$sftp" as a stream path, convert $sftp to an integer like so: "ssh2.sftp://" . intval($sftp) . "/". Then it will work just fine.

Thanks to the people who fixed the URL parsing security flaw, thanks to the people who wrote the PECL ssh2 extension, and thanks to the people who provided the fix.


Conserving On The Wrong Resource

(This is a blog post I’ve had in a "write-me!" folder for years; I am now publishing it in its unfinished form to get it off my mind.)

Programmers are acutely aware of their limited resources: CPU, memory, storage, requests-per-second, screen space, line length, and so on. If programmers are aware of a resource constraint, they conserve on it; sometimes they call this “optimizing.”

Are you aware of a loop? Then you try to “optimize” it, or conserve on it, e.g. by not using a count() in the loop declaration.

Are you aware of the number of lines? You might try to conserve on the number of lines, perhaps by removing blank lines (especially in docblocks).

Are you aware of aware of the size of an array, or the number of objects in the system, and so on? You try to conserve on those, to reduce their number.

But this is sometimes a trap, because you conserve or optimize on a low value
resource you are aware of, at the cost of not conserving on a high-value resource you are not aware of. For example, optimizing a for loop by not using count() in its declaration will have little impact if the loop is executing queries against an SQL database.

Lesson: conserve on high-value resources that you may not be aware of, not low-value ones that are obvious but of little consequence.


What the "Anti-Federalists" Got Right

First, the Anti-Federalists predicted that the federal government the Constitution created would increase its power until states were merely secondary considerations. Second, that this powerful federal government would be too big and distant for the people to effectively control, and thus vainglorious men of ambition and avarice would control it to enrich themselves off the common people.Third, that it was unwise to govern a diverse and large population from a remote and distant government that would hold such immense power over the daily lives of common people, and, as a result, politics would become something more primitive than civilized, characterized by constant discord and fighting as different factions tried to control the lives of their fellow citizens.

Source: The Anti-Federalists Predicted Today’s Political Morass, & Can Get Us Out


Faster Evolution Means More Ethnic Differences

Russian scientists showed in the 1990s that a strong selection pressure (picking out and breeding only the tamest fox pups in each generation) created what was -- in behavior as well as body -- essentially a new species in just 30 generations. That would correspond to about 750 years for humans. Humans may never have experienced such a strong selection pressure for such a long period, but they surely experienced many weaker selection pressures that lasted far longer, and for which some heritable personality traits were more adaptive than others. It stands to reason that local populations (not continent-wide "races") adapted to local circumstances by a process known as "co-evolution" in which genes and cultural elements change over time and mutually influence each other.

Source: Edge.org


The PHP 7 "Request" Extension

tl;dr: The new request extension provides server-side request and response objects for PHP 7. Use it as a replacement object for request superglobals and response functions. An equivalent PHP 5 version is available in userland as pmjones/request.


You're tired of dealing with the $_GET, $_POST, etc. superglobals in your PHP 7 application. You wish $_FILES was easer to deal with. You'd prefer to wrap them all in an object to pass around to your class methods, so they'd be easier to test. And as long as they're all in an object, it might be nice to have parsed representations of the various incoming headers. Preferably, you'd like for it to be read-only, so that the various libraries you use can't modify superglobal state out from under you.

Likewise, seeing the “Cannot modify header information - headers already sent” warning is getting on your nerves. You'd like to get away from using header(), setcookie(), and the rest. You know they're hard to inspect, and they're hard to test. What would be great is to have all that response work wrapped in another object that you can pass around, and inspect or modify it before sending the response back to the client.

You could maybe adopt a framework, but why do that for your custom project? Just a pair of server-side request and response objects would make your life so much easer. Why can't there be set of internal PHP classes for that?

Well, now there is. You can install the request extension from John Boehr and myself to get ServerRequest and ServerReponse objects as if PHP itself provided them.

ServerRequest

After you install the extension, you can issue $request = new ServerRequest(), and then:

Instead of ...                          ... use:
--------------------------------------- ---------------------------------------
$_COOKIE                                $request->cookie
$_ENV                                   $request->env
$_GET                                   $request->get
$_FILES                                 $request->files
$_POST                                  $request->post
$_SERVER                                $request->server
$_SERVER['REQUEST_METHOD']              $request->method
$_SERVER['HTTP_HEADER_NAME']            $request->headers['header-name']
file_get_contents('php://input')        $request->content
$_SERVER['HTTP_CONTENT_LENGTH']         $request->contentLength
$_SERVER['HTTP_CONTENT_MD5']            $request->contentMd5
$_SERVER['PHP_AUTH_PW']                 $request->authPw
$_SERVER['PHP_AUTH_TYPE']               $request->authType
$_SERVER['PHP_AUTH_USER']               $request->authUser

Likewise:

Instead of parsing ...                  ... use:
--------------------------------------- ---------------------------------------
isset($_SERVER['key'])                  $request->server['key'] ?? 'default'
  ? $_SERVER['key']                     (good for all superglobals)
  : 'default';
$_FILES to look more like $_POST        $request->uploads
$_SERVER['HTTP_CONTENT_TYPE']           $request->contentType and
                                        $request->contentCharset
$_SERVER['HTTP_ACCEPT']                 $request->accept
$_SERVER['HTTP_ACCEPT_CHARSET']         $request->acceptCharset
$_SERVER['HTTP_ACCEPT_ENCODING']        $request->acceptEncoding
$_SERVER['HTTP_ACCEPT_LANGUAGE']        $request->acceptLanguage
$_SERVER['HTTP_X_HTTP_METHOD_OVERRIDE'] $request->method
$_SERVER['PHP_AUTH_DIGEST']             $request->authDigest
$_SERVER['HTTP_X_REQUESTED_WITH']       $request->xhr
  == 'XmlHttpRequest'

Those properties are all read-only, so there's no chance of them being changed without you knowing. (There are some true immutables for application-related values as well; see the documentation.)

ServerResponse

For responses, you can issue $response = new ServerResponse(), and then:

Instead of ...                          ... use:
--------------------------------------- ---------------------------------------
header('Foo: bar', true);               $response->setHeader('Foo', 'bar');
header('Foo: bar', false);              $response->addHeader('Foo', 'bar');
setcookie('foo', 'bar');                $response->setCookie('foo', 'bar');
setrawcookie('foo', 'bar');             $response->setRawCookie('foo', 'bar');
echo $content;                          $response->setContent($content);

You can inspect the $response object, and then call $response->send() to
send it to the client.

Working with JSON?

// instead of ...
header('Content-Type: application/json')
echo json_encode($value);

// .. use:
$response->setContentJson($value);

Sending a $file for download?

// instead of ...
header('Content-Type: application/octet-stream');
header('Content-Transfer-Encoding: binary');
header(
    'Content-Disposition: attachment;filename="'
    . rawurlencode(basename($file)) . '"'
);
$fh = fopen($file, 'rb+');
fpasthru($fh);

// use:
$fh = fopen($file, 'rb+');
$response->setContentDownload($fh, basename($file));

Building a complex header? Pass an array instead of a string:

$response->setHeader('Cache-Control', [
    'public',
    'max-age' => '123',
    's-maxage' => '456',
    'no-cache',
]); // Cache-Control: public, max-age=123, s-maxage=456, no-cache

$response->setHeader('X-Whatever', [
    'foo',
    'bar' => [
        'baz' => 'dib',
        'zim',
        'gir' => 'irk',
    ],
    'qux' => 'quux',
]); // X-Whatever: foo, bar;baz=dib;zim;gir=irk, qux=quux

Find Out More

You can read the rest of the documentation at https://gitlab.com/pmjones/ext-request to discover more convenient functionality. And if you're stuck on PHP 5.x for now, the extension has a userland version installable via Composer as pmjones/request.

Try out the request extension today, because a pair of server-side request and response objects will make your life a lot easier.


The Conflict of Desirable Values

The central values by which most men have lived, in a great many lands at a great many times--these values, almost if not entirely universal, are not always harmonious with each other. Some are, some are not. Men have always craved for liberty, security, equality, happiness, justice, knowledge, and so on. But complete liberty is not compatible with complete equality--if men were wholly free, the wolves would be free to eat the sheep. Perfect equality means that human liberties must be restrained so that the ablest and the most gifted are not permitted to advance beyond those who would inevitably lose if there were competition. Security, and indeed freedoms, cannot be preserved if freedom to subvert them is permitted. Indeed, not everyone seeks security or peace, otherwise some would not have sought glory in battle or in dangerous sports.

Justice has always been a human ideal, but it is not fully compatible with mercy. Creative imagination and spontaneity, splendid in themselves, cannot be fully reconciled with the need for planning, organization, careful and responsible calculation. Knowledge, the pursuit of truth--the noblest of aims--cannot be fully reconciled with the happiness or the freedom that men desire, for even if I know that I have some incurable disease this will not make me happier or freer. I must always choose: between peace and excitement, or knowledge and blissful ignorance.

Source: A Message to the 21st Century | by Isaiah Berlin | The New York Review of Books