Negative number PHP array indexes, var_export and WTF

Those folks that know me in the programming, web development or technology context know that I am a PHP fanboi to the nth degree. I eat, sleep, breath and live PHP. It is my very favorite programming language and it is the language that I learned how to program in. I stand behind it and have defended it – often – against the attacks of many a strict-type-language zealot from the world of .NET or Java or even Ruby. Seriously, I love PHP with all of $my <3;

However, these same people that know my insane and indescribable love for "my" language also know that I am usually very fast to point out some of it's major stupidities and illogical implementations. Which comes first, the $haystack or the $needle? Is it a str_* function or a str* function? Are the array function args passed by reference or am I expected to handle a return? There are so many of these little issues with PHP that sometimes I, myself, have no choice but to ask... WTF?

Today is one of those days. I am working on a bug for work in which the number -99 is somehow magically being transformed into -9223372036854775808 and is causing significant problems for our application. In doing some trouble shooting, I discovered that, as part of our process of caching some information, we use the var_export() function to write out an array to a file and somewhere between the passing of the data to var_export and the file that the exported data is written to, the -99, when used as an array index, suddenly became 18446744073709551517. This did not happen when -99 was the value of a scalar variable nor when it was a value assigned to an object property.

Looking a little closer I found that this particular server was running PHP 5.2.17 on CentOS, so I decided to dig even deeper and try this on other versions of PHP. On my 5.3.6 version of PHP on my Mac I could not reproduce the issue. But I was able to reproduce it on 5.2.17 on my Mac. My colleague was likewise able to reproduce it on 5.2.10 on his Ubuntu installation. And I was able to reproduce it on several PHP versions using 3v4l.org.

I'm not exactly sure what is going on under the hood, but my suspicion is that because PHP does not support unsigned integers it needs to convert the signed int -99 to an unsigned value in order to store it in the symbol table. I suspect when var_export is called, this unsigned value is being returned rather than the signed value. At the very least, this is an issue that will more than likely never be corrected since PHP 5.2 is now out of date and this problem, as far as I can tell, does not expose itself in 5.3 save for 5.3.3.

Regardless, if you want to see this for yourself and you have a 5.2 build laying around somewhere (or a 5.3.3 build), give this little snippet a shot and see what you get:

<?php
for ($i = 10; $i > -101; $i--) {
    echo "$i...\n";
    $q = array($i => 'this');
    var_export($q);
    echo "\n";
}

Or, if you just want to see it without a bunch of output, try this:

<?php
$a[-99] = 'bilk';
var_export($a);

As an added bonus, take the broken translated value and use that as an array index, then var_export that. You might pleasantly surprised by what you see I'd bet.

All of this is to say, if you are still using PHP 5.2 - and let's be honest, who is really using PHP 5.2 any more aside from those that have it on a test server - and you are using var_export on an array with signed int indexes, you are probably not going to get what you expect out of it. You have been warned.

Also? WTF PHP?

Scope? Nope. Dope!

Today I made a classic blunder in PHP programming. It is one that many make and certainly one I have made before. But the frustrating thing about this issue is it probably would have been caught by a simple unit test or, in lieu of that, something as simple as trying to capture output of my script from the go.

I was working on the command line (something I have been spending a lot more time doing) building a background processing application that will be triggered by the web. The process is expected to take between 45 minutes and an hour each time it is ran and will be ran one time a month at the discretion of an administrator (hence the reason it is not set as a cron job).

One of the very first lines of the CLI script looks like this:

<?php
// Define our path for all includes and file writes
define('APPPATH', realpath('.'));
?>

It would seem simple enough, right? Basically that little line of code defines a constant called APPPATH that would essentially describe the path to the location of the file that is setting the constant. At least that is what I thought. And every test I ran led me to believe that I had indeed made the right choice to set the path the way I did.

I ran the script, and even other scripts that derived code from the script, from the command line on my local Ubuntu machine, from the command line on our Fedora dev machine and from the command line on our Ubuntu production server. I have similar snippets that are working on all of our machines, and these snippets work interactively, through cron and through the web.

But today something happened that I did not intend. As I attempted to run my last process test of this long running script through the web I found myself in a place where my script would not run. It was being triggered properly. It was just not actually doing anything.

After an hour of trying to figure out what I was doing wrong I solicited the wisdom and advice of two of my colleagues who are seasoned Unix professionals and after a couple of minutes I was able to begin to see what was wrong. Can you guess what it was?

That’s right… PHP, on the web, was applying its scope to the CLI script that it was calling. The reason it had worked in all cases prior to triggering it by the web was that I was logged in as me and calling the script as a CLI script interactively, through the prompt. That means that the setting of the APPPATH constant was happening as expected and to what I expected.

However, from the web, when PHP runs as an apache module, it calls the CLI script from the scope of the web server. That means that the use of realpath() on the location ‘.’ (current working directory) was assuming the working directory was the web server root, not the location of where the script actually resided.

That’s right folks, I lost track of the scope in which I was working. Like a dope.

The simple solution to this problem was to change where the script looks to set its own directory. Can you think of what I could have used, instead of ‘.’ as the location to pass to realpath() so that it knows, without a doubt, what its own path is?

<?php
// Get the real path to the current directory location OF THIS FILE
define('APPPATH', realpath(dirname(__FILE__)));/
?>

I hate it when something this simple causes such deep pain and suffering, needlessly.

Handling multiple Sybase result sets in PHP on Ubuntu 8.10

A long time ago I was tasked with creating a PHP application that would talk to our Sybase database that drives our entire enterprise at work. This is not normally a big thing to do as one can simply enable the Sybase extension for PHP and have a database connection generally within minutes. However there was one thing that was an absolute necessity in this application that made it significantly more difficult to get it working: some parts of the application would be reliant upon stored procedures that returned multiple result sets.

First off let me say that any time you have the option to build stored procedures for your database quieries, do it. They are faster, safer, compiled and centralized and they keep your code from being littered with query builders all over the place. That said, this is neither the time nor the place to talk about the merits of stored procedures in application development. Just know that where I work there are no direct queries ever allowed to touch our database servers.

Knowing that I had to work under the requirement of handling multiple Sybase result sets I set out to find out how to do it and what I found was alarming. Few database extensions in PHP handle multiple result sets. In fact, as I was looking around it seemed that only the MySQLi extension and the SQL Server extension handled multiple result sets. But after a little searching around I found something called the “PHP Sybase CT driver enhancements” project on Sourceforge that essentially provided prepared statement handling and multiple result set handling for the PHP Sybase Extension. However, after trying to install it it became evident that the extension, as it was written, was still not usable in the state it was offered up for download in.

Not to worry, I have modified the package for compiling within a PHP 5 environment and have made a few changes to it that should allow it to be used as is. The source code archive file can be downloaded (in tar.gz format) by clicking here.

Please read the notes on this source code before installing it!

Before installation

Before we get started we need to cover a few assumptions:

  • You have a licensed copy of Sybase Adaptive Server
    I have not tried this against any Sybase database server other than Adaptive server.
  • You have the Sybase client installed or at the very least have the installer available
    For this exercise we are using a Red Hat RPM of the Sybase 12.5 client. Yes, we are installing on Ubuntu, which is essentially Debian and therefore does not use RPM files, but we will burn that bridge when we come to it.
  • You are comfortable building PHP extensions from source
    There is no way around this. In order to get this to work you will need to be able to build from source.
  • You have the php-devel package installed on the machine you building this extension for
    If you don’t have it, a quick sudo apt-get install php5-dev should do the trick. This will be necessary to build the Sybase extension from source.

Installation instructions

Ok, these are the steps I took in order to build the enhanced Sybase extension for PHP on my Ubuntu 8.10 desktop machine starting with Sybase ASE client and common packages in Red Hat RPM format.

  • Convert the Sybase 12.5 OpenClient and Sybase 12.5 Common RPM archives to DEB packages and install them
    To do this I used the alien package converter tool and, while inside the directory where the RPMs were living, issued the following command:
    $ alien -k sybase-common-12.5.0.1de-1.i386.rpm
    $ alien -k sybase-openclient-12.5.0.1esd-1.i386.rpm

    This created two new archives, in the directory I was in, in debian format named

    1. sybase-common-12.5.0.1DE-1_i386.deb
    2. sybase-openclient-12.5.0.1ESD-1_i386.deb

    I then issued the following commands to install them:
    $ sudo dpkg --install sybase-common-12.5.0.1DE-1_i386.deb
    $ sudo dpkg --install sybase-openclient-12.5.0.1ESD-1_i386.deb

    This made a directory named sybase-12.5 in the /opt directory.

  • Prepare your environment to build the extension
    To make things easier (and a little more compatible with the Red Hat way of doing things) I created a link inside of /top called sybase and pointed it to the sybase-12.5 directory.
    $ cd /opt
    $ ln -s -T /opt/sybase-12.5 sybase

    I then had to export some environment vars that are needed by the extension and the web server in order to handle communication with the Sybase server:
    $ export SYBASE=/opt/sybase
    $ export SYBASE_OCS=OCS-12_5
    $ export PATH=/opt/sybase/OCS-12_5/bin:$PATH
    $ export LD_LIBRARY_PATH=/opt/sybase/OCS-12_5/lib:/opt/sybase/OCS-12_5/lib3p:$LD_LIBRARY_PATH
    $ export INCLUDE=/opt/sybase/OCS-12_5/include:$INCLUDE
    $ export LIB=/opt/sybase/OCS-12_5/lib:$LIB

    Yes, I know that each one of these commands could have been placed in a file and sourced, but for some reason source is not available to my installation of Ubuntu so it was in fact easier and faster for me to do it this way. Do this how you will, but remember the values because you will need these later.

  • Build and install the php-sybase-ct extension from source against the Sybase client you just installed
    Remember that package I told you about earlier? The one that I said you could download? If not, get it now and unpack it to a directory somewhere where you have permission to unpack stuff on your system. For me, it was ~/Temp.

    Change to the directory you unpacked the source code to and configure it for make and installation using the php-devel package:
    $ cd ~/Temp
    $ phpize
    $ ./configure --with-sybase-ct=$SYBASE/$SYBASE_OCS
    $ make
    $ sudo make install

    Find out where your extensions directory is on your machine and quickly check it to make sure there is a php_sybase_ct.so file living in it. On my machine the extension directory is /usr/lib/php5/20060613+lfs/. Yours may be different.

    At this point you can safely do a sudo make clean but you might want to hold off on that until it is all working in case you need to rebuild at all. It does happen from time to time. Just sayin’.

  • Configure PHP to use the new extension
    Now we need to tell PHP to use the new extension we just built. To do that we need to create an ini file for the extension and put it inside of the extensions directory where PHP can find it. On my machine PHP looks for ini files to parse in /etc/php5/conf.d/ so naturally that is where I am going to go to to tell PHP to use this extension.

    $ cd /etc/php5/conf.d/

    Now we need to create an ini file and put into a directive to load the extension. You can use whatever editor you like. I prefer to use vim:
    $ sudo vim sybase_ct.ini

    Inside this file place the following two lines:
    ; Enable the sybase extension
    extension=sybase_ct.so

  • Configure your environment to load the correct environment variables whenever the machine starts
    This one caused me fits for a long time in Ubuntu. In order to ensure that you can use the Sybase extension from both the CLI and the web server you will need to take all of the environment variables that exported prior to building the extension and place them in both the /etc/profile startup script AND the web servers environment variable setting script. This caused countless hours of frustration and anger for me and I hope I can save you some angst with this little snippet.

    Add the environment vars to /etc/profile. You can use any editor you like. I like vim:
    $ sudo vim /etc/profile

    At the end of the file add the environment vars:
    export SYBASE=/opt/sybase
    export SYBASE_OCS=OCS-12_5
    export PATH=/opt/sybase/OCS-12_5/bin:$PATH
    export LD_LIBRARY_PATH=/opt/sybase/OCS-12_5/lib:/opt/sybase/OCS-12_5/lib3p:$LD_LIBRARY_PATH
    export INCLUDE=/opt/sybase/OCS-12_5/include:$INCLUDE
    export LIB=/opt/sybase/OCS-12_5/lib:$LIB

    Now add these same entries into your web server’s environment variables. I am using apache and assuming you are to. If not, consult your web server’s documentation for how to do this:
    $ sudo vim /etc/apache2/envvars

    Now add these entries to the end of the file:
    export SYBASE=/opt/sybase
    export SYBASE_OCS=OCS-12_5
    export PATH=/opt/sybase/OCS-12_5/bin:$PATH
    export LD_LIBRARY_PATH=/opt/sybase/OCS-12_5/lib:/opt/sybase/OCS-12_5/lib3p:$LD_LIBRARY_PATH
    export INCLUDE=/opt/sybase/OCS-12_5/include:$INCLUDE
    export LIB=/opt/sybase/OCS-12_5/lib:$LIB

  • Restart your web server
    Like everything that involves a change to the PHP environment or configuration on your machine, restart the web server. I am assuming this is being built upon an apache server. If not, you will need to know how to stop and start your web server or, at the very least, know how to reboot your machine:
    $ sudo apache2ctl stop
    $ sudo apache2ctl start

    Run a PHP info page or CLI call to see if it is loaded:
    $ php -m

    You should see sybase_ct. If not, something went wrong. If so, you are now golden.

At this point you should be able to run queries against your Sybase server AND handle multiple result sets using the sybase_next_result() function. There is no documentation for this function, but a quick read through of the mssql_next_result() function will tell pretty well how to use it.

Enjoy! And if this was at all helpful please leave a comment to let me know.

Notes on the PHP Sybase CT Enhanced extension

Some things to keep in mind when using this extension:

  • This extension is coded in PHP 4 source.
  • When I got my hands on it the code was still much in development phase. Not much has changed about that other than I removed debug output from it.
  • It does not handle connection failure well when the server does not respond. So unwell in fact that internally it causes segmentation faults and results in blank web pages upon failure. I do not know how to fix that.
  • If you know how to program in c or want to make it better, please feel free to do so. I have contacted the original authors of this extension and of the three only one has done any real work on it and none have worked on it in the last five years and really do not intend to. I was given permission to distribute it and modify it. But I really do not know what I am doing in c. Yet. 😉