dragffy.com

I recently came across http://opencountrycodes.appspot.com/ which has an ISO 3166-1 list of country codes for various programming languages. I wanted to generate an HTML drop down list for customer to choose from based on these codes. However, there is no PHP version provoided. So taking the XML version found at http://opencountrycodes.appspot.com/xml/ I made a trivial script that would take the codes found in the XML document and organise them in to a PHP array called $XML2PHPCountryCodes and save that array in a file called country_names.php. The output file can then be included in any PHP script making the contained array available.

< ?php
$str = file_get_contents('http://opencountrycodes.appspot.com/xml/');
$xml = new SimpleXMLElement($str);
$out = '$countries'." = array(\n";
foreach ($xml->country as $country)
{
	$out .= "'{$country['code']}' => \"{$country['name']}\",\n";
}
$out .= ");";

file_put_contents('country_names.php', $out);

Shamelessly ripped from: http://gggeek.altervista.org/2007/11/26/the-completely-unofficial-xdebugini/

There are just a couple of minor annoyances with the Xdebug PHP debugger really, the first one being the absence of a proper documentation package to be downloaded and read offline.

I find well-commented ini files, in the Apache httpd.conf style, the best complement to user manuals and technical references: when you are editing the forgotten config of that vetust server that has no web access or even ssh whatsoever, awkwardly sitting on an unstable pile of extinguished hardware in the darkest corner of the server room, they will save you dozens of round trips to go googling for information.

Unfortunately the Xdebug distribution contains no such thing: no comments, no list of ini directives, no ini file at all. But since I am a nice chap, after having carved out such precious jewel, I thought it might be of interest to the community, and without further ado here it is:

[xdebug]

;;;;;;;;;;;;;;;;;;;
; About this file ;
;;;;;;;;;;;;;;;;;;;
;
; built by G. Giunta as a verbatim copy of info from xedbug website on 2007/11/08

; You must uncomment one (and only one) line from the following to load
; the xdebug extension.
zend_extension=”/usr/lib/php4/xdebug.so”
;zend_extension_ts=”/usr/lib/php4/xdebug.so”
;zend_extension_ts=”c:\php4\xdebug.dll”
;zend_extension=”c:\php4\xdebug.dll”

; When this setting is set to on, the tracing of function calls will be enabled
; just before the script is run. This makes it possible to trace code in the
; auto_prepend_file.
xdebug.auto_trace=Off

; This setting, defaulting to On, controls whether Xdebug should write the
; filename used in include(), include_once(), require() or require_once() to
; the trace files.
xdebug.collect_includes=On

; This setting, defaulting to 0, controls whether Xdebug should collect the
; parameters passed to functions when a function call is recorded in either the
; function trace or the stack trace.
; The setting defaults to Off because for very large scripts it may use huge
; amounts of memory and therefore make it impossible for the huge script to run.
; You can most safely turn this setting on, but you can expect some problems in
; scripts with a lot of function calls and/or huge data structures as parameters.
; Xdebug 2 will not have this problem with increased memory usage, as it will
; never store this information in memory. Instead it will only be written to disk.
; This means that you need to have a look at the disk usage though.
; This setting can have four different values. For each of the values a different
; amount of information is shown. Below you will see what information each of the
; values provides. See also the introduction of the feature Stack Traces for a
; few screenshots.
; Value Argument Information Shown
; 0 None.
; 1 Type and number of elements (f.e. string(6), array(8)).
; 2 Type and number of elements, with a tool tip for the full information.
; 3 Full variable contents (with the limits respected as set by
; xdebug.var_display_max_children, xdebug.var_display_max_data and
; xdebug.var_display_max_depth.
; 4 Full variable contents and variable name.
xdebug.collect_params=0

; This setting, defaulting to Off, controls whether Xdebug should write the
; return value of function calls to the trace files.
xdebug.collect_return=On

; This setting tells Xdebug to gather information about which variables are
; used in a certain scope. This analysis can be quite slow as Xdebug has to
; reverse engineer PHP’s opcode arrays. This setting will not record which
; values the different variables have, for that use xdebug.collect_params.
; This setting needs to be enabled only if you wish to use
; xdebug_get_declared_vars().
xdebug.collect_vars=Off

; If this setting is On then stacktraces will be shown by default on an error
; event. You can disable showing stacktraces from your code with xdebug_disable().
; As this is one of the basic functions of Xdebug, it is advisable to leave this
; setting set to ‘On’.
xdebug.default_enable=On

; These seven settings control which data from the superglobals is shown when an
; error situation occurs. Each php.ini setting can consist of a comma seperated
; list of variables from this superglobal to dump, but make sure you do not add
; spaces in this setting. In order to dump the REMOTE_ADDR and the REQUEST_METHOD
; when an error occurs, add this setting: xdebug.dump.SERVER = REMOTE_ADDR,REQUEST_METHOD
xdebug.dump.COOKIE=
xdebug.dump.FILES=
xdebug.dump.GET=
xdebug.dump.POST=
xdebug.dump.REQUEST=
xdebug.dump.SERVER=
xdebug.dump.SESSION=

; Controls whether the values of the superglobals as defined by the xdebug.dump.*
; settings whould be shown or not.
xdebug.dump_globals=On

; Controls whether the values of the superglobals should be dumped on all error
; situations (set to Off) or only on the first (set to On).
xdebug.dump_once=On

; If you want to dump undefined values from the superglobals you should set this
; setting to On, otherwise leave it set to Off.
xdebug.dump_undefined=Off

; Controls whether Xdebug should enforce ‘extended_info’ mode for the PHP parser;
; this allows Xdebug to do file/line breakpoints with the remote debugger. When
; tracing or profiling scripts you generally want to turn off this option as PHP’s
; generated oparrays will increase with about a third of the size slowing down
; your scripts. This setting can not be set in your scripts with ini_set(), but
; only in php.ini.
xdebug.extended_info=1

; Introduced in Xdebug 2.1
; This setting determines the format of the links that are made in the display
; of stack traces where file names are used. This allows IDEs to set up a
; link-protocol that makes it possible to go directly to a line and file by
; clicking on the filenames that Xdebug shows in stack traces. An example format
; might look like: myide://%f@%l
; The possible format specifiers are:
; %f the filename
; %l the line number
xdebug.file_link_format=

; Controls which IDE Key Xdebug should pass on to the DBGp debugger handler.
; The default is based on environment settings. First the environment setting
; DBGP_IDEKEY is consulted, then USER and as last USERNAME. The default is set
; to the first environment variable that is found. If none could be found the
; setting has as default ”.
xdebug.idekey=

; This is the base url for the links from the function traces and error message
; to the manual pages of the function from the message. It is advisable to set
; this setting to use the closest mirror.
xdebug.manual_url=http://www.php.net

; Controls the protection mechanism for infinite recursion protection. The value
; of this setting is the maximum level of nested functions that are allowed before
; the script will be aborted.
xdebug.max_nesting_level=100

; Introduced in Xdebug 2.1
; By default Xdebug overloads var_dump() with its own improved version for displaying
; variables when the html_errors php.ini setting is set to 1. In case you do not
; want that, you can set this setting to 0, but check first if it’s not smarter
; to turn off html_errors.
xdebug.overload_var_dump=On

; When this setting is set to 1, profiler files will not be overwritten when a
; new request would map to the same file (depnding on the xdebug.profiler_output_name setting.
; Instead the file will be appended to with the new profile.
xdebug.profiler_append=0

; Enables Xdebug’s profiler which creates files in the profile output directory.
; Those files can be read by KCacheGrind to visualize your data. This setting
; can not be set in your script with ini_set().
xdebug.profiler_enable=0

; When this setting is set to 1, you can trigger the generation of profiler
; files by using the XDEBUG_PROFILE GET/POST parameter. This will then write
; the profiler data to defined directory.
xdebug.profiler_enable_trigger=0

; The directory where the profiler output will be written to, make sure that the
; user who the PHP will be running as has write permissions to that directory.
; This setting can not be set in your script with ini_set().
xdebug.profiler_output_dir=/tmp

; This setting determines the name of the file that is used to dump traces into.
; The setting specifies the format with format specifiers, very similar to sprintf()
; and strftime(). There are several format specifiers that can be used to format
; the file name.
; See the xdebug.trace_output_name documentation for the supported specifiers.
xdebug.profiler_output_name=cachegrind.out.%p

; Normally you need to use a specific HTTP GET/POST variable to start remote debugging.
; When this setting is set to ‘On’ Xdebug will always attempt to start a remote
; debugging session and try to connect to a client, even if the GET/POST/COOKIE
; variable was not present.
xdebug.remote_autostart=Off

; This switch controls whether Xdebug should try to contact a debug client which
; is listening on the host and port as set with the settings xdebug.remote_host
; and xdebug.remote_port. If a connection can not be established the script will
; just continue as if this setting was Off.
xdebug.remote_enable=Off

; Can be either ‘php3′ which selects the old PHP 3 style debugger output, ‘gdb’
; which enables the GDB like debugger interface or ‘dbgp’ – the brand new debugger
; protocol. The DBGp protocol is more widely supported by clients. See more
; information in the introduction for Remote Debugging.
xdebug.remote_handler=dbgp

; Selects the host where the debug client is running, you can either use a host
; name or an IP address.
xdebug.remote_host=localhost

; If set to a value, it is used as filename to a file to which all remote debugger
; communications are logged. The file is always opened in append-mode, and will
; therefore not be overwritten by default. There is no concurrency protection
; available.
xdebug.remote_log=

; Selects when a debug connection is initiated. This setting can have two different values:
; req Xdebug will try to connect to the debug client as soon as the script starts.
; hit Xdebug will only try to connect to the debug client as soon as an error condition occurs.
xdebug.remote_mode=req

; The port to which Xdebug tries to connect on the remote host. Port 9000 is the
; default for both the client and the bundled debugclient. As many clients use
; this port number, it is best to leave this setting unchanged.
xdebug.remote_port=9000

; When this setting is set to 1, Xdebug will show a stack trace whenever an
; exception is raised – even if this exception is actually caught.
xdebug.show_exception_trace=0

; When this setting is set to something != 0 Xdebug’s generated stack dumps in
; error situations will also show all variables in the top-most scope. Beware
; that this might generate a lot of information, and is therefore turned off by default.
xdebug.show_local_vars=0

; When this setting is set to something != 0 Xdebug’s human-readable generated
; trace files will show the difference in memory usage between function calls.
; If Xdebug is configured to generate computer-readable trace files then they
; will always show this information.
xdebug.show_mem_delta=0

; The format of the trace file.
; 0 shows a human readable indented trace file with: time index, memory usage,
; memory delta (if the setting xdebug.show_mem_delta is enabled), level,
; function name, function parameters (if the setting xdebug.collect_params
; is enabled, filename and line number.
; 1 writes a computer readable format which has two different records. There
; are different records for entering a stack frame, and leaving a stack frame
xdebug.trace_format=0

; When set to ’1′ the trace files will be appended to, instead of being overwritten
; in subsequent requests.
xdebug.trace_options=0

; The directory where the tracing files will be written to, make sure that the
; user who the PHP will be running as has write permissions to that directory.
xdebug.trace_output_dir=/tmp

; This setting determines the name of the file that is used to dump traces into.
; The setting specifies the format with format specifiers, very similar to
; sprintf() and strftime(). There are several format specifiers that can be used
; to format the file name. The ‘.xt’ extension is always added automatically.
;The possible format specifiers are:
; %c crc32 of the current working directory trace.%c trace.1258863198.xt
; %p pid trace.%p trace.5174.xt
; %r random number trace.%r trace.072db0.xt
; %s script name cachegrind.out.%s cachegrind.out._home_httpd_html_test_xdebug_test_php
; %t timestamp (seconds) trace.%t trace.1179434742.xt
; %u timestamp (microseconds) trace.%u trace.1179434749_642382.xt
; %H $_SERVER['HTTP_HOST'] trace.%H trace.kossu.xt
; %R $_SERVER['REQUEST_URI'] trace.%R trace._test_xdebug_test_php_var=1_var2=2.xt
; %S session_id (from $_COOKIE if set) trace.%S trace.c70c1ec2375af58f74b390bbdd2a679d.xt
; %% literal % trace.%% trace.%%.xt
xdebug.trace_output_name=trace.%c

; Controls the amount of array children and object’s properties are shown when
; variables are displayed with either xdebug_var_dump(), xdebug.show_local_vars
; or through Function Traces. This setting does not have any influence on the
; number of children that is send to the client through the Remote Debugging feature.
xdebug.var_display_max_children=128

; Controls the maximum string length that is shown when variables are displayed
; with either xdebug_var_dump(), xdebug.show_local_vars or through Function Traces.
; This setting does not have any influence on the amount of data that is send to
; the client through the Remote Debugging feature.
xdebug.var_display_max_data=512

; Controls how many nested levels of array elements and object properties are
; when variables are displayed with either xdebug_var_dump(),
; xdebug.show_local_vars or through Function Traces. This setting does not have
; any influence on the depth of children that is send to the client through the
; Remote Debugging feature.
xdebug.var_display_max_depth=3

For installation, just copy and paste it at the end of your php.ini file (or, if you’re on Debian or similarly configured distros, you can save it as separate file in the php ini directory).

Note1: the zend_extension lines at the top of the file are tailored for a php4 install. Make sure to change them to fit your environment.

If you develop multiple sites and you need virtual hosting functionality, scroll down to the end of the /private/etc/apache2/httpd.conf file and uncomment the following:

# Include /private/etc/apache2/extra/httpd-vhosts.conf

Next, you’ll need to setup whatever virtual hosts you have in the virtual hosts file /private/etc/apache2/extra/httpd-vhosts.conf

You need to make an entry in the httpd-vhosts.conf file like so:

<virtualhost *:80>
   ServerName beta-site-1.com
   ServerAlias www.beta-site-1.com
   ServerAdmin webmaster@beta-site-1.com
   ErrorLog "/private/var/log/apache2/dummy-host2.example.com-error_log"
   CustomLog "/private/var/log/apache2/dummy-host2.example.com-access_log" common

   DocumentRoot "/Library/WebServer/beta-site-1"
   ScriptAlias /cgi-bin/ "/Library/WebServer/beta-site-1/cgi-bin"
   
     Options FollowSymLinks MultiViews Includes
     AllowOverride All
     Order allow,deny
     Allow from all
   
</virtualhost>

The examples provided by Apple in the vhosts file are slightly incorrect and if you use the CustomLog lines as is you will get errors the following errors if you run: apachectl -t -D DUMP_VHOSTS:
Syntax error on line 40 of /private/etc/apache2/extra/httpd-vhosts.conf:
CustomLog takes two or three arguments, a file name, a custom log format string or format name, and an optional "env=" clause (see docs)

This is because
CustomLog "/private/var/log/apache2/dummy-host.example.com-access_log common

Should actually read:
CustomLog "/private/var/log/apache2/dummy-host.example.com-access_log" common

Modern web applications almost all make use of AJAX to enhance their user appeal. However, AJAX can be hard to debug, especially when all the traditional tools do little to help. When doing standard PHP coding CodeIgniter (CI) offers a profiler which will be appended to the bottom of generated pages. This gives you information on the request that was recently processed, including any POST or GET values passed to the script, how long execution took and how long any SQL queries took as well as what those queries actually were. During development I find this information indispensable to make sure my SQL queries are correct and things are working as I want.

However, when coding up AJAX aspects of the application this doesn’t work very well, any page fragments fed back to the JavaScript will have the profiler stuck at the bottom, and since these fragments are usually inserted inside DIVs the profiler cannot be read in it’s entirety, or it breaks the layout, or more likely, both of the above. Therefore I set out to find a way of making it AJAX compatible, so that it would always end up at the bottom of the page, even when it was originally returned appended to a page fragment. In the end this turned out easier than anticipated.

One small change needs to be made to your footer, you need to create an empty div with an ID of “debug”, this div will hold the profiler, so place the div where you want the profiler. The advantage with this is that you cannot see it when it is empty so it could even be left in for production code. The div looks like this:
<div id="debug"></div>

Now we need to extend, or rather override the CI profiler so create the file system/application/libraries/MY_Profiler.php

Replace MY_ with the value of $config['subclass_prefix'] found in your config file, by default it is MY_.

Next paste the following code in to it. Remember to change the name of the class as dictated by the filename you use, also change the name of the constructor function too. You can also change the script line to load the jQuery library from one of your servers, or even rewrite the entire script section to use any JavaScript you like.

<?php if (!defined('BASEPATH')) exit('No direct script access allowed');

class MY_Profiler extends CI_Profiler
{
    function MY_profiler()
    {
        parent::CI_Profiler();
    }

    function run()
    {
            $output = <<<ENDJS
<script src="http://code.jquery.com/jquery-latest.js" />
<script type="text/javascript" language="javascript" charset="utf-8">
// < ![CDATA[
    $(document).ready(function() {
        var html = $('#codeigniter_profiler').clone();
        $('#codeigniter_profiler').remove();
        $('#debug').hide().empty().append(html).fadeIn('slow');
    });
// ]]>
</script>
ENDJS;
            $output .= "<div id='codeigniter_profiler' style='font-size: 0.7em; clear:both;background-color:#fff;padding:10px;'>";
            $output .= $this->_compile_uri_string();
            $output .= $this->_compile_controller_info();
            $output .= $this->_compile_memory_usage();
            $output .= $this->_compile_benchmarks();
            $output .= $this->_compile_get();
            $output .= $this->_compile_post();
            $output .= $this->_compile_queries();
            $output .= '</div>';
            return $output;
    }
}

If you have already loaded a jQuery library in your header somewhere then make sure this ajax fragment doesn’t load jQuery again – otherwise all your event bindings and everything else will fail. If this is the case simply remove the line:

<script src="http://code.jquery.com/jquery-latest.js" />

From the above code.

That should be it, hit me back if you have any feedback, questions, or need some help.

Last week I setup MySQL replication for a database with two slaves. I wanted to see how resilient it was, and one of the tests I wanted to perform was to create a record on the master and then interrupt the replication of the same record on the slave. This would allow me to see whether MySQL could recover from such an error.

So I thought the easiest way of doing this would be to to insert a large 700mb
file in to MySQL – to give me enough time to interrupt a slave before
replication had completed. Thus being able to test MySQL’s resilience to
loss of connectivity during UPDATE/INSERT queries.

I have never used BLOB columns before so my quest took my on a journey during which I learned a lot about these blobs. Now, even though a MySQL long blob can hold 4GB of data I have encountered
problem after problem dealing with it:

PHP scripts are limited to 32MB memory maximum, this can be upped, but
is not scalable and not a good solution. It means that you cannot simply
read the data of file and perform an INSERT since the file data is
stored in PHP’s memory first. This meant the development of a PHP script
that could read chunks (just a meg or so) from a file and perform UPDATE
queries along with MySQL CONCAT function to effectively append each
chunk.

CONCAT, as I eventually found out, returns NULL if any of it’s arguments
are NULL. In my case when attempting to UPDATE a row with additional
chunks the resulting blob size was always zero. This was because the
very first time CONCAT is used the blob field is empty (NULL), therefore
CONCAT always returned NULL – and so, the data in the blob column never
grew. The query looked like this:
UPDATE `$table` SET `data` = CONCAT(data, '{mysqli_real_escape_string($buffer)}') WHERE `id`=$id";

I then combined CONCAT with COALESCE which will return an alternative
value that you specify in the eventuality the first argument is NULL. By
combining these two I could overcome the above problem.
Resulting query structure:
UPDATE `$table` SET `data` = COALESCE(CONCAT(data, '{mysqli_real_escape_string($buffer)}'), '') WHERE `id`=$id";

Great, now we’re finally adding our chunks on. Then my computer ran out
of disc space – the reason: 7 gigs of MySQL replication logs. Took a
while to find that out. Fixed.

Next problem, although I was certain (via in-depth PHP debugging) that
PHP was correctly reading chunks from the file in a consecutive order,
of the correct size, and the SQL query was being properly executed
without any sly errors the blob column would never end up growing above
12MB (according the PHP MyAdmin). Through even more intensive debugging
I found that after concatenating each chunk the blob would grow by the
correct amount up to 16MB, then it would zero and start again. The
result of pushing a 700MB ISO in to it would always be less 16MB.

Working from a hunch I upped the max_packet_size in my.conf from 16MB to
36MB, the result was as above but this time the blob would grow to 32MB
before zeroing itself.

I then found this MySQL bug report:
http://bugs.mysql.com/bug.php?id=22853

It is considered a bug simply because by upping max_packet_size to
something like 4GB or anything substantial leaves the MySQL server open
to network DoS attacks. There is no work around. A fix is scheduled for
version 6.

Anyway, since we don’t care about security I maxed out the packet_size,
and my chunking script worked, but very quickly I noticed a huge amount of
slowdown once the blob had grown to just a few tens of megs. In fact it
took about 20 minutes to get to 200MB. Without supporting my theory with
any evidence I reckon the CONCAT function reads all the data out of the
database in to memory, does it’s stuff and then stuffs it back in to the
database. The result is an increasingly slow query.

It would seem that although MySQL supports blobs of about 4GB that you
can never get that size unless you:

1. Change the max_packet_size to a stupidly high number, leaving your
server open to DoS attacks so then you can just do a single INSERT. Using CONCAT
is out since it would take forever.

2. Change the memory limit of PHP scripts to be greater than 4GB.

3. Have absolutely bags of RAM since three copies of the file are stored
in RAM (certainly two copies). Since you cannot chunk the file and
perform CONCATS PHP will read the entire file in to it’s memory. This
will all then be buffered by the MySQL driver. Then this is passed to
the database. Therefore if you wish to insert a 4 GB file you will
require >12GB of RAM.

One work around is to chunk the file as before but each chunk is
represented by a single row. These can then be linked together by giving
each row a master file ID, so that all necessary rows can be retrieved.

For my purposes this doesn’t help, I needed queries that took a while,
inserting rows like this will take but a fraction of a second each.

Came accross the following post at:
http://www.pluralsight.com/community/blogs/fritz/archive/2007/06/19/47771.aspx

“I spent more time than I care to admit tracking down this one, perhaps this post will save someone else the trouble…

When defining a number of functions in a prototype in JavaScript, do not include a trailing comma after the last function:

MyType.prototype = {
    foo : function() {
          // ...
    },

    bar : function() {
        //...
    }, //< - fails in IE!
 }

What was especially tricky about tracking this problem down was that FireFox works with or without the trailing comma, so it only fails in IE!"

Frustrated at having to create my own drop-down country lists in CI I created a helper for it. It can even work with the CI Validation class, so if the form is submitted and fails validation when it is redisplayed the same country is selected as the form was submitted with.

Download the file MY_form_helper.php and put it in the folder: /system/application/helpers/.

Download the file countries.php and place that in /system/application/config/.

You will now be able to create drop-down lists of countries in your views like this:
< ?=form_countries( 'country', $this->validation->country, array( 'style' => 'width: 250px' ) )?>

Ensure that you create a validation rule called “country” in your controller and that you are loading the form helper.

form_countries() takes three arguments. The first is the name you would like to give the select box, this is required. The second is the country to select by default – this could be something like ‘GB’, or $this->validation->country, otherwise you can leave it blank. The third argument is any additional properties to give the select box, you can leave this blank, or pass an array. In the example above we also set the width to 250x. The country chosen by the user will be returned to the script as a two-letter ISO country code, if you are validating this then you can set the min and max length values as 2, and accept alphabetical characters only. If using MySQL as your database you will get best performance by creating a column of the type char with a length of 2.

Profiling an application is a great way to see how it is performing. CodeIgniter comes with a profiling class, which you can make calls to from your controllers. However, during development I find profiling so helpful that I want it on all my pages, without having to specifically call it each time. I also want to be able to deploy my development site to a production server – making calls to the profiler throughout the controllers means I would need to edit each and every controller to make sure the profiler wasn’t active on the live server.

After much searching I discovered a better solution. This allows you to add profiling to the bottom of every page (along with its debug info and SQL query info). When you copy the dev site live you just exclude 1 file, and profiling will be removed totally. All you need to do is create a file called MY_Output.php in system/application/libraries that extends the Output core class, with the following contents:

< ?php
# /system/application/libraries/MY_Output.php
if (!defined('BASEPATH')) exit('No direct script access allowed');
class MY_Output extends CI_Output {
function __construct() {
parent::__construct();
$this->enable_profiler( TRUE );
}
}
?>


This will enable profiling output on all your pages for development and debug purposes. When you copy your site to a production server just make sure you don’t copy MY_Output.php file. I normally use rsync for copying live and just add: --exclude="MY_Output.php" to the rsync command.

The condition to using the name MY_Output.php is that you left the variable $config['subclass_prefix'] in config.php as default. This defaults to $config['subclass_prefix'] = 'MY_';, so if you change it you will need to alter the filename of your new class.