Archive for the ‘Programming’ Category

Bzr over SMB/CIFS using SMBMOUNT

on Wednesday 16th September, 2009 Gabe speculated thusly…

Ensure that the remote uig and gid are set correctly!

smbmount //foo/bar /media/bar -o username=myusername, password=mypassword, dir_mode=0755, file_mode=0664, noperm, uid=501, gid=501

Posted in Bazaar, Development, Programming, Revision Control, Server

No Comments »

Xdebug.ini

on Wednesday 9th September, 2009 Gabe speculated thusly…

Shamelessly ripped from: http://gggeek.altervista.org/2007/11/26/the-completely-unofficial-xdebugini/

There are just a couple of minor annoyances with the Xdebug PHP debugger really, the first one being the absence of a proper documentation package to be downloaded and read offline.

I find well-commented ini files, in the Apache httpd.conf style, the best complement to user manuals and technical references: when you are editing the forgotten config of that vetust server that has no web access or even ssh whatsoever, awkwardly sitting on an unstable pile of extinguished hardware in the darkest corner of the server room, they will save you dozens of round trips to go googling for information.

Unfortunately the Xdebug distribution contains no such thing: no comments, no list of ini directives, no ini file at all. But since I am a nice chap, after having carved out such precious jewel, I thought it might be of interest to the community, and without further ado here it is:

[xdebug]

;;;;;;;;;;;;;;;;;;;
; About this file ;
;;;;;;;;;;;;;;;;;;;
;
; built by G. Giunta as a verbatim copy of info from xedbug website on 2007/11/08

; You must uncomment one (and only one) line from the following to load
; the xdebug extension.
zend_extension=”/usr/lib/php4/xdebug.so”
;zend_extension_ts=”/usr/lib/php4/xdebug.so”
;zend_extension_ts=”c:\php4\xdebug.dll”
;zend_extension=”c:\php4\xdebug.dll”

; When this setting is set to on, the tracing of function calls will be enabled
; just before the script is run. This makes it possible to trace code in the
; auto_prepend_file.
xdebug.auto_trace=Off

; This setting, defaulting to On, controls whether Xdebug should write the
; filename used in include(), include_once(), require() or require_once() to
; the trace files.
xdebug.collect_includes=On

; This setting, defaulting to 0, controls whether Xdebug should collect the
; parameters passed to functions when a function call is recorded in either the
; function trace or the stack trace.
; The setting defaults to Off because for very large scripts it may use huge
; amounts of memory and therefore make it impossible for the huge script to run.
; You can most safely turn this setting on, but you can expect some problems in
; scripts with a lot of function calls and/or huge data structures as parameters.
; Xdebug 2 will not have this problem with increased memory usage, as it will
; never store this information in memory. Instead it will only be written to disk.
; This means that you need to have a look at the disk usage though.
; This setting can have four different values. For each of the values a different
; amount of information is shown. Below you will see what information each of the
; values provides. See also the introduction of the feature Stack Traces for a
; few screenshots.
; Value Argument Information Shown
; 0 None.
; 1 Type and number of elements (f.e. string(6), array(8)).
; 2 Type and number of elements, with a tool tip for the full information.
; 3 Full variable contents (with the limits respected as set by
; xdebug.var_display_max_children, xdebug.var_display_max_data and
; xdebug.var_display_max_depth.
; 4 Full variable contents and variable name.
xdebug.collect_params=0

; This setting, defaulting to Off, controls whether Xdebug should write the
; return value of function calls to the trace files.
xdebug.collect_return=On

; This setting tells Xdebug to gather information about which variables are
; used in a certain scope. This analysis can be quite slow as Xdebug has to
; reverse engineer PHP’s opcode arrays. This setting will not record which
; values the different variables have, for that use xdebug.collect_params.
; This setting needs to be enabled only if you wish to use
; xdebug_get_declared_vars().
xdebug.collect_vars=Off

; If this setting is On then stacktraces will be shown by default on an error
; event. You can disable showing stacktraces from your code with xdebug_disable().
; As this is one of the basic functions of Xdebug, it is advisable to leave this
; setting set to ‘On’.
xdebug.default_enable=On

; These seven settings control which data from the superglobals is shown when an
; error situation occurs. Each php.ini setting can consist of a comma seperated
; list of variables from this superglobal to dump, but make sure you do not add
; spaces in this setting. In order to dump the REMOTE_ADDR and the REQUEST_METHOD
; when an error occurs, add this setting: xdebug.dump.SERVER = REMOTE_ADDR,REQUEST_METHOD
xdebug.dump.COOKIE=
xdebug.dump.FILES=
xdebug.dump.GET=
xdebug.dump.POST=
xdebug.dump.REQUEST=
xdebug.dump.SERVER=
xdebug.dump.SESSION=

; Controls whether the values of the superglobals as defined by the xdebug.dump.*
; settings whould be shown or not.
xdebug.dump_globals=On

; Controls whether the values of the superglobals should be dumped on all error
; situations (set to Off) or only on the first (set to On).
xdebug.dump_once=On

; If you want to dump undefined values from the superglobals you should set this
; setting to On, otherwise leave it set to Off.
xdebug.dump_undefined=Off

; Controls whether Xdebug should enforce ‘extended_info’ mode for the PHP parser;
; this allows Xdebug to do file/line breakpoints with the remote debugger. When
; tracing or profiling scripts you generally want to turn off this option as PHP’s
; generated oparrays will increase with about a third of the size slowing down
; your scripts. This setting can not be set in your scripts with ini_set(), but
; only in php.ini.
xdebug.extended_info=1

; Introduced in Xdebug 2.1
; This setting determines the format of the links that are made in the display
; of stack traces where file names are used. This allows IDEs to set up a
; link-protocol that makes it possible to go directly to a line and file by
; clicking on the filenames that Xdebug shows in stack traces. An example format
; might look like: myide://%f@%l
; The possible format specifiers are:
; %f the filename
; %l the line number
xdebug.file_link_format=

; Controls which IDE Key Xdebug should pass on to the DBGp debugger handler.
; The default is based on environment settings. First the environment setting
; DBGP_IDEKEY is consulted, then USER and as last USERNAME. The default is set
; to the first environment variable that is found. If none could be found the
; setting has as default ”.
xdebug.idekey=

; This is the base url for the links from the function traces and error message
; to the manual pages of the function from the message. It is advisable to set
; this setting to use the closest mirror.
xdebug.manual_url=http://www.php.net

; Controls the protection mechanism for infinite recursion protection. The value
; of this setting is the maximum level of nested functions that are allowed before
; the script will be aborted.
xdebug.max_nesting_level=100

; Introduced in Xdebug 2.1
; By default Xdebug overloads var_dump() with its own improved version for displaying
; variables when the html_errors php.ini setting is set to 1. In case you do not
; want that, you can set this setting to 0, but check first if it’s not smarter
; to turn off html_errors.
xdebug.overload_var_dump=On

; When this setting is set to 1, profiler files will not be overwritten when a
; new request would map to the same file (depnding on the xdebug.profiler_output_name setting.
; Instead the file will be appended to with the new profile.
xdebug.profiler_append=0

; Enables Xdebug’s profiler which creates files in the profile output directory.
; Those files can be read by KCacheGrind to visualize your data. This setting
; can not be set in your script with ini_set().
xdebug.profiler_enable=0

; When this setting is set to 1, you can trigger the generation of profiler
; files by using the XDEBUG_PROFILE GET/POST parameter. This will then write
; the profiler data to defined directory.
xdebug.profiler_enable_trigger=0

; The directory where the profiler output will be written to, make sure that the
; user who the PHP will be running as has write permissions to that directory.
; This setting can not be set in your script with ini_set().
xdebug.profiler_output_dir=/tmp

; This setting determines the name of the file that is used to dump traces into.
; The setting specifies the format with format specifiers, very similar to sprintf()
; and strftime(). There are several format specifiers that can be used to format
; the file name.
; See the xdebug.trace_output_name documentation for the supported specifiers.
xdebug.profiler_output_name=cachegrind.out.%p

; Normally you need to use a specific HTTP GET/POST variable to start remote debugging.
; When this setting is set to ‘On’ Xdebug will always attempt to start a remote
; debugging session and try to connect to a client, even if the GET/POST/COOKIE
; variable was not present.
xdebug.remote_autostart=Off

; This switch controls whether Xdebug should try to contact a debug client which
; is listening on the host and port as set with the settings xdebug.remote_host
; and xdebug.remote_port. If a connection can not be established the script will
; just continue as if this setting was Off.
xdebug.remote_enable=Off

; Can be either ‘php3′ which selects the old PHP 3 style debugger output, ‘gdb’
; which enables the GDB like debugger interface or ‘dbgp’ – the brand new debugger
; protocol. The DBGp protocol is more widely supported by clients. See more
; information in the introduction for Remote Debugging.
xdebug.remote_handler=dbgp

; Selects the host where the debug client is running, you can either use a host
; name or an IP address.
xdebug.remote_host=localhost

; If set to a value, it is used as filename to a file to which all remote debugger
; communications are logged. The file is always opened in append-mode, and will
; therefore not be overwritten by default. There is no concurrency protection
; available.
xdebug.remote_log=

; Selects when a debug connection is initiated. This setting can have two different values:
; req Xdebug will try to connect to the debug client as soon as the script starts.
; hit Xdebug will only try to connect to the debug client as soon as an error condition occurs.
xdebug.remote_mode=req

; The port to which Xdebug tries to connect on the remote host. Port 9000 is the
; default for both the client and the bundled debugclient. As many clients use
; this port number, it is best to leave this setting unchanged.
xdebug.remote_port=9000

; When this setting is set to 1, Xdebug will show a stack trace whenever an
; exception is raised – even if this exception is actually caught.
xdebug.show_exception_trace=0

; When this setting is set to something != 0 Xdebug’s generated stack dumps in
; error situations will also show all variables in the top-most scope. Beware
; that this might generate a lot of information, and is therefore turned off by default.
xdebug.show_local_vars=0

; When this setting is set to something != 0 Xdebug’s human-readable generated
; trace files will show the difference in memory usage between function calls.
; If Xdebug is configured to generate computer-readable trace files then they
; will always show this information.
xdebug.show_mem_delta=0

; The format of the trace file.
; 0 shows a human readable indented trace file with: time index, memory usage,
; memory delta (if the setting xdebug.show_mem_delta is enabled), level,
; function name, function parameters (if the setting xdebug.collect_params
; is enabled, filename and line number.
; 1 writes a computer readable format which has two different records. There
; are different records for entering a stack frame, and leaving a stack frame
xdebug.trace_format=0

; When set to ’1′ the trace files will be appended to, instead of being overwritten
; in subsequent requests.
xdebug.trace_options=0

; The directory where the tracing files will be written to, make sure that the
; user who the PHP will be running as has write permissions to that directory.
xdebug.trace_output_dir=/tmp

; This setting determines the name of the file that is used to dump traces into.
; The setting specifies the format with format specifiers, very similar to
; sprintf() and strftime(). There are several format specifiers that can be used
; to format the file name. The ‘.xt’ extension is always added automatically.
;The possible format specifiers are:
; %c crc32 of the current working directory trace.%c trace.1258863198.xt
; %p pid trace.%p trace.5174.xt
; %r random number trace.%r trace.072db0.xt
; %s script name cachegrind.out.%s cachegrind.out._home_httpd_html_test_xdebug_test_php
; %t timestamp (seconds) trace.%t trace.1179434742.xt
; %u timestamp (microseconds) trace.%u trace.1179434749_642382.xt
; %H $_SERVER['HTTP_HOST'] trace.%H trace.kossu.xt
; %R $_SERVER['REQUEST_URI'] trace.%R trace._test_xdebug_test_php_var=1_var2=2.xt
; %S session_id (from $_COOKIE if set) trace.%S trace.c70c1ec2375af58f74b390bbdd2a679d.xt
; %% literal % trace.%% trace.%%.xt
xdebug.trace_output_name=trace.%c

; Controls the amount of array children and object’s properties are shown when
; variables are displayed with either xdebug_var_dump(), xdebug.show_local_vars
; or through Function Traces. This setting does not have any influence on the
; number of children that is send to the client through the Remote Debugging feature.
xdebug.var_display_max_children=128

; Controls the maximum string length that is shown when variables are displayed
; with either xdebug_var_dump(), xdebug.show_local_vars or through Function Traces.
; This setting does not have any influence on the amount of data that is send to
; the client through the Remote Debugging feature.
xdebug.var_display_max_data=512

; Controls how many nested levels of array elements and object properties are
; when variables are displayed with either xdebug_var_dump(),
; xdebug.show_local_vars or through Function Traces. This setting does not have
; any influence on the depth of children that is send to the client through the
; Remote Debugging feature.
xdebug.var_display_max_depth=3

For installation, just copy and paste it at the end of your php.ini file (or, if you’re on Debian or similarly configured distros, you can save it as separate file in the php ini directory).

Note1: the zend_extension lines at the top of the file are tailored for a php4 install. Make sure to change them to fit your environment.

Posted in HowTo, Information, PHP, Programming, Server

1 Comment »

CodeIgniter 1.7: The URI you submitted has disallowed characters.

on Tuesday 14th July, 2009 Gabe speculated thusly…

I was getting the above error from CodeIgniter after urlencoding some base64 encoded binary data and passing it as a URI parameter to CodeIgniter. I looked in the config.php file and found that percent signs were allowed here:
$config['permitted_uri_chars'] = 'a-z 0-9~%.:_\-';

After some hunting around I found that CodeIgniter is not accepting this in the URI because the actual characters (that are encoded) fall outside the ASCII range. CodeIgniter is actually decoding the characters before testing them. The fix was quite simple, open up libraries/URI.php and go to line 189 where it says:
if ( ! preg_match("|^[".preg_quote($this->config->item('permitted_uri_chars'))."]+$|i", $str))
and change it to:

if ( ! preg_match("|^[".preg_quote($this->config->item('permitted_uri_chars'))."]+$|i", rawurlencode($str)))

Basically wrapping $str in the urlencode() function.

Posted in CodeIgniter, Development, Frameworks, Information, Programming

6 Comments »

Using SSHFS with Bazaar (Bzr) or Git

on Thursday 4th June, 2009 Gabe speculated thusly…

I use SSHFS to mount remote drives locally. The remote computers don’t have Bzr installed, and it’s difficult to get them up to date enough to install the latest Bzr. Therefore, I execute Bzr on my local computer on the mounted filesystem. Using default mount options for SSHFS leads to errors in both Git and Bzr, this is because (I believe) SSHFS doesn’t directly support file renaming. It is possible to avoid these problems by using the -oworkaround=rename switch with the mount command.

As an example my mount command looks similar to this:
sshfs -oworkaround=rename user@computer1:/var/www /media/computer1

Posted in Bazaar, Git, Information, Linux, Operating System, Programming, Revision Control, Server

No Comments »

Apache Virtual Hosts on OS X Leopard

on Sunday 31st May, 2009 Gabe speculated thusly…

If you develop multiple sites and you need virtual hosting functionality, scroll down to the end of the /private/etc/apache2/httpd.conf file and uncomment the following:

# Include /private/etc/apache2/extra/httpd-vhosts.conf

Next, you’ll need to setup whatever virtual hosts you have in the virtual hosts file /private/etc/apache2/extra/httpd-vhosts.conf

You need to make an entry in the httpd-vhosts.conf file like so:

<virtualhost *:80>
   ServerName beta-site-1.com
   ServerAlias www.beta-site-1.com
   ServerAdmin webmaster@beta-site-1.com
   ErrorLog "/private/var/log/apache2/dummy-host2.example.com-error_log"
   CustomLog "/private/var/log/apache2/dummy-host2.example.com-access_log" common

   DocumentRoot "/Library/WebServer/beta-site-1"
   ScriptAlias /cgi-bin/ "/Library/WebServer/beta-site-1/cgi-bin"
   
     Options FollowSymLinks MultiViews Includes
     AllowOverride All
     Order allow,deny
     Allow from all
   
</virtualhost>

The examples provided by Apple in the vhosts file are slightly incorrect and if you use the CustomLog lines as is you will get errors the following errors if you run: apachectl -t -D DUMP_VHOSTS:
Syntax error on line 40 of /private/etc/apache2/extra/httpd-vhosts.conf:
CustomLog takes two or three arguments, a file name, a custom log format string or format name, and an optional "env=" clause (see docs)

This is because
CustomLog "/private/var/log/apache2/dummy-host.example.com-access_log common

Should actually read:
CustomLog "/private/var/log/apache2/dummy-host.example.com-access_log" common

Posted in Development, Frameworks, HowTo, Information, Leopard, Linux, Operating System, OS X, PHP, Server, Ubuntu

2 Comments »

Get PHP MySQL working on Leopard OS X

on Sunday 31st May, 2009 Gabe speculated thusly…

Installing MySQL on OS X has become infinately easier since you can now download an Apple DMG from the MySQL website which takes care of the fine detail.

However, one thing that changed with Leopard is the socket for Mysql. It moved to /private/tmp, so you may need to configure your php.ini file to point it to the new location.

To do so, open the file /private/etc/php.ini, (if no such file exists, then make a copy of /private/etc/php.ini.default and rename it to php.ini) and edit that.

You have two lines to modify:

mysql.default_socket =

becomes:

mysql.default_socket = /private/tmp/mysql.sock

and mysqli.default_socket =

becomes:

mysqli.default_socket = /private/tmp/mysql.sock

Posted in Information, Leopard, MySQL, Operating System, OS X

No Comments »

Change MySQL Root Password

on Sunday 31st May, 2009 Gabe speculated thusly…

Change default password:
$ mysqladmin -u root password NEWPASSWORD

Change existing password:
$ mysqladmin -u root -p'oldpassword' password newpass

Posted in MySQL

No Comments »

MySQL Column Types and Storage Capabilities

on Sunday 24th May, 2009 Gabe speculated thusly…

I am making a note of this for ease of future reference. I can never quite remember whether a medium blob is 16MB or whatever it is, the capabilities are on the MySQL site but I am going to write it in plain english here so I don’t have to keep getting my calculator out to remind me what 2^24 is in terms I am more familiar with…

TinyInt: -128 to 127 (0 to 255 if unsigned).
SmallInt: -32768 to 32767 (0 to 65535 if unsigned).
MediumInt: -8588608 to 8388607 (0 to 16777215 if unsigned).
Int: -2147483648 to 2147483647 (0 to 4294967295 if unsigned).
BigInt: -9223372936854775808 to 9223372036854775807 (0 to18446744073709551615).

Float: 0 and +-1.175494351E-38 to +-3.402823466E+38.
Double: 0 and +-2.2250738585072014-308 to +-1.7976931348623157E+38.
Decimal[(M,D)]: As for DOUBLE but constrained by M and D.

Char(M): M may take any integer value from 0 to 255, with a CHAR(0) column able to store only two values: NULL and ” (empty string), which occupy a single bit.
VarChar(M): 1 to 255 (number of characters to store). Trailing spaces are stripped before storage.

Text type columns do case insensitive comparisons and sorts, whereas blobs are case sensitive.
TinyBlob/TinyText: Max. length 255 characters. Very similar to VarChar but trailing spaces are not stripped before storage.
Text/Blob: Max. length 65535 characters (65KB).
MediumBlob/MediumText: Max. length 6777215 characters (16.8MB).
LongBlob/LongText: Max. length 4294967295 characters (4.3GB).

Enum: One value chosen from up to 65535 possibilities.
Set: Up to 64 values in a given set column.

Date: ’1000-01-01′ to ’9999-12-31′, and ’0000-00-00′.
Time: ‘-838:59:59′ to ’838:59:59′.
DateTime: ’1000-01-01 00:00:00′ to ’9999-12-31 23:59:59′.
Year: 1901 to 2155, and 0000.
TimeStamp: 19700101000000 to sometime in 2037 on current systems.

Posted in Information, MySQL, Programming

No Comments »

An alternative img() helper function for CodeIgniter

on Wednesday 20th May, 2009 Gabe speculated thusly…

It is my opinion that CodeIgniter’s default img() function that comes in the HTML Helper could have been easier to use. As it is if you want to give your image a name css class you have to define an array consisting of these attributes.

That takes up space and makes the code less readable for non-PHP programmers. Since I work closely with a web designer who is good with HTML but gets lost in PHP I want to leverage the pragmatic power of PHP to automate repetitive tasks (like typing out an entire HTML image tag), but at the same time the result needs to be obvious to a non-programmer but also shorter than the HTML equivalent. I feel the img() helper in CodeIgniter falls short of both these requirements.

CodeIgniter’s img() helper also takes a second parameter, a boolean, this decides whether the index.php file is included in the image path, good if you are using a media controller. For 99.9% of my sites I don’t need these. Therefore I wrote my own helper.

The file is called MY_html_helper.php and lives inside the folder system/application/helpers. As with other extensions to the CI core the prefix MY_ is determined in your config file, so change MY_ to whatever it should be. Add the following code to the file:

<?php if (!defined('BASEPATH')) exit('No direct script access allowed');

function img($imgName, $attrs=FALSE)
{
	$CI =& get_instance();

	if (strpos($imgName, 'http') === 0) return;

	$imgPath = $CI->config->item('base_url');

	if ( ! $imgDir = $CI->config->item('image_dir')) $imgDir = 'assets/images/';

	$img = $imgPath.$imgDir.$imgName;

	$str = '<img src="'.$img.'" ';

	if ($attrs) $str .= $attrs." ";

	$str .= "/>";

	return $str;
}

All you have to make sure you do is load the html helper: $this->load->helper('html'); in your controller,
or put it in the array of helpers in system/application/config/autoload.php.

The helper assumes images live in the folder assets/images which lives alongside the system folder. If this is not where you put your images then you can specify an alternative directory in your config.php file. Simply add a line that looks like this to config.php: $config['image_dir'] = 'alternative/path/images/';. Don’t forget the trailing slash at the end. This alternative folder would live at the very top level of your application, alongside index.php.

Using the helper is easy and straightforward: <?=img('example.gif')?>

Will produce <img src="http://mysite.com/assets/images/example.gif" />.

Any additional attributes you want in the html tag can be written as per usual as the second parameter. For example if you want to give the image an alt attribute and class:
<?=img('example.gif', 'class="myclass" alt="Example Image"')?>

Will produce:

<img src="http://mysite.com/assets/images/example.gif" class="myclass" alt="Example Image" />

The only hard part about this is making sure you get the single and double quotes correct in the second parameter.

Cheers

Posted in CodeIgniter, Development, Frameworks, Information, Linux, Operating System, Programming, Server

1 Comment »

HANDY ONE-LINERS FOR RUBY

on Wednesday 15th April, 2009 Gabe speculated thusly…

Taken from http://www.fepus.net/ruby1line.txt

HANDY ONE-LINERS FOR RUBY                             November 16, 2005
compiled by David P Thomas          version 1.0

Latest version of this file can be found at:

http://www.fepus.net/ruby1line.txt

Last Updated: Wed Nov 16 08:35:02 CST 2005

FILE SPACING:

# double space a file
    $  ruby -pe 'puts' < file.txt
# triple space a file
    $  ruby -pe '2.times {puts}' < file.txt
# undo double-spacing (w/ and w/o whitespace in lines)
    $  ruby -lne 'BEGIN{$/="\n\n"}; puts $_' < file.txt
    $  ruby -ne 'BEGIN{$/="\n\n"}; puts $_.chomp' < file.txt
    $  ruby -e 'puts STDIN.readlines.to_s.gsub(/\n\n/, "\n")' < file.txt

NUMBERING:

# number each line of a file (left justified).
    $  ruby -ne 'printf("%-6s%s", $., $_)' < file.txt
# number each line of a file (right justified).
    $  ruby -ne 'printf("%6s%s", $., $_)' < file.txt
# number each line of a file, only print non-blank lines
    $  ruby -e 'while gets; end; puts $.' < file.txt
# count lines (emulates 'wc -l')
    $  ruby -ne 'END {puts $.}' < file.txt
    $  ruby -e 'while gets; end; puts $.' < file.txt

TEXT CONVERSION AND SUBSTITUTION:

# convert DOS newlines (CR/LF) to Unix format (LF)
# - strip newline regardless; re-print with unix EOL
    $  ruby -ne 'BEGIN{$\="\n"}; print $_.chomp' < file.txt

# convert Unix newlines (LF) to DOS format (CR/LF)
# - strip newline regardless; re-print with dos EOL
    $  ruby -ne 'BEGIN{$\="\r\n"}; print $_.chomp' < file.txt

# delete leading whitespace (spaces/tabs/etc) from beginning of each line
    $  ruby -pe 'gsub(/^\s+/, "")' < file.txt

# delete trailing whitespace (spaces/tabs/etc) from end of each line
# - strip newline regardless; replace with default platform record separator
    $  ruby -pe 'gsub(/\s+$/, $/)' < file.txt

# delete BOTH leading and trailing whitespace from each line
    $  ruby -pe 'gsub(/^\s+/, "").gsub(/\s+$/, $/)' < file.txt

# insert 5 blank spaces at the beginning of each line (ie. page offset)
    $  ruby -pe 'gsub(/%/, "   ")' < file.txt
    FAILS! $  ruby -pe 'gsub(/%/, 5.times{putc " "})' < file.txt

# align all text flush right on a 79-column width
    $  ruby -ne 'printf("%79s", $_)' < file.txt

# center all text in middle of 79-column width
    $  ruby -ne 'puts $_.chomp.center(79)' < file.txt
    $  ruby -lne 'puts $_.center(79)' < file.txt

# substitute (find and replace) "foo" with "bar" on each line
    $  ruby -pe 'gsub(/foo/, "bar")' < file.txt

# substitute "foo" with "bar" ONLY for lines which contain "baz"
    $  ruby -pe 'gsub(/foo/, "bar") if $_ =~ /baz/' < file.txt

# substitute "foo" with "bar" EXCEPT for lines which contain "baz"
    $  ruby -pe 'gsub(/foo/, "bar") unless $_ =~ /baz/' < file.txt

# substitute "foo" or "bar" or "baz".... with "baq"
    $  ruby -pe 'gsub(/(foo|bar|baz)/, "baq")' < file.txt

# reverse order of lines (emulates 'tac') IMPROVE
    $  ruby -ne 'BEGIN{@arr=Array.new}; @arr.push $_; END{puts @arr.reverse}' < file.txt

# reverse each character on the line (emulates 'rev')
    $  ruby -ne 'puts $_.chomp.reverse' < file.txt
    $  ruby -lne 'puts $_.reverse' < file.txt

# join pairs of lines side-by-side (like 'paste')
    $  ruby -pe '$_ = $_.chomp + " " + gets if $. % 2' < file.txt

# if a line ends with a backslash, append the next line to it
    $  ruby -pe 'while $_.match(/\\$/); $_ = $_.chomp.chop + gets; end' < file.txt
    $  ruby -e 'puts STDIN.readlines.to_s.gsub(/\\\n/, "")' < file.txt

# if a line begins with an equal sign, append it to the previous line (Unix)
    $  ruby -e 'puts STDIN.readlines.to_s.gsub(/\n=/, "")' < file.txt

# add a blank line every 5 lines (after lines 5, 10, 15, etc)
    $  ruby -pe 'puts if $. % 6 == 0' < file.txt

SELECTIVE PRINTING OF CERTAIN LINES

# print first 10 lines of a file (emulate 'head')
    $  ruby -pe 'exit if $. > 10' < file.txt

# print first line of a file (emulate 'head -1')
    $  ruby -pe 'puts $_; exit' < file.txt

# print the last 10 lines of a file (emulate 'tail'); NOTE reads entire file!
    $  ruby -e 'puts STDIN.readlines.reverse!.slice(0,10).reverse!' < file.txt

# print the last 2 lines of a file (emulate 'tail -2'); NOTE reads entire file!
    $  ruby -e 'puts STDIN.readlines.reverse!.slice(0,2).reverse!' < file.txt

# print the last line of a file (emulates 'tail -1')
    $  ruby -ne 'line = $_; END {puts line}' < file.txt

# print only lines that match a regular expression (emulates 'grep')
    $  ruby -pe 'next unless $_ =~ /regexp/' < file.txt

# print only lines that DO NOT match a regular expression (emulates 'grep')
    $  ruby -pe 'next if $_ =~ /regexp/' < file.txt

# print the line immediately before a regexp, but not the regex matching line
    $  ruby -ne 'puts @prev if $_ =~ /regex/; @prev = $_;' < file.txt

# print the line immediately after a regexp, but not the regex matching line
    $  ruby -ne 'puts $_ if @prev =~ /regex/; @prev = $_;' < file.txt

# grep for foo AND bar AND baz (in any order)
    $  ruby -pe 'next unless $_ =~ /foo/ && $_ =~ /bar/ && $_ =~ /baz/' < file.txt

# grep for foo AND bar AND baz (in order)
    $  ruby -pe 'next unless $_ =~ /foo.*bar.*baz/' < file.txt

# grep for foo OR bar OR baz
    $  ruby -pe 'next unless $_ =~ /(foo|bar|baz)/' < file.txt

# print paragraph if it contains regexp; blank lines separate paragraphs
    $  ruby -ne 'BEGIN{$/="\n\n"}; print $_ if $_ =~ /regexp/' < file.txt

# print paragraph if it contains foo AND bar AND baz (in any order); blank lines separate paragraphs
    $  ruby -ne 'BEGIN{$/="\n\n"}; print $_ if $_ =~ /foo/ && $_ =~ /bar/ && $_ =~ /baz/' < file.txt

# print paragraph if it contains foo AND bar AND baz (in order); blank lines separate paragraphs
    $  ruby -ne 'BEGIN{$/="\n\n"}; print $_ if $_ =~ /(foo.*bar.*baz)/' < file.txt

# print paragraph if it contains foo OR bar OR baz; blank lines separate paragraphs
    $  ruby -ne 'BEGIN{$/="\n\n"}; print $_ if $_ =~ /(foo|bar|baz)/' < file.txt

# print only lines of 65 characters or greater
    $  ruby -pe 'next unless $_.chomp.length >= 65' < file.txt
    $  ruby -lpe 'next unless $_.length >= 65' < file.txt

# print only lines of 65 characters or less
    $  ruby -pe 'next unless $_.chomp.length < 65' < file.txt
    $  ruby -lpe 'next unless $_.length < 65' < file.txt

# print section of file from regex to end of file
    $  ruby -pe '@found=true if $_ =~ /regex/; next unless @found' < file.txt

# print section of file based on line numbers (eg. lines 2-7 inclusive)
    $  ruby -pe 'next unless $. >= 2 && $. < = 7' < file.txt

# print line number 52
    $  ruby -pe 'next unless $. == 52' < file.txt

# print every 3rd line starting at line 4
    $  ruby -pe 'next unless $. >= 4 && $. % 3 == 0' < file.txt

# print section of file between two regular expressions, /foo/ and /bar/
    $  ruby -ne '@found=true if $_ =~ /foo/; next unless @found; puts $_; exit if $_ =~ /bar/' < file.txt

SELECTIVE DELETION OF CERTAIN LINES

# print all of file except between two regular expressions, /foo/ and /bar/
    $  ruby -ne '@found = true if $_ =~ /foo/; puts $_ unless @found; @found = false if $_ =~ /bar/' < file.txt

# print file and remove duplicate, consecutive lines from a file (emulates 'uniq')
    $  ruby -ne 'puts $_ unless $_ == @prev; @prev = $_' < file.txt

# print file and remove duplicate, non-consecutive lines from a file (careful of memory!)
    $  ruby -e 'puts STDIN.readlines.sort.uniq!.to_s' < file.txt

# print file except for first 10 lines
    $  ruby -pe 'next if $. <= 10' < file.txt

# print file except for last line
    $  ruby -e 'lines=STDIN.readlines; puts lines[0,lines.size-1]' < file.txt

# print file except for last 2 lines
    $  ruby -e 'lines=STDIN.readlines; puts lines[0,lines.size-2]' < file.txt

# print file except for last 10 lines
    $  ruby -e 'lines=STDIN.readlines; puts lines[0,lines.size-10]' < file.txt

# print file except for every 8th line
    $  ruby -pe 'next if $. % 8 == 0' < file.txt

# print file except for blank lines
    $  ruby -pe 'next if $_ =~ /^\s*$/' < file.txt

# delete all consecutive blank lines from a file except the first
    $  ruby -e 'BEGIN{$/=nil}; puts STDIN.readlines.to_s.gsub(/\n(\n)+/, "\n\n")' < file.txt

# delete all consecutive blank lines from a file except for the first 2
    $  ruby -e 'BEGIN{$/=nil}; puts STDIN.readlines.to_s.gsub(/\n(\n)+/, "\n\n")' < file.txt

# delete all leading blank lines at top of file
    $  ruby -pe '@lineFound = true if $_ !~ /^\s*$/; next if !@lineFound' < file.txt

If you have any additional scripts to contribute or if you find errors
in this document, please send an e-mail to the compiler.  Indicate the
version of ruby you used, the operating system it was compiled for, and
the nature of the problem.  Various scripts in this file were written or
contributed by:

 David P Thomas     # author of this document

Tue Jun 26 18:17:36 CDT 2007
 * Thanks to Taylor Carpenter  for feedback on improving redirection format.

Posted in Programming, Ruby

No Comments »