dragffy.com

if (typeof yourFunctionName == 'function')
{
    yourFunctionName();
}

Taken from http://www.fepus.net/ruby1line.txt

HANDY ONE-LINERS FOR RUBY                             November 16, 2005
compiled by David P Thomas          version 1.0

Latest version of this file can be found at:

http://www.fepus.net/ruby1line.txt

Last Updated: Wed Nov 16 08:35:02 CST 2005

FILE SPACING:

# double space a file
    $  ruby -pe 'puts' < file.txt
# triple space a file
    $  ruby -pe '2.times {puts}' < file.txt
# undo double-spacing (w/ and w/o whitespace in lines)
    $  ruby -lne 'BEGIN{$/="\n\n"}; puts $_' < file.txt
    $  ruby -ne 'BEGIN{$/="\n\n"}; puts $_.chomp' < file.txt
    $  ruby -e 'puts STDIN.readlines.to_s.gsub(/\n\n/, "\n")' < file.txt

NUMBERING:

# number each line of a file (left justified).
    $  ruby -ne 'printf("%-6s%s", $., $_)' < file.txt
# number each line of a file (right justified).
    $  ruby -ne 'printf("%6s%s", $., $_)' < file.txt
# number each line of a file, only print non-blank lines
    $  ruby -e 'while gets; end; puts $.' < file.txt
# count lines (emulates 'wc -l')
    $  ruby -ne 'END {puts $.}' < file.txt
    $  ruby -e 'while gets; end; puts $.' < file.txt

TEXT CONVERSION AND SUBSTITUTION:

# convert DOS newlines (CR/LF) to Unix format (LF)
# - strip newline regardless; re-print with unix EOL
    $  ruby -ne 'BEGIN{$\="\n"}; print $_.chomp' < file.txt

# convert Unix newlines (LF) to DOS format (CR/LF)
# - strip newline regardless; re-print with dos EOL
    $  ruby -ne 'BEGIN{$\="\r\n"}; print $_.chomp' < file.txt

# delete leading whitespace (spaces/tabs/etc) from beginning of each line
    $  ruby -pe 'gsub(/^\s+/, "")' < file.txt

# delete trailing whitespace (spaces/tabs/etc) from end of each line
# - strip newline regardless; replace with default platform record separator
    $  ruby -pe 'gsub(/\s+$/, $/)' < file.txt

# delete BOTH leading and trailing whitespace from each line
    $  ruby -pe 'gsub(/^\s+/, "").gsub(/\s+$/, $/)' < file.txt

# insert 5 blank spaces at the beginning of each line (ie. page offset)
    $  ruby -pe 'gsub(/%/, "   ")' < file.txt
    FAILS! $  ruby -pe 'gsub(/%/, 5.times{putc " "})' < file.txt

# align all text flush right on a 79-column width
    $  ruby -ne 'printf("%79s", $_)' < file.txt

# center all text in middle of 79-column width
    $  ruby -ne 'puts $_.chomp.center(79)' < file.txt
    $  ruby -lne 'puts $_.center(79)' < file.txt

# substitute (find and replace) "foo" with "bar" on each line
    $  ruby -pe 'gsub(/foo/, "bar")' < file.txt

# substitute "foo" with "bar" ONLY for lines which contain "baz"
    $  ruby -pe 'gsub(/foo/, "bar") if $_ =~ /baz/' < file.txt

# substitute "foo" with "bar" EXCEPT for lines which contain "baz"
    $  ruby -pe 'gsub(/foo/, "bar") unless $_ =~ /baz/' < file.txt

# substitute "foo" or "bar" or "baz".... with "baq"
    $  ruby -pe 'gsub(/(foo|bar|baz)/, "baq")' < file.txt

# reverse order of lines (emulates 'tac') IMPROVE
    $  ruby -ne 'BEGIN{@arr=Array.new}; @arr.push $_; END{puts @arr.reverse}' < file.txt

# reverse each character on the line (emulates 'rev')
    $  ruby -ne 'puts $_.chomp.reverse' < file.txt
    $  ruby -lne 'puts $_.reverse' < file.txt

# join pairs of lines side-by-side (like 'paste')
    $  ruby -pe '$_ = $_.chomp + " " + gets if $. % 2' < file.txt

# if a line ends with a backslash, append the next line to it
    $  ruby -pe 'while $_.match(/\\$/); $_ = $_.chomp.chop + gets; end' < file.txt
    $  ruby -e 'puts STDIN.readlines.to_s.gsub(/\\\n/, "")' < file.txt

# if a line begins with an equal sign, append it to the previous line (Unix)
    $  ruby -e 'puts STDIN.readlines.to_s.gsub(/\n=/, "")' < file.txt

# add a blank line every 5 lines (after lines 5, 10, 15, etc)
    $  ruby -pe 'puts if $. % 6 == 0' < file.txt

SELECTIVE PRINTING OF CERTAIN LINES

# print first 10 lines of a file (emulate 'head')
    $  ruby -pe 'exit if $. > 10' < file.txt

# print first line of a file (emulate 'head -1')
    $  ruby -pe 'puts $_; exit' < file.txt

# print the last 10 lines of a file (emulate 'tail'); NOTE reads entire file!
    $  ruby -e 'puts STDIN.readlines.reverse!.slice(0,10).reverse!' < file.txt

# print the last 2 lines of a file (emulate 'tail -2'); NOTE reads entire file!
    $  ruby -e 'puts STDIN.readlines.reverse!.slice(0,2).reverse!' < file.txt

# print the last line of a file (emulates 'tail -1')
    $  ruby -ne 'line = $_; END {puts line}' < file.txt

# print only lines that match a regular expression (emulates 'grep')
    $  ruby -pe 'next unless $_ =~ /regexp/' < file.txt

# print only lines that DO NOT match a regular expression (emulates 'grep')
    $  ruby -pe 'next if $_ =~ /regexp/' < file.txt

# print the line immediately before a regexp, but not the regex matching line
    $  ruby -ne 'puts @prev if $_ =~ /regex/; @prev = $_;' < file.txt

# print the line immediately after a regexp, but not the regex matching line
    $  ruby -ne 'puts $_ if @prev =~ /regex/; @prev = $_;' < file.txt

# grep for foo AND bar AND baz (in any order)
    $  ruby -pe 'next unless $_ =~ /foo/ && $_ =~ /bar/ && $_ =~ /baz/' < file.txt

# grep for foo AND bar AND baz (in order)
    $  ruby -pe 'next unless $_ =~ /foo.*bar.*baz/' < file.txt

# grep for foo OR bar OR baz
    $  ruby -pe 'next unless $_ =~ /(foo|bar|baz)/' < file.txt

# print paragraph if it contains regexp; blank lines separate paragraphs
    $  ruby -ne 'BEGIN{$/="\n\n"}; print $_ if $_ =~ /regexp/' < file.txt

# print paragraph if it contains foo AND bar AND baz (in any order); blank lines separate paragraphs
    $  ruby -ne 'BEGIN{$/="\n\n"}; print $_ if $_ =~ /foo/ && $_ =~ /bar/ && $_ =~ /baz/' < file.txt

# print paragraph if it contains foo AND bar AND baz (in order); blank lines separate paragraphs
    $  ruby -ne 'BEGIN{$/="\n\n"}; print $_ if $_ =~ /(foo.*bar.*baz)/' < file.txt

# print paragraph if it contains foo OR bar OR baz; blank lines separate paragraphs
    $  ruby -ne 'BEGIN{$/="\n\n"}; print $_ if $_ =~ /(foo|bar|baz)/' < file.txt

# print only lines of 65 characters or greater
    $  ruby -pe 'next unless $_.chomp.length >= 65' < file.txt
    $  ruby -lpe 'next unless $_.length >= 65' < file.txt

# print only lines of 65 characters or less
    $  ruby -pe 'next unless $_.chomp.length < 65' < file.txt
    $  ruby -lpe 'next unless $_.length < 65' < file.txt

# print section of file from regex to end of file
    $  ruby -pe '@found=true if $_ =~ /regex/; next unless @found' < file.txt

# print section of file based on line numbers (eg. lines 2-7 inclusive)
    $  ruby -pe 'next unless $. >= 2 && $. < = 7' < file.txt

# print line number 52
    $  ruby -pe 'next unless $. == 52' < file.txt

# print every 3rd line starting at line 4
    $  ruby -pe 'next unless $. >= 4 && $. % 3 == 0' < file.txt

# print section of file between two regular expressions, /foo/ and /bar/
    $  ruby -ne '@found=true if $_ =~ /foo/; next unless @found; puts $_; exit if $_ =~ /bar/' < file.txt

SELECTIVE DELETION OF CERTAIN LINES

# print all of file except between two regular expressions, /foo/ and /bar/
    $  ruby -ne '@found = true if $_ =~ /foo/; puts $_ unless @found; @found = false if $_ =~ /bar/' < file.txt

# print file and remove duplicate, consecutive lines from a file (emulates 'uniq')
    $  ruby -ne 'puts $_ unless $_ == @prev; @prev = $_' < file.txt

# print file and remove duplicate, non-consecutive lines from a file (careful of memory!)
    $  ruby -e 'puts STDIN.readlines.sort.uniq!.to_s' < file.txt

# print file except for first 10 lines
    $  ruby -pe 'next if $. <= 10' < file.txt

# print file except for last line
    $  ruby -e 'lines=STDIN.readlines; puts lines[0,lines.size-1]' < file.txt

# print file except for last 2 lines
    $  ruby -e 'lines=STDIN.readlines; puts lines[0,lines.size-2]' < file.txt

# print file except for last 10 lines
    $  ruby -e 'lines=STDIN.readlines; puts lines[0,lines.size-10]' < file.txt

# print file except for every 8th line
    $  ruby -pe 'next if $. % 8 == 0' < file.txt

# print file except for blank lines
    $  ruby -pe 'next if $_ =~ /^\s*$/' < file.txt

# delete all consecutive blank lines from a file except the first
    $  ruby -e 'BEGIN{$/=nil}; puts STDIN.readlines.to_s.gsub(/\n(\n)+/, "\n\n")' < file.txt

# delete all consecutive blank lines from a file except for the first 2
    $  ruby -e 'BEGIN{$/=nil}; puts STDIN.readlines.to_s.gsub(/\n(\n)+/, "\n\n")' < file.txt

# delete all leading blank lines at top of file
    $  ruby -pe '@lineFound = true if $_ !~ /^\s*$/; next if !@lineFound' < file.txt

If you have any additional scripts to contribute or if you find errors
in this document, please send an e-mail to the compiler.  Indicate the
version of ruby you used, the operating system it was compiled for, and
the nature of the problem.  Various scripts in this file were written or
contributed by:

 David P Thomas     # author of this document

Tue Jun 26 18:17:36 CDT 2007
 * Thanks to Taylor Carpenter  for feedback on improving redirection format.

I was recently trying to get a Ruby script working on Ubuntu. This script required Hpricot and using Ruby Gems to install Hpricot always resulted in an error:

 gem install hpricot --remote
ERROR:  While executing gem ... (Gem::FilePermissionError)
    You don't have write permissions into the /var/lib/gems/1.8 directory.
gabriel@windsor-telecom-2874:~/Music$ sudo gem install hpricot --remote
Building native extensions.  This could take a while...
ERROR:  Error installing hpricot:
	ERROR: Failed to build gem native extension.

/usr/bin/ruby1.8 extconf.rb install hpricot --remote
extconf.rb:1:in `require': no such file to load -- mkmf (LoadError)
	from extconf.rb:1

Gem files will remain installed in /var/lib/gems/1.8/gems/hpricot-0.7 for inspection.
Results logged to /var/lib/gems/1.8/gems/hpricot-0.7/ext/hpricot_scan/gem_make.out

The answer was simple:
sudo aptitude install ruby-dev

Just install the Ruby Dev package, this will also allow you to install lots of other gems such as SQLite3, etc.

find ./ -type f -name '*.*bk' -exec rm {} \;

Always forget how to do this:
find ./ -type f -exec sed -i.bk ’s/string1/string2/g’ {} \;

You will get an internal server error unless you set:
magic_quotes_gpc = Off
in your php.ini file.

Modern web applications almost all make use of AJAX to enhance their user appeal. However, AJAX can be hard to debug, especially when all the traditional tools do little to help. When doing standard PHP coding CodeIgniter (CI) offers a profiler which will be appended to the bottom of generated pages. This gives you information on the request that was recently processed, including any POST or GET values passed to the script, how long execution took and how long any SQL queries took as well as what those queries actually were. During development I find this information indispensable to make sure my SQL queries are correct and things are working as I want.

However, when coding up AJAX aspects of the application this doesn’t work very well, any page fragments fed back to the JavaScript will have the profiler stuck at the bottom, and since these fragments are usually inserted inside DIVs the profiler cannot be read in it’s entirety, or it breaks the layout, or more likely, both of the above. Therefore I set out to find a way of making it AJAX compatible, so that it would always end up at the bottom of the page, even when it was originally returned appended to a page fragment. In the end this turned out easier than anticipated.

One small change needs to be made to your footer, you need to create an empty div with an ID of “debug”, this div will hold the profiler, so place the div where you want the profiler. The advantage with this is that you cannot see it when it is empty so it could even be left in for production code. The div looks like this:
<div id="debug"></div>

Now we need to extend, or rather override the CI profiler so create the file system/application/libraries/MY_Profiler.php

Replace MY_ with the value of $config['subclass_prefix'] found in your config file, by default it is MY_.

Next paste the following code in to it. Remember to change the name of the class as dictated by the filename you use, also change the name of the constructor function too. You can also change the script line to load the jQuery library from one of your servers, or even rewrite the entire script section to use any JavaScript you like.

<?php if (!defined('BASEPATH')) exit('No direct script access allowed');

class MY_Profiler extends CI_Profiler
{
    function MY_profiler()
    {
        parent::CI_Profiler();
    }

    function run()
    {
            $output = <<<ENDJS
<script src="http://code.jquery.com/jquery-latest.js" />
<script type="text/javascript" language="javascript" charset="utf-8">
// < ![CDATA[
    $(document).ready(function() {
        var html = $('#codeigniter_profiler').clone();
        $('#codeigniter_profiler').remove();
        $('#debug').hide().empty().append(html).fadeIn('slow');
    });
// ]]>
</script>
ENDJS;
            $output .= "<div id='codeigniter_profiler' style='font-size: 0.7em; clear:both;background-color:#fff;padding:10px;'>";
            $output .= $this->_compile_uri_string();
            $output .= $this->_compile_controller_info();
            $output .= $this->_compile_memory_usage();
            $output .= $this->_compile_benchmarks();
            $output .= $this->_compile_get();
            $output .= $this->_compile_post();
            $output .= $this->_compile_queries();
            $output .= '</div>';
            return $output;
    }
}

If you have already loaded a jQuery library in your header somewhere then make sure this ajax fragment doesn’t load jQuery again – otherwise all your event bindings and everything else will fail. If this is the case simply remove the line:

<script src="http://code.jquery.com/jquery-latest.js" />

From the above code.

That should be it, hit me back if you have any feedback, questions, or need some help.

Last week I setup MySQL replication for a database with two slaves. I wanted to see how resilient it was, and one of the tests I wanted to perform was to create a record on the master and then interrupt the replication of the same record on the slave. This would allow me to see whether MySQL could recover from such an error.

So I thought the easiest way of doing this would be to to insert a large 700mb
file in to MySQL – to give me enough time to interrupt a slave before
replication had completed. Thus being able to test MySQL’s resilience to
loss of connectivity during UPDATE/INSERT queries.

I have never used BLOB columns before so my quest took my on a journey during which I learned a lot about these blobs. Now, even though a MySQL long blob can hold 4GB of data I have encountered
problem after problem dealing with it:

PHP scripts are limited to 32MB memory maximum, this can be upped, but
is not scalable and not a good solution. It means that you cannot simply
read the data of file and perform an INSERT since the file data is
stored in PHP’s memory first. This meant the development of a PHP script
that could read chunks (just a meg or so) from a file and perform UPDATE
queries along with MySQL CONCAT function to effectively append each
chunk.

CONCAT, as I eventually found out, returns NULL if any of it’s arguments
are NULL. In my case when attempting to UPDATE a row with additional
chunks the resulting blob size was always zero. This was because the
very first time CONCAT is used the blob field is empty (NULL), therefore
CONCAT always returned NULL – and so, the data in the blob column never
grew. The query looked like this:
UPDATE `$table` SET `data` = CONCAT(data, '{mysqli_real_escape_string($buffer)}') WHERE `id`=$id";

I then combined CONCAT with COALESCE which will return an alternative
value that you specify in the eventuality the first argument is NULL. By
combining these two I could overcome the above problem.
Resulting query structure:
UPDATE `$table` SET `data` = COALESCE(CONCAT(data, '{mysqli_real_escape_string($buffer)}'), '') WHERE `id`=$id";

Great, now we’re finally adding our chunks on. Then my computer ran out
of disc space – the reason: 7 gigs of MySQL replication logs. Took a
while to find that out. Fixed.

Next problem, although I was certain (via in-depth PHP debugging) that
PHP was correctly reading chunks from the file in a consecutive order,
of the correct size, and the SQL query was being properly executed
without any sly errors the blob column would never end up growing above
12MB (according the PHP MyAdmin). Through even more intensive debugging
I found that after concatenating each chunk the blob would grow by the
correct amount up to 16MB, then it would zero and start again. The
result of pushing a 700MB ISO in to it would always be less 16MB.

Working from a hunch I upped the max_packet_size in my.conf from 16MB to
36MB, the result was as above but this time the blob would grow to 32MB
before zeroing itself.

I then found this MySQL bug report:
http://bugs.mysql.com/bug.php?id=22853

It is considered a bug simply because by upping max_packet_size to
something like 4GB or anything substantial leaves the MySQL server open
to network DoS attacks. There is no work around. A fix is scheduled for
version 6.

Anyway, since we don’t care about security I maxed out the packet_size,
and my chunking script worked, but very quickly I noticed a huge amount of
slowdown once the blob had grown to just a few tens of megs. In fact it
took about 20 minutes to get to 200MB. Without supporting my theory with
any evidence I reckon the CONCAT function reads all the data out of the
database in to memory, does it’s stuff and then stuffs it back in to the
database. The result is an increasingly slow query.

It would seem that although MySQL supports blobs of about 4GB that you
can never get that size unless you:

1. Change the max_packet_size to a stupidly high number, leaving your
server open to DoS attacks so then you can just do a single INSERT. Using CONCAT
is out since it would take forever.

2. Change the memory limit of PHP scripts to be greater than 4GB.

3. Have absolutely bags of RAM since three copies of the file are stored
in RAM (certainly two copies). Since you cannot chunk the file and
perform CONCATS PHP will read the entire file in to it’s memory. This
will all then be buffered by the MySQL driver. Then this is passed to
the database. Therefore if you wish to insert a 4 GB file you will
require >12GB of RAM.

One work around is to chunk the file as before but each chunk is
represented by a single row. These can then be linked together by giving
each row a master file ID, so that all necessary rows can be retrieved.

For my purposes this doesn’t help, I needed queries that took a while,
inserting rows like this will take but a fraction of a second each.

You can use cURL for normal copies, but it will also resume interrupted transfers:
curl -C - -O file:///foo/bar.dat

So we all know we can use uname -a to find out the kernel version, but which kernel matches up with which Ubuntu release? I for one cannot remember all the permutations. If you want to get the name of the version of Ubuntu that you are running an alternative command can help:
lsb_release -a

Which gives me the output:

No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 8.04.1
Release: 8.04
Codename: hardy


Easy! I’m running Hardy Heron!