Bit Blender

Making an executable Ruby archive distribution

2011-02-14T21:07:00.006-05:00

As a follow-up to my previous post, you can even make a single executable file of your Ruby archive. It relies on the fact that zip files are pretty resilient to extra garbage data in the beginning. Let's say I have two files, lib/greetings.rb and bin/hello.rb. greetings.rb defines a make_hello() function that is used by hello.rb. Here's hello.rb:

require "lib/greetings"

puts "Enter your name:"
name = gets.strip

puts "\n" + make_hello(name)

You can run it like this:

$ ruby bin/hello.rb 
Enter your name:
Steve

Hello, Steve!
$

First, we make a zip file that includes all the required source files:

$ zip -r hello.zip bin lib
  adding: bin/ (stored 0%)
  adding: bin/hello.rb (deflated 11%)
  adding: lib/ (stored 0%)
  adding: lib/greetings.rb (deflated 7%)
$ unzip -l hello.zip 
Archive:  hello.zip
  Length     Date   Time    Name
 --------    ----   ----    ----
        0  02-14-11 21:01   bin/
       98  02-14-11 21:01   bin/hello.rb
        0  02-14-11 20:59   lib/
       46  02-14-11 20:59   lib/greetings.rb
 --------                   -------
      144                   4 files

Now we need a ruby header for the zip file, which we'll put into header.rb:

#!/usr/bin/ruby -rubygems -x

require "zip/ziprequire"
$:.push $0
require "bin/hello.rb"
__END__

It runs the ruby interpreter, which then loads zip/ziprequire, adds itself (which will be the zip file) to the path, and then requires our main file, bin/hello.rb. We now prepend this header to the zip file using cat header.rb hello.zip > hello_tmp.zip. If you now try to run unzip -l on this file, you'll get:

$ unzip -l hello_tmp.zip 
Archive:  hello_tmp.zip
warning [hello_tmp.zip]:  97 extra bytes at beginning or within zipfile
  (attempting to process anyway)
  Length     Date   Time    Name
 --------    ----   ----    ----
        0  02-14-11 21:01   bin/
       98  02-14-11 21:01   bin/hello.rb
        0  02-14-11 20:59   lib/
       46  02-14-11 20:59   lib/greetings.rb
 --------                   -------
      144                   4 files

We need to fix this zip file to make it valid. Thankfully zip has an option for this:


$ zip --fix hello_tmp.zip --out hellox.zip
Fix archive (-F) - assume mostly intact archive
Zip entry offsets appear off by 97 bytes - correcting...
 copying: bin/
 copying: bin/hello.rb
 copying: lib/
 copying: lib/greetings.rb

We now have a valid zip archive in hellox.zip, we just need to make it executable by running chmod +x hellox.zip. Now your whole application is a single executable file that you can run:


$ ./hellox.zip 
Enter your name:
Mike

Hello, Mike!

Distributing a Ruby application as an archive

2011-02-08T22:15:00.004-05:00

Let's say you have a Ruby application you've written, and it consists of multiple files that you require inside your code. You want to run this application on some remote machines. To make it easier to deploy this application, you want to distribute it as a single file (archive.) This is possible with the rubyzip gem. Let's say your main application file (myapp.rb) looks like this:

require "lib/mylib"
require "lib/otherlib"
require "vendorlib/something"

# Do stuff.

Normally you might run your application with ruby ./myapp.rb. To run it on another machine, we can use the zip/ziprequire library in rubyzip. First, make a zip file containing all your application files:

zip -r myapp.zip myapp.rb lib vendorlib

Copy myapp.zip to the remote machine, and you can run it like this:

ruby -rubygems -Imyapp.zip -e 'require "zip/ziprequire"' -e 'require "myapp"'

See rubyzip documentation, specifically ziprequire.rb for more information.

Using FileMerge with Mercurial

2011-01-11T09:42:00.002-05:00

Mac OS comes with a GUI merge tool called FileMerge. This can be used for merges in Mercurial - Mercurial will do this if its internal merge fails. The binary for FileMerge (opendiff) cannot be used as is, so we need to create a small shell script. You can put this script anywhere in your path, and call it hopendiff:

#!/bin/sh
`/usr/bin/opendiff "$@"`

Then modify your ~/.hgrc and add:

[extensions]
hgext.extdiff =

[extdiff]
cmd.opendiff = hopendiff

[merge-tools]
filemerge.executable = hopendiff
filemerge.args = $local $other -ancestor $base -merge $output

If you already have some of those sections (like [extensions]), then just add the corresponding lines to those sections.

That's it. When you do a merge in Mercurial, it'll open FileMerge if the internal merge fails. You can also use FileMerge for normal diffs by using hg opendiff instead of hg diff.

Binary and character set safe MySQL dump and restore

2011-01-06T17:44:00.002-05:00

To dump some MySQL databases in a binary-safe way:

mysqldump -uroot -p -hdbhost \ 
  --skip-extended-insert \ 
  --default-character-set=binary \ 
  --databases dbone dbtwo dbthree \ 
  --add-drop-database --master-data=2 | gzip > /tmp/dump.sql.gz

Replace --master-data=2 with --lock-all-tables if dumping from a slave, and record the output of SHOW SLAVE STATUS while everything is locked.

To import this dump:

zcat /tmp/dump.sql.gz | mysql -uroot -p -hotherdbhost \ 
   --unbuffered --batch --default-character-set=binary

To set up replication:

Use the Relay_Master_Log_File:Exec_Master_Log_Pos values from SHOW SLAVE STATUS that you recorded. See this page for explanation:

http://www.mysqlperformanceblog.com/2008/07/07/how-show-slave-status-relates-to-change-master-to/

For example:

CHANGE MASTER TO MASTER_HOST = 'abc123.dklfjalddkj.com',
  MASTER_USER = 'rep', MASTER_PASSWORD = 'CHANGEME',
  MASTER_LOG_FILE = 'abc123-bin.003292',
  MASTER_LOG_POS = 283072761;

Microsoft Dynamics CRM and PHP

2010-09-02T13:40:00.014-04:00

I needed to use PHP for two things:

Given a lead's email address, retrieve the name, title, phone and email address of this lead's owner.
Insert a new lead into the system.

I spent a lot of time searching the web, and saw many posts of people being unsuccessful with using SoapClient, so I didn't even bother trying. The first order of business was to figure out how to login using IFD (Internet Facing Deployment.) I found this page, which explained how to authenticate. It didn't work, but eventually we figured out that our IFD was not configured properly. In the IFD configuration tool, you can't just leave the "local" subnets blank. At least one IP range must be added, even if you want everything to use IFD. We added 127.0.0.1-127.0.0.255 as the "local" range.

I wrote two classes to handle the CRM. These are in the mscrm.php file.

<?
// vim: filetype=php

class MSCRMServiceException extends Exception {}
class MSCRMLoginFailed extends MSCRMServiceException {}

class MSCRMService {
  protected $host, $org, $domain, $user, $password, $curl_handle, $ticket;

  function __construct($crm_host, $organization, $domain, $user, $password) {
    $this->host = $crm_host;
    $this->org = $organization;
    $this->domain = $domain;
    $this->user = $user;
    $this->password = $password;
    $this->init_curl();
    $this->login();
  }

  function __destruct() {
    curl_close($this->curl_handle);
  }

  public function domain_user() {
    return $this->domain . '\\' . $this->user;
  }

  protected function init_curl() {
    $this->curl_handle = curl_init();
    curl_setopt($this->curl_handle, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($this->curl_handle, CURLOPT_CONNECTTIMEOUT, 10);
    curl_setopt($this->curl_handle, CURLOPT_TIMEOUT, 30);
    curl_setopt($this->curl_handle, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
  }

  protected function login() {
    $request = '<?xml version="1.0" encoding="utf-8"?' . '>
      <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
        <soap:Body>
          <Execute xmlns="http://schemas.microsoft.com/crm/2007/CrmDiscoveryService">
            <Request xsi:type="RetrieveCrmTicketRequest">
              <OrganizationName>' . $this->org . '</OrganizationName>
              <UserId>' . $this->domain_user() . '</UserId>
              <Password>' . $this->password . '</Password>
            </Request>
          </Execute>
        </soap:Body>
      </soap:Envelope>';
    $headers = array(
      'Host: ' . $this->host,
      'Connection: Keep-Alive',
      'SOAPAction: "http://schemas.microsoft.com/crm/2007/CrmDiscoveryService/Execute"',
      'Content-type: text/xml; charset="utf-8"',
      'Content-length: ' . strlen($request)
    );
    curl_setopt($this->curl_handle, CURLOPT_URL, 'http://' . $this->host . '/MSCRMServices/2007/SPLA/CrmDiscoveryService.asmx');
    curl_setopt($this->curl_handle, CURLOPT_HTTPHEADER, $headers);
    curl_setopt($this->curl_handle, CURLOPT_POST, true);
    curl_setopt($this->curl_handle, CURLOPT_POSTFIELDS, $request);

    $response = curl_exec($this->curl_handle);
    $matches = array();
    if(preg_match('/<CrmTicket>([^<]*)<\/CrmTicket>/', $response, $matches)) {
      $this->ticket = $matches[1];
    } else {
      throw new MSCRMLoginFailed;
    }
  }

  public function xml_request($xml, $service, $action) {
    $request = '<?xml version="1.0" encoding="utf-8"?' . '>
      <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
        <soap:Header>
          <CrmAuthenticationToken xmlns="http://schemas.microsoft.com/crm/2007/WebServices">
            <AuthenticationType xmlns="http://schemas.microsoft.com/crm/2007/CoreTypes">2</AuthenticationType>
            <OrganizationName xmlns="http://schemas.microsoft.com/crm/2007/CoreTypes">' . $this->org . '</OrganizationName>
          </CrmAuthenticationToken>
        </soap:Header>
        <soap:Body>
          <' . $action . ' xmlns="http://schemas.microsoft.com/crm/2007/WebServices">
            ' . $xml . '
          </' . $action . '>
        </soap:Body>
      </soap:Envelope>';
    $headers = array(
      'Host: ' . $this->host,
      'Connection: Keep-Alive',
      'SOAPAction: "http://schemas.microsoft.com/crm/2007/WebServices/' . $action . '"',
      'Content-type: text/xml; charset="utf-8"',
      'Content-length: ' . strlen($request)
    );
    curl_setopt($this->curl_handle, CURLOPT_URL, 'http://' . $this->host . '/MSCrmServices/2007/' . $service . '.asmx ');
    curl_setopt($this->curl_handle, CURLOPT_HTTPHEADER, $headers);
    curl_setopt($this->curl_handle, CURLOPT_POST, true);
    curl_setopt($this->curl_handle, CURLOPT_POSTFIELDS, $request);
    // Ticket goes into a cookie.
    curl_setopt($this->curl_handle, CURLOPT_COOKIE, 'MSCRMSession=ticket=' . $this->ticket);
    $response = curl_exec($this->curl_handle);
    return $response;
  }

  // Retrieves given $attributes for given $entity where $condition_attribute equals $condition_string.
  public function retrieve_multiple($entity, $attributes, $condition_attribute, $condition_string) {
    $attributes_xml = "";
    foreach($attributes as $attribute) {
      $attributes_xml .= "<q1:Attribute>" . htmlspecialchars($attribute) . "</q1:Attribute>\n";
    }
    $xml = '
      <query xmlns:q1="http://schemas.microsoft.com/crm/2006/Query" xsi:type="q1:QueryExpression">
        <q1:EntityName>' . htmlspecialchars($entity) . '</q1:EntityName>
        <q1:ColumnSet xsi:type="q1:ColumnSet">
          <q1:Attributes>
            ' . $attributes_xml . '
          </q1:Attributes>
        </q1:ColumnSet>
        <q1:Distinct>false</q1:Distinct>
        <q1:Criteria>
          <q1:FilterOperator>And</q1:FilterOperator>
          <q1:Conditions>
            <q1:Condition>
              <q1:AttributeName>' . htmlspecialchars($condition_attribute) . '</q1:AttributeName>
              <q1:Operator>Equal</q1:Operator>
              <q1:Values>
                <q1:Value xsi:type="xsd:string">' . htmlspecialchars($condition_string) . '</q1:Value>
              </q1:Values>
            </q1:Condition>
          </q1:Conditions>
        </q1:Criteria>
      </query>
    ';
    $response_xml = $this->xml_request($xml, "CrmService", "RetrieveMultiple");
    $response = array();
    foreach($attributes as $attribute) {
      $matches = array();
      if(preg_match('/<q1:' . $attribute . '[^>]*>([^<]*)<\/q1:' . $attribute . '>/', $response_xml, $matches)) {
        $response[$attribute] = $matches[1];
      }
    }
    return $response;
  }

  // Create an entity.  $attributes should have attributes as keys and their values as values.
  public function create_entity($entity, $attributes) {
    $xml = "<entity xsi:type=\"$entity\">\n";
    foreach($attributes as $k => $v) {
      $xml .= "<$k>" . htmlspecialchars($v) . "</$k>\n";
    }
    $xml .= "</entity>\n";
    return $this->xml_request($xml, "CrmService", "Create");
  }

  // Usage: $options = retrieve_attribute("lead", "leadsourcecode")
  public function retrieve_attribute($entity, $attribute) {
    $xml = '
      <Request xsi:type="RetrieveAttributeRequest">
        <EntityLogicalName>' . $entity . '</EntityLogicalName>
        <LogicalName>' . $attribute . '</LogicalName>
      </Request>
    ';
    $xml = $this->xml_request($xml, "MetadataService", "Execute");
    // Get a list of <Option> tags.
    $matches = array();
    if(!preg_match_all('/<Option>(.*?)<\/Option>/', $xml, $matches)) {
      return null;
    }
    $options_xml = $matches[1];
    $options = array();
    foreach($options_xml as $option_xml) {
      $matches = array();
      if(!preg_match('/<Value[^>]*>([^<]*)<\/Value>/', $option_xml, $matches)) {
        continue;
      }
      $code = $matches[1];
      if(!preg_match('/<Label>([^<]*)<\/Label>/', $option_xml, $matches)) {
        continue;
      }
      $name = $matches[1];
      $options[] = array("code" => $code, "name" => $name);
    }
    return $options;
  }

  public function find_code_by_name($entity, $attribute, $name) {
    $codes = $this->retrieve_attribute($entity, $attribute);
    foreach($codes as $code) {
      if($code["name"] == $name) {
        return $code["code"];
      }
    }
    return null;
  }
}

class MSCRMLead {
  protected $service;

  function __construct($service) {
    $this->service = $service;
  }

  public function find_user($guid) {
    $retrieve_fields = array("firstname", "lastname", "title", "internalemailaddress", "address1_telephone1");
    return $this->service->retrieve_multiple("systemuser", $retrieve_fields, "systemuserid", $guid);
  }

  public function find_lead_by_email($email) {
    $retrieve_fields = array("leadsourcecode", "firstname", "lastname",
                             "jobtitle", "companyname", "industrycode", "websiteurl", "address1_telephone1", "emailaddress1",
                             "address1_city", "address1_stateorprovince", "address1_country", "ownerid");
    return $this->service->retrieve_multiple("lead", $retrieve_fields, "emailaddress1", $email);
  }

  public function find_owner_by_email($email) {
    $lead_info = $this->find_lead_by_email($email);
    if(!$lead_info["ownerid"]) {
      return null;
    }
    $owner = $this->find_user($lead_info["ownerid"]);
    return $owner;
  }

  public function find_leadsourcecode($leadsource) {
    return $this->service->find_code_by_name("lead", "leadsourcecode", $leadsource);
  }

  public function find_industrycode($industry) {
    return $this->service->find_code_by_name("lead", "industrycode", $industry);
  }

  public function create($lead_info) {
    return $this->service->create_entity("lead", $lead_info);
  }
}

?>

You can now retrieve a lead's owner information like this:

require_once("mscrm.php");

$mscrm_service = new MSCRMService($host, $organization, $domain, $user, $password);
$mscrm_lead = new MSCRMLead($mscrm_service);
$owner = $mscrm_lead->find_owner_by_email($lead_email);

Creating a new lead is a little more involved, since you need to look up codes for picklists, like leadsourcecode. You can do something like this:

require_once("mscrm.php");

$mscrm_service = new MSCRMService($host, $organization, $domain, $user, $password);
$mscrm_lead = new MSCRMLead($mscrm_service);

$lead_info = array();

$leadsourcecode = $mscrm_lead->find_leadsourcecode($lead_source);
if($leadsourcecode) {
  $lead_info["leadsourcecode"] = $leadsourcecode;
}

$lead_info["firstname"] = $firstname;
$lead_info["lastname"] = $lastname;
$lead_info["jobtitle"] = $jobtitle;
$lead_info["companyname"] = $companyname;

$industrycode = $mscrm_lead->find_industrycode($industry);
if($industrycode) {
  $lead_info["industrycode"] = $industrycode;
}

$lead_info["websiteurl"] = $websiteurl;
$lead_info["address1_telephone1"] = $phone;
$lead_info["emailaddress1"] = $email;
$lead_info["address1_city"] = $city;
$lead_info["address1_stateorprovince"] = $state;
$lead_info["address1_country"] = $country;

$mscrm_lead->create($lead_info);

Hopefully this will save someone days of headaches.

Transparent TCP proxy in ruby (jruby)

2010-01-06T01:39:00.007-05:00

This is a transparent TCP proxy. I only tested it in jruby, but I see no reason why it wouldn't work in ruby as well. The proxy is multithreaded, it starts a new thread to handle every connection, up to a limit. The client and the server can talk at the same time, and IO.select is used to figure out who has data to send.

require "socket"

remote_host = "www.google.com"
remote_port = 80
listen_port = 5000
max_threads = 5
threads = []

puts "starting server"
server = TCPServer.new(nil, listen_port)
while true
  # Start a new thread for every client connection.
  puts "waiting for connections"
  threads << Thread.new(server.accept) do |client_socket|
    begin
      puts "#{Thread.current}: got a client connection"
      begin
        server_socket = TCPSocket.new(remote_host, remote_port)
      rescue Errno::ECONNREFUSED
        client_socket.close
        raise
      end
      puts "#{Thread.current}: connected to server at #{remote_host}:#{remote_port}"

      while true
        # Wait for data to be available on either socket.
        (ready_sockets, dummy, dummy) = IO.select([client_socket, server_socket])
        begin
          ready_sockets.each do |socket|
            data = socket.readpartial(4096)
            if socket == client_socket
              # Read from client, write to server.
              puts "#{Thread.current}: client->server #{data.inspect}"
              server_socket.write data
              server_socket.flush
            else
              # Read from server, write to client.
              puts "#{Thread.current}: server->client #{data.inspect}"
              client_socket.write data
              client_socket.flush
            end
          end
        rescue EOFError
          break
        end
      end
    rescue StandardError => e
      puts "Thread #{Thread.current} got exception #{e.inspect}"
    end
    puts "#{Thread.current}: closing the connections"
    client_socket.close rescue StandardError
    server_socket.close rescue StandardError
  end

  # Clean up the dead threads, and wait until we have available threads.
  puts "#{threads.size} threads running"
  threads = threads.select { |t| t.alive? ? true : (t.join; false) }
  while threads.size >= max_threads
    sleep 1
    threads = threads.select { |t| t.alive? ? true : (t.join; false) }
  end
end

Connecting to Multiple Databases in Ruby on Rails

2008-10-26T00:11:00.018-04:00

If you keep different tables in different databases, you've probably been looking for a way to allow Rails to connect to more than one database. You've probably found that you can add another configuration (besides development, test and production) to config/database.yml. There's a problem with that though. If you added a configuration called foo, you are now always connecting to foo, in production mode, test mode and development mode. This is especially bad in test mode, since the database would get wiped, but you probably want a different database for development and production too.

Here's how to do it. Suppose you have already set up your regular development/test/production databases in config/database.yml:

development:
  adapter: jdbc
  driver: com.mysql.jdbc.Driver
  url: jdbc:mysql://devdb.yourdomain.com:3306/mstore
  username: cart
  password: secret

# Warning: The database defined as 'test' will be erased and
# re-generated from your development database when you run 'rake'.
# Do not set this db to the same as development or production.
test:
  adapter: jdbc
  driver: com.mysql.jdbc.Driver
  url: jdbc:mysql://localhost:3306/mstore
  username: cart
  password: secret

production:
  adapter: jdbc
  driver: com.mysql.jdbc.Driver
  url: jdbc:mysql://proddb.yourdomain.com:3306/mstore
  username: cart
  password: secret

Now you decided to store all the user account information in a different database. You still want a separate development, test and production databases for your account information. You'll need to add three more entries to the database configuration then, one for each of the account databases. You'll need to name them with an appropriate suffix to specify development/test/production:

account_development:
  adapter: jdbc
  driver: com.mysql.jdbc.Driver
  url: jdbc:mysql://devdb.yourdomain.com:3306/account
  username: cart
  password: secret

# Warning: The database defined as 'test' will be erased and
# re-generated from your development database when you run 'rake'.
# Do not set this db to the same as development or production.
account_test:
  adapter: jdbc
  driver: com.mysql.jdbc.Driver
  url: jdbc:mysql://localhost:3306/account
  username: cart
  password: secret

account_production:
  adapter: jdbc
  driver: com.mysql.jdbc.Driver
  url: jdbc:mysql://accountdb.yourdomain.com:3306/account
  username: cart
  password: secret

Note that these account databases could be on the same hosts or on different hosts from the main databases. Now on to the models. You need to create a base class model for all your account tables, so that only one connection to the account database gets created. If you just call establish_connection in every model, it'll open that many database connections, one for each model. Here's a base model for the account tables:

class AccountBase < ActiveRecord::Base
  # No corresponding table in the DB.
  self.abstract_class = true

  # Open a connection to the appropriate database depending
  # on what RAILS_ENV is set to.
  establish_connection("account_#{RAILS_ENV}")
end

Now your account models need to inherit from AccountBase instead of ActiveRecord::Base, like this:

class Login < AccountBase
end

Because it inherits from AccountBase, it'll use the database connection already established by AccountBase.

/etc/hosts and Mac OS X 10.5 Leopard

2008-10-20T20:58:00.012-04:00

Instead of using /etc/hosts on Mac OS Leopard, the proper way to add hosts is using the dscl utility. First, let's list the hosts already there:

$ dscl localhost -list /Local/Default/Hosts
$

Nothing. Let's add one. I want to be able to access localhost as me.blogger.com. Right now that doesn't work:

$ ping -q -c 1 me.blogger.com
ping: cannot resolve me.blogger.com: Unknown host
$

Create an entry for it and try again:

$ sudo dscl localhost -create /Local/Default/Hosts/me.blogger.com \
  IPAddress 127.0.0.1
$ ping -q -c 1 me.blogger.com
PING me.blogger.com (127.0.0.1): 56 data bytes

--- me.blogger.com ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.077/0.077/0.077/0.000 ms

You can list the hosts you have set up, where they point, and you can delete them:

$ dscl localhost -list /Local/Default/Hosts
me.blogger.com
$
$ dscl localhost -read /Local/Default/Hosts/me.blogger.com
AppleMetaNodeLocation: /Local/Default
IPAddress: 127.0.0.1
RecordName: me.blogger.com
RecordType: dsRecTypeStandard:Hosts
$
$ sudo dscl localhost -delete /Local/Default/Hosts/me.blogger.com
$ dscl localhost -list /Local/Default/Hosts
$

CyberSource, SOAP and Ruby

2008-08-13T22:04:00.007-04:00

CyberSource has a SOAP API. They don't officially support Ruby, but I don't see why it shouldn't work. Ruby has built-in support for SOAP. I knew absolutely nothing about SOAP or WSDL an hour ago, so I'm still trying to figure out how this works. SOAP is a protocol for calling web services and getting the responses from them. WSDL files describe the available web services. It looks like CyberSource publishes a new WSDL file for every API version that it supports. The latest at the moment seems to be 1.38. You can see a list of their WSDL files here. Let's try something:

require "soap/wsdlDriver"

wsdl_file = "https://ics2ws.ic3.com/commerce/1.x/" +
  "transactionProcessor/CyberSourceTransaction_1.38.wsdl"

# Create a driver from the definition provided in wsdl_file.
driver = SOAP::WSDLDriverFactory.new(wsdl_file).create_rpc_driver

# See what's available in this driver.
puts driver.methods.sort.join("\n")

This prints out a long list of method names available on the driver that is created from the WSDL file. AFAICT, the only method you're supposed to call is runTransaction. I don't know how to call it yet.

MySQL On Mac OS X Using MacPorts

2008-08-10T10:45:00.011-04:00

If you only want to use the MySQL client, all you have to do is install it using the port command:

sudo port install mysql5

You can now connect to remote servers with something like this:

mysql5 -usomeuser -psomepassword -hsomehost.somedomain.com

However, if you want to run a MySQL server on your local machine, there are few more things that you have to do. First, you need to initialize MySQL system tables:

sudo mysql_install_db5 --user=mysql

This initializes MySQL databases under /opt/local/var/db/mysql5/ . We also need to create a directory under MacPorts that is identical to /var/run:

$ ls -ld /var/run
drwxrwxr-x  27 root  daemon  918 Aug 10 09:07 /var/run
$ sudo mkdir /opt/local/var/run
$ sudo chgrp daemon /opt/local/var/run
$ sudo chmod g+w /opt/local/var/run
$ ls -ld /opt/local/var/run
drwxrwxr-x  2 root  daemon  68 Aug 10 15:31 /opt/local/var/run

You can now start the MySQL server, but it is NOT SECURE yet. Make it only listen for local connections using the --bind-address option. This makes it impossible to connect to MySQL from the outside. If you intend this MySQL installation to be used for development only, you should always start it like this. If you want to use this in production and connect to it from other machines, you'll need to remove the --bind-address option, but only AFTER SECURING THE INSTALLATION.

sudo mysqld_safe5 --bind-address 127.0.0.1 > /dev/null 2>&1 &

MySQL comes with a nice script to secure it, called mysql_secure_installation. Unfortunately, if you installed MySQL using the sudo port install mysql5 command, the script will fail, because it's looking for the mysql command to be in your path, but the command is really called mysql5. You can make a symlink to fix it, though I'm not sure if this will clash with future MacPorts MySQL installations. You can remove the symlink after you're done if you wish.

sudo ln -s mysql5 /opt/local/bin/mysql

Now you can run the secure script. Be sure to set the root password, remove anonymous users, disable remote root login, remove the test database, and reload privilege tables. Basically answer Y to everything.

sudo mysql_secure_installation5

You should now be able to connect using your new root password (you'll be prompted for it):

mysql5 -uroot -p

To remove the symlink you made:

# Make sure it's really a symlink to mysql5.
$ ls -l /opt/local/bin/mysql
lrwxr-xr-x  1 root  admin  6 Aug 10 16:11 /opt/local/bin/mysql -> mysql5
$ sudo rm /opt/local/bin/mysql

To shut down MySQL, you can use the mysqladmin command. It will prompt for your MySQL root password:

mysqladmin5 shutdown -uroot -p

If you are planning to use this server in production, you should probably create a my.cnf with all the needed options, and make MySQL start up when the machine boots. You should also create some non-root users, even if it's only used for development. But this is a topic for another time.

Ruby On Rails on Mac OS X

2008-08-06T23:27:00.005-04:00

My laptop already had ruby installed (part of the standard OS install), but I decided to get the latest versions and recompile them. Thankfully this is trivial with MacPorts. To install MacPorts, you'll need to have the developer's toolkit installed (gcc, make, etc) - this on the second CD that comes with the laptop.

Once you have MacPorts installed, make sure it's in your path, put this in your .bash_profile :

# MacPorts
PATH=/opt/local/bin:/opt/local/sbin:$PATH; export PATH
MANPATH=/opt/local/share/man:$MANPATH; export MANPATH

Now install ruby and rubygems:

sudo port install ruby rb-rubygems

Use gem to install rails:

sudo gem install rails

Mongrel is much better than WEBrick, so install it:

sudo gem install mongrel

You'll need a database to start using Rails. I'll probably install a local MySQL for that.

Converting from Subversion to Mercurial

2008-07-29T11:45:00.001-04:00

Nice post about converting your Subversion (svn) code repository to Mercurial (hg):

SNH

Towing Features

2008-07-22T10:07:00.003-04:00

	Wheelbase	Payload	GVWR	Towing capacity
Jeep Wrangler Unlimited Rubicon	116	1150	5465	3500
Toyota FJ Cruiser	105.9	1277	5567	5000
Toyota Tacoma Access Cab	127.8	1395	5360	6500

MacBook Air Ordered

2008-07-14T00:46:00.004-04:00

Just put in an order for a refurbished Apple MacBook Air. It's $1500, which is $300 less than a new one with the same specifications (1.6GHz, 80GB disk). I think that's a smoking deal, for something that still has a 1 year warranty.

NoVA Traffic Capacity

2008-06-23T22:46:00.006-04:00

The Northern Virginia road system is severely under capacity. I don't know if it was always under capacity, or just became insufficient as the amount of traffic grew, but it definitely cannot handle the current traffic levels. Peak hours already see miles-long traffic jams, even without any accidents or road work. And accidents or any other capacity-reducing events jam the system even more.

I'm thinking about it from a network and computer systems design perspective. When designing a system, the aim is to create an N + 2 or greater capacity, where N is the peak traffic level. Let's say you need 5 machines to handle peak load. You need at least 7 machines in the pool. That way you can take out one machine for maintenance at any time, and still be able to handle a failure of another machine, without any effect on serving traffic.

The NoVA road system is more like N - 1. It can't even handle peak load, let alone be able to take a failure or scheduled maintenance and sustain the same traffic levels. There's only so much expansion that can be done, as there's simply not enough space to build more roads or bridges. So the solution must be to reduce traffic levels. Metro really needs to expand. Lane sharing should be allowed for motorcycles and scooters - that will encourage more people to ride, and reduce traffic levels, since motorcycles take up a lot less space on the road than cars do.

Chicken Kebab

2008-06-21T21:13:00.010-04:00

You'll need:

Marinade ingredients below
2 lbs of chicken breast
grape tomatoes
1 cup basmati rice

Marinade - mix ingredients well with a fork:

1 cup lemon juice
2 tablespoons olive oil
1/4 cup hot water
1 thinly sliced large onion
2 crushed garlic cloves
2 teaspoons salt
2 teaspoons ground black pepper
2 teaspoons paprika

Chicken - 2 lbs of chicken breast, cut into 1 inch pieces. Mix the chicken with the marinade, toss well. Cover and marinate in the refrigerator overnight (at least 8 hours), preferably 12 to 24 hours. Mix up the chicken and the marinade again halfway through this time.

Put 1 cup of basmati rice and 2 cups of water in a dutch oven. Let soak for 20 minutes. Bring water to a boil, then reduce heat to low and cover tightly, simmer for 15 minutes. Turn off heat (do not open the oven) and let stand for another 20 minutes.

Preheat the oven broiler. Skewer the chicken alternately with grape tomatoes. Put the skewers on an oven tray and put the tray under the broiler for 7 minutes on each side.

Serve chicken on skewers over a bed of rice.

Timezone conversion in Ruby

2008-06-05T10:17:00.010-04:00

The time in the feed is supplied with 2 fields - date and time. It is in UTC. The date looks like 20080605, and the time looks like 142115. I need to display the current local date and time. I could've used a regular expression to parse the date/time into components, but I wanted to use strptime. The Time module does not have strptime, but the DateTime module does. Once parsed, the DateTime object is in UTC. To get local time, use the new_offset method.

require "Date"

# time_str will look like "20080605 142115 UTC".
time_str = feed_date + " " + feed_time + " UTC"

# Parse time_str into a DateTime object.
dt = DateTime.strptime(time_str, "%Y%m%d %H%M%S %Z")

# local_dt will contain the date and time in the local timezone.
local_dt = dt.new_offset(DateTime.now.offset)

display_dt_str = local_dt.strftime("%Y-%m-%d %H:%M")

Camera system selection

2008-05-24T10:34:00.005-04:00

What I'm looking for when selecting an SLR camera system:

For travel, I need a lightweight body, a wide to telephoto zoom, and an even wider prime. Ideally, I would want a 15mm prime, and a 24-120 zoom. Unfortunately, nobody makes a system that has these.

For non-travel use, the body can be heavier, and I'd want the same lenses as above, plus a 50mm prime, plus a much longer zoom: 70-200 or longer. I want image stabilization in everything.

In the Canon world, the closest I can come to this system with a lightweight body is:

Digital Rebel body (1.6x multiplier for lenses)
Canon EF-S 17-55 f/2.8 IS USM (27-88 equivalent), excellent lens, but not as wide or as long as I would like
Canon EF-S 10-22 f/3.5-4.5 USM (16-35 equivalent), slow, no IS, but the only lens this wide available
Canon EF 14mm f/2.8L USM (22mm equivalent), an alternative to the 10-22 above, but not nearly as wide as I would like
Sigma 30/1.4 EX DC HSM (48mm equivalent), excellent standard lens
Canon EF 70-200mm f/2.8L IS USM (112-320 equivalent), excellent long zoom lens

If I drop the lightweight requirement, I can get a much heavier and more expensive Canon EOS-5D, which has a full-frame sensor. It's still the lightest full-frame camera available, but weighs almost twice as much as the Rebel.

Canon EOS-5D (full-frame sensor)
Canon EF 24-70mm f/2.8L USM, no IS
Canon EF 14mm f/2.8L USM, no IS
Canon EF 16-35mm f/2.8L USM, an alternative to the 24-70 and the 14mm above, would also require the 50mm prime below
Canon EF 50mm f/1.4 USM, standard prime
Canon EF 70-200mm f/2.8L IS USM

Dynamically adding methods to objects

2008-05-23T09:14:00.020-04:00

Let's say you have a class named Color, which defines instance methods red, green and blue, which return the corresponding color component (0 to 255). You have an instance of Color called santorin, among other instances. You just calculated a transparency component for this color and want to add that in. Problem is, the Color class doesn't have a transparency attribute. We don't want transparency in any other instance of this class anyway. The proper way to do this would probably be to subclass Color as TransparentColor and add a transparency attribute there. But that's more code than we want for this simple task. You could just define an extra method for the santorin instance:

def santorin.transparency
  5
end

This works, and will return 5 when you call it. But our calculated transparency is in a variable called trans, so we need to do this:

def santorin.transparency
  trans
end

Now that doesn't work, because when you try to call this method, you'll get this:

NameError: undefined local variable or method `trans' for ...
  ...

The problem is that trans is evaluated when you call the method, not when you define it. The quickest solution here is to use eval to force evaluation of the variable at definition time:

eval "def santorin.transparency() #{trans}; end"

Mid-size Travel Camera

2008-05-21T21:42:00.006-04:00

For traveling, it is useful to have a mid-size camera. Something with a good zoom lens, but smaller than a big DSLR. This is in addition to a small pocket camera (or cameraphone) that you just keep in your pocket. I have a Sony DSC-H5, which I like, except that it takes AA batteries - I want a lithium ion battery instead, so I can recharge whenever I want. I'm also considering the smallest DSLR for this purpose - the Olympus E-420. Camera specifications are from dpreview.com.

	Sony DSC-H5	Nikon Coolpix P80	Olympus E-420 w/Zuiko 12-60 lens
Megapixels	7.1	10.1	10
Sensor Size	1/2.5"	1/2.33"	4/3"
Zoom (35mm eq)	36 - 432 (11.4x)	28 - 486 (12.8x)	24 - 120 (5x)
Image Stabilization	Lens	Sensor	Digital?
Max Aperture	f/2.8 - f/3.7	f/2.8 - f/4.0	f/2.8 - f/4.0
Shutter	30 sec - 1/1000 sec	Unknown	60 sec (+ Bulb) - 1/4000 sec
Built-in Flash	pop-up	Yes	pop-up
Aperture Priority	Yes	Yes	Yes
Shutter Priority	Yes	Yes	Yes
Movie Clips	VGA 30 fps	640x480 30 fps	No
Self-timer	10 sec	3 sec or 10 sec	2 sec or 12 sec
Memory Card	MS Duo / Pro Duo	SD/MMC/SDHC	CF
RAW	No	No	Yes
LCD	3.0", 230,000 pixels	2.7", 230,000 pixels	2.7", 230,000 pixels
Battery	2xAA	Lithium Ion	Lithium Ion
Weight	490g	405g	440g + 575g = 1015g
Dimensions	108 x 81 x 92	110 x 79 x 78	130 x 91 x ~145