Synchronising Highrise, Basecamp and Google Shared Contacts–Part 2

Background: In Part 1 of this series I wrote about a recent project that I worked on to synchronise contact data between a number of web applications, and how I had to work around the (current) shortcomings of their APIs.

As Scott Hanselman says, “talk is cheap – show me the code”.  So here we go…

Basecamp & Highrise APIs

Implementing an interface to the Basecamp & Highrise APIs is relatively straightforward – I did it using PHP’s curl().  My implementation doesn’t support all of the API’s functions, but you should be able to easily add any additional calls that you need.

The functions centre around a single function which does the hard work on making the request:

private function _getData($uriExtension) {
$base_url = $this->url.$uriExtension;
$session = curl_init();
curl_setopt($session, CURLOPT_URL, $base_url);
curl_setopt($session, CURLOPT_USERPWD, $this->apiKey.':X');
curl_setopt($session, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
curl_setopt($session, CURLOPT_HEADER, false);
curl_setopt($session, CURLOPT_HTTPGET, 1);
curl_setopt($session, CURLOPT_HTTPHEADER, array('Accept: application/xml', 'Content-Type: application/xml'));
curl_setopt($session, CURLOPT_RETURNTRANSFER, true);
curl_setopt($session, CURLOPT_SSL_VERIFYPEER,false);
curl_setopt($session, CURLOPT_FRESH_CONNECT, true);
$response = curl_exec($session);
curl_close($session);
$xmlObj = new SimpleXMLElement($response);
return ($xmlObj);
}

Google Shared Contacts

This was simply done using the Zend GData classes which you can grab here.  Here’s some sample code for updating a shared contact in Google:

$client=$this->googleClientAuth('cp');
$client->setHeaders('If-Match: *');
$gdata=new Zend_Gdata($client);
$gdata->setMajorProtocolVersion(3);
// get the existing contact.
$query = new Zend_Gdata_Query($id);
$entry = $gdata->getEntry($query);
$xml = simplexml_load_string($entry->getXML());
// update title.
$xnl->title = $title;
// update the fullname.
$xml->name->fullName = $title;
// update entry.
$entryResult = $gdata->updateEntry($xml->saveXML(), $entry->getEditLink()->href);

Get more details over at the Google Apps Shared Contacts API page.

Basecamp & Highrise – Filling in the API gaps

So the big problem with the Basecamp & Highrise APIs is that they have bits missing – rather significant bits it turns out.  In particular the ability to create and update companies.

To get around this I wrote a couple of classes that use a combination of curl() and the awesome phpQuery to walk through the front-end pages and execute URLs just as a real user would.

Whilst this is not impervious to UI changes by 37Signals, it’s a fairly generic implementation and it’s a best endeavours approach given the current lack in the APIs.

Here’s some sample code:

public function createCompany($companyName) {

$success = true;

// go to the company page first, to get the token.
$resp = $this->_fetchUrl($this->_urls['base'].$this->_urls['companies'], 'GET', array(), false, $this->_ch);

phpQuery::newDocument($resp['data']);
$authToken = pq('input[name=authenticity_token]')->val();

$formFields = array(
'authenticity_token' => $authToken,
'company[name]' => $companyName,
'commit' => 'Create Company'
);

$resp = $this->_fetchUrl($this->_urls['base'].$this->_urls['companies'], 'POST', $formFields, false, $this->_ch);

// check - has this company already been created?
phpQuery::newDocument($resp['data']);
if(pq('div.flash_alert')->text() != '') {
$success = false;
$this->_errMsg = pq('div.flash_alert')->text();
}

return $success;
}

Show me the code!

So, in the hope of helping others to get synchronisation working (and work around the lack in the APIs) you can download the code at GitHub for your own use.

Enjoy!

Synchronising Highrise, Basecamp and Google Shared Contacts–Part 2

Background: In Part 1 of this series I wrote about a recent project that I worked on to synchronise contact data between a number of web applications, and how I had to work around the (current) shortcomings of their APIs.

As Scott Hanselman says, “talk is cheap – show me the code”.  So here we go…

Basecamp & Highrise APIs

Implementing an interface to the Basecamp & Highrise APIs is relatively straightforward – I did it using PHP’s curl().  My implementation doesn’t support all of the API’s functions, but you should be able to easily add any additional calls that you need.

The functions centre around a single function which does the hard work on making the request:

private function _getData($uriExtension) {
  $base_url = $this->url.$uriExtension;
  $session = curl_init();
  curl_setopt($session, CURLOPT_URL, $base_url);
  curl_setopt($session, CURLOPT_USERPWD, $this->apiKey.':X');
  curl_setopt($session, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
  curl_setopt($session, CURLOPT_HEADER, false);
  curl_setopt($session, CURLOPT_HTTPGET, 1);
  curl_setopt($session, CURLOPT_HTTPHEADER, array('Accept: application/xml', 'Content-Type: application/xml'));
  curl_setopt($session, CURLOPT_RETURNTRANSFER, true);
  curl_setopt($session, CURLOPT_SSL_VERIFYPEER,false);
  curl_setopt($session, CURLOPT_FRESH_CONNECT, true);
  $response = curl_exec($session);
  curl_close($session);
  $xmlObj = new SimpleXMLElement($response);
  return ($xmlObj);
}

Google Shared Contacts

This was simply done using the Zend GData classes which you can grab here.  Here’s some sample code for updating a shared contact in Google:

$client=$this->googleClientAuth('cp');
$client->setHeaders('If-Match: *');
$gdata=new Zend_Gdata($client);
$gdata->setMajorProtocolVersion(3);
// get the existing contact.
$query = new Zend_Gdata_Query($id);
$entry = $gdata->getEntry($query);
$xml = simplexml_load_string($entry->getXML());
// update title.
$xnl->title = $title;           
// update the fullname.
$xml->name->fullName = $title;
// update entry.
$entryResult = $gdata->updateEntry($xml->saveXML(), $entry->getEditLink()->href);

Get more details over at the Google Apps Shared Contacts API page.

Basecamp & Highrise – Filling in the API gaps

So the big problem with the Basecamp & Highrise APIs is that they have bits missing – rather significant bits it turns out.  In particular the ability to create and update companies.

To get around this I wrote a couple of classes that use a combination of curl() and the awesome phpQuery to walk through the front-end pages and execute URLs just as a real user would.

Whilst this is not impervious to UI changes by 37Signals, it’s a fairly generic implementation and it’s a best endeavours approach given the current lack in the APIs.

Here’s some sample code:

public function createCompany($companyName) {

    $success = true;

    // go to the company page first, to get the token.
    $resp = $this->_fetchUrl($this->_urls['base'].$this->_urls['companies'], 'GET', array(), false, $this->_ch);

    phpQuery::newDocument($resp['data']);
    $authToken = pq('input[name=authenticity_token]')->val();

    $formFields = array(
        'authenticity_token' => $authToken,
        'company[name]' => $companyName,
        'commit' => 'Create Company'
    );

    $resp = $this->_fetchUrl($this->_urls['base'].$this->_urls['companies'], 'POST', $formFields, false, $this->_ch);

    // check - has this company already been created?
    phpQuery::newDocument($resp['data']);
    if(pq('div.flash_alert')->text() != '') {
        $success = false;
        $this->_errMsg = pq('div.flash_alert')->text();
    }

    return $success;
}

Show me the code!

So, in the hope of helping others to get synchronisation working (and work around the lack in the APIs) you can download the code at GitHub for your own use.

Enjoy!

The Novice’s Guide to Replacing a Bathroom–Part 1

IMG_0573Yes, bank holiday weekend. Time for Easter reflections, visiting family, putting your feet up.

Also a great opportunity to get some DIY done. And with the wife and son away for the weekend, I decided to start one of several pending projects: replacing the main bathroom.

 

A bit of background

Our house was built in 1988, and both the main bathroom and the en-suite had not been touched since. From the beautiful stripy toothpaste style tiles to the faded pink sanitary ware, they were relics of a by-gone era of interior design, albeit functional relics.  When we moved in we had the en-suite re-fitted (I only did the flooring and painting), so the other bathroom has continued to age over the last 12 months.

My background in DIY is fairly limited – I can paint, lay laminate flooring, and generally solve most basic household problems. However I’ve never done any plumbing, tiling or bathroom work.

The (Rough) Plan

So the rough plan was to:

  • Replace the bath, hand basin and toilet
  • Replace the light fittings
  • Keep the existing electric shower
  • Re-paint and re-tile

We only decided to get on with this early in the week, so we got a move on with choosing colours, flooring and tiling.  We’ve had the toilet and hand basin in the garage since we moved in.

Sequence of Events

So I started off by removing the light fittings (after turning off the circuits of course) and the radiator.  Removing a radiator is easy – but don’t make my mistake which was answering the door, chatting for 10 minutes, and then hearing the pitter-patter of water dripping through the kitchen because I hadn’t fully tightened up the water pipe valves.

Next I removed the electric shower – again this was easy, but I had to take note of how it all went together.  And of course turn off the water before hand (and drain the system).

Removing my Shower

Then I attacked the tiles with a bolster chisel and club hammer – most of them came off fairly easily (although it’s still hard work).

So far so good.

Then came the plumbing.

I cut the pipes to the sink taps with a pipe slice, unscrewed the supporting bolts and disconnected the waste pipe. Out came the basin and pedestal. Easy.

I popped two pipe caps over the supply pipes and moved on.

Off came the side of the bath and more pipes to disconnect.  In the following two hours I used up my quota of expletives as I wrestled with getting a spanner (which had to be the largest one I had) into an impossibly small hole behind the bath with an overflow pipe in the way.

One blistered hand later, I got the bath out (gingerly moving it and hopping it over the waste pipe).

At 10pm last night I was feeling very pleased with myself – all pipes capped, two units removed, most of the tiles off. I had even turned the water back on to check the pipe work was OK and no leaks.

Then pop, and splash, and more dripping.

One of the pipe caps had come off.

A cycle that I’ve been through many times: switch off the water, drain, mop up, catch the drips in the kitchen and fixing isolation valves.  Switch the water back on, wait and check for leaks.

Today the toilet came out (surprisingly easily) leaving more pipes to sort out. 

New building regulations state that isolation valves should be fitted to all water pipes in new bathrooms.  These valves mean that you can switch off the water supply to an appliance without having to turn off the mains water supply.

I’ve spent hours today testing, tightening and re-testing these valves (and some deal of mopping up).

My conclusion: wires are nice as they bend and you can disconnect them. Pipes do not bend and cannot be turned off.  Electricity can only spark – water drips everywhere.

Electrics = good

Plumbing = bad

Top Tips

My top tips from two days’ of bathroom work:

  • Check your pipe sizes before buying anything – don’t rely on B&Q advice (like I did).  This probably means removing the side of your bath to have a look (my hot water was 22mm rather than 15mm).
  • Cut pipes as carefully as possibly to avoid deforming them and cutting them square
  • Remove the door – space becomes limited very quickly
  • Give yourself plenty of time!

References

A couple of sites that have been very useful:

How to find a bathroom suite – DIY Doctor

Remove and Replace a Toilet – DIY How To

More?

Now that I’ve removed the old bathroom, I’ve got to fit the new one, decorate and finish off…

The Novice’s Guide to Replacing a Bathroom–Part 1

IMG_0573Yes, bank holiday weekend. Time for Easter reflections, visiting family, putting your feet up.

Also a great opportunity to get some DIY done. And with the wife and son away for the weekend, I decided to start one of several pending projects: replacing the main bathroom.

 

A bit of background

Our house was built in 1988, and both the main bathroom and the en-suite had not been touched since. From the beautiful stripy toothpaste style tiles to the faded pink sanitary ware, they were relics of a by-gone era of interior design, albeit functional relics.  When we moved in we had the en-suite re-fitted (I only did the flooring and painting), so the other bathroom has continued to age over the last 12 months.

My background in DIY is fairly limited – I can paint, lay laminate flooring, and generally solve most basic household problems. However I’ve never done any plumbing, tiling or bathroom work.

The (Rough) Plan

So the rough plan was to:

  • Replace the bath, hand basin and toilet
  • Replace the light fittings
  • Keep the existing electric shower
  • Re-paint and re-tile

We only decided to get on with this early in the week, so we got a move on with choosing colours, flooring and tiling.  We’ve had the toilet and hand basin in the garage since we moved in.

Sequence of Events

So I started off by removing the light fittings (after turning off the circuits of course) and the radiator.  Removing a radiator is easy – but don’t make my mistake which was answering the door, chatting for 10 minutes, and then hearing the pitter-patter of water dripping through the kitchen because I hadn’t fully tightened up the water pipe valves.

Next I removed the electric shower – again this was easy, but I had to take note of how it all went together.  And of course turn off the water before hand (and drain the system).

Removing my Shower

Then I attacked the tiles with a bolster chisel and club hammer – most of them came off fairly easily (although it’s still hard work).

So far so good.

Then came the plumbing.

I cut the pipes to the sink taps with a pipe slice, unscrewed the supporting bolts and disconnected the waste pipe. Out came the basin and pedestal. Easy.

I popped two pipe caps over the supply pipes and moved on.

Off came the side of the bath and more pipes to disconnect.  In the following two hours I used up my quota of expletives as I wrestled with getting a spanner (which had to be the largest one I had) into an impossibly small hole behind the bath with an overflow pipe in the way.

One blistered hand later, I got the bath out (gingerly moving it and hopping it over the waste pipe).

At 10pm last night I was feeling very pleased with myself – all pipes capped, two units removed, most of the tiles off. I had even turned the water back on to check the pipe work was OK and no leaks.

Then pop, and splash, and more dripping.

One of the pipe caps had come off.

A cycle that I’ve been through many times: switch off the water, drain, mop up, catch the drips in the kitchen and fixing isolation valves.  Switch the water back on, wait and check for leaks.

Today the toilet came out (surprisingly easily) leaving more pipes to sort out. 

New building regulations state that isolation valves should be fitted to all water pipes in new bathrooms.  These valves mean that you can switch off the water supply to an appliance without having to turn off the mains water supply.

I’ve spent hours today testing, tightening and re-testing these valves (and some deal of mopping up).

My conclusion: wires are nice as they bend and you can disconnect them. Pipes do not bend and cannot be turned off.  Electricity can only spark – water drips everywhere.

Electrics = good

Plumbing = bad

Top Tips

My top tips from two days’ of bathroom work:

  • Check your pipe sizes before buying anything – don’t rely on B&Q advice (like I did).  This probably means removing the side of your bath to have a look (my hot water was 22mm rather than 15mm).
  • Cut pipes as carefully as possibly to avoid deforming them and cutting them square
  • Remove the door – space becomes limited very quickly
  • Give yourself plenty of time!

References

A couple of sites that have been very useful:

How to find a bathroom suite – DIY Doctor

Remove and Replace a Toilet – DIY How To

More?

Now that I’ve removed the old bathroom, I’ve got to fit the new one, decorate and finish off…

Synchronising Highrise, Basecamp and Google Shared Contacts–Part 1

NOTE: You can read part 2 here or just simply get the code.

I recently worked on a project where we needed to synchronise contact data between Highrise CRM (the source), Basecamp (for project management) and Google Shared Contacts (for email).  This is a common issue for small businesses – how to keep disparate systems in sync.  There are already some proprietary systems out there which help with this but they didn’t fulfil the needs of this client so we opted for a custom tool.

I’ll write up the steps I took to get all of this working in a series of posts, but the overall architecture is thus:

image

Feasibility

At the time of writing I determined the feasibility of synchronising all of these three systems by looking in detail at the APIs provided.  I’ve summarised my findings here:

Google Shared Contacts

It’s worth pointing out that we needed to load up NOT Google’s personal contacts, but the domain shared contacts.  This is a repository of contacts that Google provides on its Google Apps product, and works like a Global Address List, providing a single store of contact data to all domain users.  However, for some puzzling reason Google does not provide any user interface to manage this database.  Instead you must use the Google Shared Contacts API.

IMPORTANT POINT: I got very confused as to which versions of Google Apps supports the Google Shared Contacts API. Even though the documentation pages state that it is only supported by Apps for Business (Premium) and Education, the confusing point is that YOU CAN EXECUTE THE API CALLS SUCCESSFULLY and the data IS PERSISTED.

What I later discovered was that although all of the API calls work, the data is not exposed in Google Apps.  This is very frustrating and I wish Google would make this clearer in their documentation.

Also don’t get confused between the Google Contacts Data API and the Google Shared Contacts API.

Highrise & Basecamp

Both of these products are made by 37Signals, and if you’ve ever read their books, you’ll know that they are keen proponents of open systems.  They have well documented APIs which are very easy to get up and working with.

However, when you start to use them, you discover that some key elements are missing. The APIs they have provided allow you to get all information out, but when it comes to creating data, the calls just aren’t there. Despite numerous forum posts, 37Signals have not yet given good reasons for the lack of these API calls.

To get around this, I wrote a cheeky HTTP GET, Parse and Re-POST class which effectively walks through the front-end of the site programmatically, as if it were a standard user. Whilst this has been very stable so far, if either of the products publish changes to their UI then this code may need modification.

Coding Approach

I wrote all of the code in PHP 5, so all the samples here will be in PHP, but you could easily apply the concepts to different programming languages.

I built singelton classes for each of the products, and then a controlling class to tie them up all to achieve the synchronisation. I used PHP curl() to do the web requests, and for the Google API stuff I used Zend’s GData classes to speed up the development.

Synchronisation Approach

In order to synchronise these systems I took the following approach:

  • Get all Highrise Contacts
  • Add to Basecamp, storing the IDs of the two data objects in a synchronisation file
  • Add to Google Contacts, storing the ID of the Highrise Contact in the Notes field

Next Steps

In the next post I’ll start introducing the code I wrote to hook up to the Basecamp and Highrise APIs.

Synchronising Highrise, Basecamp and Google Shared Contacts–Part 1

NOTE: You can read part 2 here or just simply get the code.

I recently worked on a project where we needed to synchronise contact data between Highrise CRM (the source), Basecamp (for project management) and Google Shared Contacts (for email).  This is a common issue for small businesses – how to keep disparate systems in sync.  There are already some proprietary systems out there which help with this but they didn’t fulfil the needs of this client so we opted for a custom tool.

I’ll write up the steps I took to get all of this working in a series of posts, but the overall architecture is thus:

image

Feasibility

At the time of writing I determined the feasibility of synchronising all of these three systems by looking in detail at the APIs provided.  I’ve summarised my findings here:

Google Shared Contacts

It’s worth pointing out that we needed to load up NOT Google’s personal contacts, but the domain shared contacts.  This is a repository of contacts that Google provides on its Google Apps product, and works like a Global Address List, providing a single store of contact data to all domain users.  However, for some puzzling reason Google does not provide any user interface to manage this database.  Instead you must use the Google Shared Contacts API.

IMPORTANT POINT: I got very confused as to which versions of Google Apps supports the Google Shared Contacts API. Even though the documentation pages state that it is only supported by Apps for Business (Premium) and Education, the confusing point is that YOU CAN EXECUTE THE API CALLS SUCCESSFULLY and the data IS PERSISTED.

What I later discovered was that although all of the API calls work, the data is not exposed in Google Apps.  This is very frustrating and I wish Google would make this clearer in their documentation.

Also don’t get confused between the Google Contacts Data API and the Google Shared Contacts API.

Highrise & Basecamp

Both of these products are made by 37Signals, and if you’ve ever read their books, you’ll know that they are keen proponents of open systems.  They have well documented APIs which are very easy to get up and working with.

However, when you start to use them, you discover that some key elements are missing. The APIs they have provided allow you to get all information out, but when it comes to creating data, the calls just aren’t there. Despite numerous forum posts, 37Signals have not yet given good reasons for the lack of these API calls.

To get around this, I wrote a cheeky HTTP GET, Parse and Re-POST class which effectively walks through the front-end of the site programmatically, as if it were a standard user. Whilst this has been very stable so far, if either of the products publish changes to their UI then this code may need modification.

Coding Approach

I wrote all of the code in PHP 5, so all the samples here will be in PHP, but you could easily apply the concepts to different programming languages.

I built singelton classes for each of the products, and then a controlling class to tie them up all to achieve the synchronisation. I used PHP curl() to do the web requests, and for the Google API stuff I used Zend’s GData classes to speed up the development.

Synchronisation Approach

In order to synchronise these systems I took the following approach:

  • Get all Highrise Contacts
  • Add to Basecamp, storing the IDs of the two data objects in a synchronisation file
  • Add to Google Contacts, storing the ID of the Highrise Contact in the Notes field

Next Steps

In the next post I’ll start introducing the code I wrote to hook up to the Basecamp and Highrise APIs.