Kids are sick? Take coffee and code

I had some time since several nights (thanks kids to be sick…) to start watching how to implement a REST API using node.js (my new friend).

I plan to create an OW2 API this year; but before I started to wrap some existing services as proof of concept. The first feature focuses on how to get projects data from the gitorious instance we run at OW2. There is no clear API for gitorious but after some googling and source code search, it looks that there is a read-only XML API. Since I am targeting node.js, let’s translate this API into a JSON one: xml2js will do the job.

This results in a first library (gitoriou.js) which just call the gitorious instance and translates the XML data into JSON without any other data mapping. The following gist is showing how to get information from a project:

	/**
	* Use the OW2.org gitorious instance for samples.
	*
	* Copyright(c) 2013 Christophe Hamerling <christophe.hamerling@gmail.com>
	* MIT Licensed
	*/
	var Gitorious = require('../index').Gitorious
	, _ = require('underscore');
	var client = new Gitorious({url:'http://gitorious.ow2.org&#39;});
	console.log('# Getting OW2 Shelbie information : ')
	client.getProject('ow2-shelbie', function(err, result) {
	if (err) {
	console.log('Error:', err);
	return;
	}
	// console.log(JSON.stringify(result, null, 4));
	console.log('## Title:', result.project.title);
	console.log('### Repositories');
	_.each(result.project.repositories.mainlines.repository, function(item) {
	console.log(' – ', item.clone_url);
	})
	});

view raw

gitorious.js

hosted with ❤ by GitHub

I talked about having a REST API, let’s use express.js for that and let’s run it on Heroku (this platform rules, just took 2 minutes to create, deploy and run the service and it is up at http://ow2apisample.herokuapp.com/).

	/**
	* OW2 API Sample
	*
	* Copyright(c) 2013 Christophe Hamerling <christophe.hamerling@gmail.com>
	* MIT Licensed
	*/
	var express = require('express')
	, http = require('http')
	, path = require('path')
	, app = express();
	var Gitorious = require('gitoriou.js').Gitorious;
	var config = {
	url : 'http://gitorious.ow2.org&#39;,
	port : 3000
	}
	var client = new Gitorious(config);
	app.configure(function() {
	app.set('port', process.env.PORT || config.port);
	app.use(express.logger('dev'));
	app.use(express.bodyParser());
	app.use(express.static(path.join(__dirname, '../public')));
	});
	app.configure('development', function() {
	app.use(express.errorHandler({ dumpExceptions: true, showStack: true }));
	});
	app.configure('production', function() {
	app.use(express.errorHandler());
	});
	app.get('/projects', function(req, res) {
	client.getProjects(function(err, result) {
	if (err) {
	res.json(500, err);
	} else {
	res.json(200, result);
	}
	});
	});
	app.get('/project/:name', function(req, res) {
	client.getProject(req.param('name'), function(err, result) {
	if (err) {
	res.json(500, err);
	} else {
	res.json(200, result);
	}
	});
	});
	var server = http.createServer(app);
	server.listen(app.get('port'), function () {
	console.log('OW2 Server is started and listening on', app.get('port'));
	});

view raw

server.js

hosted with ❤ by GitHub

To get information (as JSON) about the Jasmine project, just HTTP GET http://ow2apisample.herokuapp.com/project/ow2-jasmine

If you master any scripting language, you will be able to get all the repositories URLs from the JSON response, if not you can use the ow2git project to clone them all. Assuming node.js and npm are available on your system (you already have java, ruby and every other stuff installed, why not adding node?), you can install the binary:

npm install -g ow2git

and then clone all the repositories from any gitorious project

ow2git –clone ow2-jasmine

Will clone all the ow2-jasmine repositories into your current folder.

I am really impressed by node.js runtime, tools and community. Even if all of this is just a proof of concept, not really well designed, you can have something running quickly without many effort. Time to get something real. Soon…

OAuth and Desktop Apps

I started to develop QuickHub some weeks ago by focusing on features and without taking into account security issues such as this critical information which are login and password credentials… As mentioned in the GitHub developer pages, I started to use Basic Auth for all the requests QuickHub does to get retrieve data from GitHub. I started to think about moving to OAuth when a user said me that he bought QuickHub but that he did not use it because of Basic Auth. He was afraid that I can rob its credentials and so have a look to its repositories and more. I totally understand this argument and so I started to think that it limits QuickHub adoption by developers and that I should do something.

So, I discussed with GitHub guys through their support channel (Here I want to say that I am really impressed on how fast and professional is the GitHub support team!). Of course, they also told me that they will never use QuickHub if OAuth is not provided. After discussions with the guys, I started to implement a workaround to provide OAuth support in QuickHub by using the OAuth Web flow and adding some stuff to QuickHub.

Let’s see how we can do that in a generic way so that you can use it in your app if needed since every serious Internet service also provide OAuth support… Note that I will not give many details about OAuth itself but I will use some terms, so have a look to OAuth documentation for more details.

Register a Web application

The first step is to register your application to the developer portal. Here you need to provide some information such as the app name and its callback URL. Even if we are developing a desktop app and not a Web one, we need to be able to provide this HTTP based callback URL. We will see in the next section what is inside this application. By registering our application, the service will generate and give us some keys to be used in the next steps, mostly to recognize us when calling the service.

Create a Web application

We need a Web application to handle callback calls from the service we want to use OAuth. This application will only be used at authorization time and just have to be able to receive the service call, get the code sent by the service, then call the service again with the code and our application keys (there is a secret one to be used here). As a result, the service will send you back your OAuth token to be used in all the future service calls. This token is used to authenticate your service calls, no more password stuff inside! Here is a quick sequence diagram showing all the exchanges between actors…

On the QuickHub side, I chose to create this Web application by using the Play Framework I already mentioned several times in this blog. I used the Heroku paas to provide the Web application publicly.

As showed in this Gist (https://gist.github.com/1466592), the callback method is really simple (as usual with Play!): just get the code and call GitHub with it and your application keys. When all lis done, just display a result page with a specific URL. Here it starts with ‘quickhubapp://oauth?’. Let’s see what it means in the next section…

Add custom URL handlers to your desktop app

Once the desktop application is authorized, our Web application redirects the user to a Web page which embeds a button targeting an URL starting by ‘quickhubapp://oauth?’. Here you already understand that by clicking on such a link must drive you to something special. The cocoa framework allows developers to register specific URL handlers for their applications. So we need to register QuickHub handlers so that OS X knows that every link  with the quickhubapp scheme must me routed to QuickHub. This is as easy as showed in this Gist https://gist.github.com/1466628.

This code snippet just tells QuickHub to invoke the getUrl method when it receives a quickhubapp event. Up to the getUrl method to handle things. In QuickHub, I just extract the OAuth token in order to persist it locally for future use.

Done!

So now we are able to use OAuth Web flows for desktop applications. In the best of the worlds, every service provider may provide a real desktop flow to be used without the need to create this additional web application. In the real world, it is not the case but as you see it can be bypassed quite easily.

I will provide more technical details on the second section ‘Create a Web application’ with real sample and code in the next weeks.