Category Archives: Web Services

FamilySearch API

I just got word this week that the LDS Church is opening up an API for their new FamilySearch. I believe this will open up a whole array of innovative family history/genealogy applications. Never before has there been an API which would allow developers to gain access to such powerful genealogical resources.

It’s amazing to see how the innovations and changes that happen in the world seem to come together to help the church in moving the work forward. Richard Miller and the rest of the team at the More Good Foundation have been keeping up with the latest web trends and have been using new technology to help move the work along. It’s cool to now see the technical department of the church grab hold of some of the web 2.0 principles and put them to work.

Update: I’ve been developing for a few months now with this new FamilySearch API, and I’m loving it. I’ve got an Open Source Ruby API wrapper project hosted at Google Code.

Benefits of Service Oriented Architecture

I’m a believer in Service Oriented Archicecture (SOA). Some of the main benefits that I see from SOA is that it defines clear ownership of data, provides reusability of enterprise resources, maximizes system interoperability.

Clear Ownership of Data

Service oriented architecture provides a clear ownership of data. A typical web service will store, retrieve and manipulate data in a database that is hidden from the view of the client that is making calls to that service. This is good because we know that the web service is the owner of the data in that database. 10 different applications might be requesting data from the web service or posting transactions to the web service, but they don’t touch the database directly.

Because only the web service is touching the database directly, business logic and data controls can be put in place in one spot. If a bug is introduced into the system and the data in the database gets corrupted, it’s the web service’s fault. It has clear ownership and responsibility for the data that it maintains.

Reusability of Enterprise Resources

Service oriented architecture also allows for reusability of enterprise resources. Divisions within an enterprise often need access to the information that another division mantains. Setting up services for a single division can open up that data and processing to other divisions that would otherwise be unavailable or not shared as easily. By allowing multiple business functions share the same resources, costs are brought down, and the organization can run more efficiently.

Of course, access controls will need to be in place so that only the right people are gaining access to or manipulating sensitive information.

System Interoperability

Web services are breaking down the walls of incompatibility. It doesn’t matter which language you are programming in, which platform you are running on, web services provide a way for systems to talk to each other. Platforms and programming languages all have their strengths and weaknesses. Web services allow developers to play to the different strengths of a particular programming language to solve a problem that is most appropriate for that language. This can lead to more efficient developer cycles.

This doesn’t mean that you particularly want to have a conglomerate of languages and operating systems that make up your information system. That could quickly become very difficult to maintain. However, it does provide flexibilty in the construction of an information system.

So much to learn

I recently bought Enterprise SOA by Dirk Krafzig, Karl Banke, and Dirk Slama, and am trying to make my way through the concepts in that book. SOA is more than just being able to slap a SOAP inteface on some software that you’ve written. It requires a new way of thinking when designing and implementing information systems. I think SOA will have a large impact on IT in the future. I’m excited to continue learning.

SOAP Server with PHP5 – part3: the glue code

Part 2 of this series of blog posts focused on creating your wsdl. Now we will move on to the “glue code” and putting it all together. In part 2, I created a wsdl that defined the application programming interface (API) for an inventory web service. We defined one simple function called getItemCount which takes a upc in the form of a string as a parameter and returns the count in the form of an integer.

Let’s go ahead and code up the function:

<?php //inventory_functions.php
function getItemCount($upc){
//in reality, this data would be coming from a database
$items = array('12345'=>5,'19283'=>100,'23489'=>234);
return $items[$upc];
}
?>

It is important to test all of your functions outside the scope of your web service to iron out all of the bugs, otherwise, you’ll be dealing with some nasty debugging as soon as you connect this with the SOAP glue code. I like to keep the functions or classes that are connected to the web service separated from the glue code, unlike the php soap extension tutorial does. That way I can test it more easily.

The glue code

The glue code is responsible for handling incoming soap requests and returning valid soap xml. It takes the wsdl service definition and it connects it with your function or class. Here’s what it looks like:

<?php //server.php
require 'inventory_functions.php';
ini_set("soap.wsdl_cache_enabled", "0"); // disabling WSDL cache
$server = new SoapServer("inventory.wsdl");
$server->addFunction("getItemCount");
$server->handle();
?>

The inventory_functions.php contains my getItemCount function. It is important that the function has been included (or required) before the $server->addFunction("functionName"); is called. Also, make sure that you have no white space before or after your opening and closing php tags.

Once you know that your wsdl is correct, and you won’t be changing it, remove the ini_set function from the glue code.

Important: Make sure your wsdl’s service block is pointing to the http accessible url of the glue code (see part 2 for details).

Testing the service

Now is the moment of truth, to see if your service works like you had anticipated. Simply create a script like this one:

<?php //client-test.php
ini_set("soap.wsdl_cache_enabled", "0"); // disabling WSDL cache
$client = new SoapClient("http://[path to the service]/inventory.wsdl");
$return = $client->getItemCount('12345');
print_r($return);
?>

Substitute the [path to the service] with your url path, and execute the script. Hopefully, you should be seing the results that you expected.

Tutorial Files

For all the files that have been discussed in this tutorial, download the tutorial.zip

Sorry, this post has received a barrage of spam. I had to shut down comments. Try one of the other posts in this series.

SOAP Server with PHP5 – part 2: fun with wsdl

Part 1 of this series of blog posts should help you get started with setting up a simple soap server with one function, a stockquote request. It points you to the soap extension tutorial on the Zend Developer Zone.

In this part, I’m going to explain some of the different parts of the wsdl so that the wsdl isn’t so intimidating. So let’s begin by examining the wsdl found on the tutorial here.

I’m going to start from the bottom of the wsdl and work my way to the top.

The ‘service’ block

<service name='StockQuoteService'>
<port name='StockQuotePort' binding='StockQuoteBinding'>
<soap:address location='http://[insert real path here]/server1.php'/>
</port>
</service>

This ‘service’ block of the wsdl gives some basic definitions for your soap server. You’ll want to pick a service name and stay consistent throughout the wsdl. This example uses ‘StockQuote’ for all its naming but you could use something like ‘Weather’ or ‘Inventory.’ Just stay consistent. So, if you were to make an ‘Inventory’ service, your service block of the wsdl would look something like this:

<service name='InventoryService'>
<port name='InventoryPort' binding='InventoryBinding'>
<soap:address location='http://[insert real path here]/server1.php'/>
</port>
</service>

The port name and binding connect this block with the other parts of the wsdl, so just keep to your naming convention and you’ll be fine.

The address location points to the location of what I call the “glue code” or the php script that glues your function to the soap extensions server libraries. The glue code is what handles all of your xml messages and passes the right parameters to your functions.

The ‘binding’ block

<binding name='StockQuoteBinding' type='tns:StockQuotePortType'>
<soap:binding style='rpc'
transport='http://schemas.xmlsoap.org/soap/http'/>
<operation name='getQuote'>
<soap:operation soapAction='urn:xmethods-delayed-quotes#getQuote'/>
<input>
<soap:body use='encoded' namespace='urn:xmethods-delayed-quotes'
encodingStyle='http://schemas.xmlsoap.org/soap/encoding/'/>
</input>
<output>
<soap:body use='encoded' namespace='urn:xmethods-delayed-quotes'
encodingStyle='http://schemas.xmlsoap.org/soap/encoding/'/>
</output>
</operation>
</binding>

The ‘binding’ block has a lot of stuff in it, but most of it is just defining how the soap client and server will communicate. There is very little code here that you’ll need to change to fit your service. There are 4 parts that you’ll need to modify in order to map the service to the function that you wish to put in the service. Those parts have been bolded and italicized above. Change the binding name and type, the operation functionName and the #functionName at the end of the soap:operation.

So, if we were to modify this for our inventory example we might change the getQuote to getItemCount and our binding block would be as follows:

<binding name='InventoryBinding' type='tns:InventoryPortType'>
<soap:binding style='rpc'
transport='http://schemas.xmlsoap.org/soap/http'/>
<operation name='getItemCount'>
<soap:operation soapAction='urn:xmethods-delayed-quotes#getItemCount'/>
<input>
<soap:body use='encoded' namespace='urn:xmethods-delayed-quotes'
encodingStyle='http://schemas.xmlsoap.org/soap/encoding/'/>
</input>
<output>
<soap:body use='encoded' namespace='urn:xmethods-delayed-quotes'
encodingStyle='http://schemas.xmlsoap.org/soap/encoding/'/>
</output>
</operation>
</binding>

If you were to add another function to your service, you would just copy the ‘operation’ section of this block, and paste it right below, and change the function name. Pretty simple.

The ‘portType’ block

<portType name='StockQuotePortType'>
<operation name='getQuote'>
<input message='tns:getQuoteRequest'/>
<output message='tns:getQuoteResponse'/>
</operation>
</portType>

The ‘portType’ block connects your binding block to the message blocks which define your method parameter and return types. The name should match the type from the binding block. You’ll also need to change the operation name and the input and output message attributes. The message attribute can be named however you wish as long as it matches the name of its corresponding message block, which we’ll talk about next.

For the inventory example, we’d change our portType block to the following:

<portType name='InventoryPortType'>
<operation name='getItemCount'>
<input message='tns:getItemCountRequest'/>
<output message='tns:getItemCountResponse'/>
</operation>
</portType>

Again, if you wish to add another method to your service, you would just have two operation blocks within the portType block.

The ‘message’ block(s)

<message name='getQuoteRequest'>
<part name='symbol' type='xsd:string'/>
</message>
<message name='getQuoteResponse'>
<part name='Result' type='xsd:float'/>
</message>

Here we have 2 message blocks that correspond to the message attributes found in the portType block of the stockquote.wsdl. This is where the meat of your wsdl resides.

The part element in the document specifies parameters for the requests messages, and the return structure for the variable returned by your function. For parameters, make sure you mape the name with the parameter name, and specify the type. Here is a list of primitive data types accepted: http://www.w3.org/TR/xmlschema-2/#built-in-primitive-datatypes.

For our inventory example, we might modify this section like so:

<message name='getItemCountRequest'>
<part name='upc' type='xsd:string'/>
</message>
<message name='getItemCountResponse'>
<part name='Result' type='xsd:integer'/>
</message>

The ‘definitions’ header

The definitions header is what defines the namespaces of your wsdl document. You can mostly leave this as it is from the example, but you should change a few parts. Here is how I’ve modified my inventory service header:

<definitions name='Inventory'
targetNamespace='urn:JimmyzInventory'
xmlns:tns='urn:JimmyzInventory'
xmlns:soap='http://schemas.xmlsoap.org/wsdl/soap/'
xmlns:xsd='http://www.w3.org/2001/XMLSchema'
xmlns:soapenc='http://schemas.xmlsoap.org/soap/encoding/'
xmlns:wsdl='http://schemas.xmlsoap.org/wsdl/'
xmlns='http://schemas.xmlsoap.org/wsdl/'>

And that’s pretty much it. Creating your own wsdl isn’t really as bad as it appears.

My final inventory wsdl is located here.

See part 3 for implementing the web service.

SOAP Server with PHP5 – part 1

Setting up a SOAP web service with php used to seem pretty intimidating to me, until I finally decided that I would hunker down and build my own WSDL. Now that I’ve successfully gone through the process of building a SOAP web service with php5, I would say that it really isn’t all that bad. I was actually quite surprised at how easy it was in the end.

To get started, I recommend the PHP Soap Extension article found on the Zend Developer Zone website. Before going through the tutorial you’ll need to make sure that you are running on php 5 with the soap extension installed. If you’re not sure if you are on version 5 or if the SOAP extension is indeed installed, create a script with the following code:

<?php phpinfo(); ?>

That should give you all of your server configuration information that you need. Search that page for “SOAP” and if you find it there with some configuration variables, you’ll know that you’re good to go.

I recommend setting up the SOAP server that is given in the tutorial which requires 3 files: stockquote.wsdl, server1.php, and client3.php. Place those files in a directory that is accessible via http and modify the stockquote.wsdl file, replacing http://[insert real path here]/server1.php with the path to your server1.php file.

Test the service by running the client3.php script. As you modify the service to return more complex data types, you’ll want to change the print function with a print_r to get a dump of the whole data schema that has been returned.

Once you have that working, you can begin to modify the function or WSDL to do what you need it to do.

Important: Before you make changes to the WSDL, add the following line above the client code and at the top of the server1.php script:

ini_set("soap.wsdl_cache_enabled", "0"); // disabling WSDL cache

Also, in the wsdl there is a part that has urn:xmethods-delayed-quotes. This has nothing to do with stockquotes, and you should just leave it alone.

See part 2 for an explanation of the wsdl.

SOAP, XML-RPC, and REST

We’ve been talking for a couple of days now in our information architecture class about Service Oriented Architecture (SOA). Although SOA isn’t exclusively based around web services, web services play an important role. So far, we’ve only discussed SOAP web services in class, but I’d like to talk about some of the other popular web services protocols, xml-rpc, and REST. I’m going to start with the most simple web service and work my way to the more complicated.

REST – stands for Representational State Transfer (wikipedia reference) but that’s not really that important. Basically, REST web services consist of using plain old HTTP to make method calls and you get XML back. The true RESTful way is to use GET, POST, PUT, and DELETE calls to access CRUD (Create, Read, Update, Delete) operations on an object. In that light, a GET request is your “Read,” POST is your “Create,” PUT is your “Update,” and DELETE is your “Delete” operation. Any parameters are sent as normal http parameters, either in the query string of a GET request, or in the query string of a POST request.

While the use of the above HTTP methods are ideal for this kind of web services interactions, it’s not commonly practiced. Rails is the first framework that I’ve seen that is doing any serious development and seamless support for this type of web services integration with its Active Resources library. It doesn’t come standard in Ruby on Rails, but you can get it as a plug-in.

There are a ton of web services out there that offer a REST like web services interface. They all just don’t adhere to the GET, POST, PUT, standards. They are more like xml document requests via GET or POST. For example, Yahoo! has a handful of web services that you can access via REST, including its Answers, Local, and Flickr services.

REST style web services are what power AJAX applications. The browser makes a request to a REST web service via JavaScript’s XMLHttpRequest object. The browser then parses the xml and replaces elements in the page with the data.

REST is really quite simple. If you would like to connect to some REST web services, check out my previous post on resources to help you connect to web services.

XML-RPC – is a remote procedure call protocol which uses XML for marking up both requests and responses (see wikipedia’s reference). It was created by Dave Winer of UserLand Software and Microsoft. UserLand was one of the early providers of blogging software, and thus XML-RPC is commonly used for blog ping services such as Technorati and Ping-o-Matic.

XML-RPC protocol is really simple. You pass requests which contain the method name, and parameters wrapped in xml that defines their data types. The response comes back with similar data.

XML-RPC servers are easier to implement than SOAP when using scripting languages such as php which have very loose data typing. Php has a nice PEAR library that you can install and use to handle XML-RPC client and server scripting. Ruby on Rails has built-in support for creating and connecting to xml-rpc web service. I’m sure there are other nice tools for the other languages that help you seamlessly connect to and create XML-RPC web services. If anyone knows of any good libraries, please comment.

According to wikipedia, XML-RPC is the predecessor to SOAP, which took XML-RPC to a more complex level.

SOAP – stands for Simple Object Access Protocol (see wikipedia’s reference). Accessing a SOAP web service is super easy with any language that has SOAP libraries to help you. All you have to do is point your code to the url of the web service’s WSDL, and you’re off and running, making remote calls as you would to a local object.

SOAP begins to be not so simple when you pull back the curtains and look at the innards that make it all work. Reading or creating a WSDL document is not an easy task. However, I’ve written a tutorial on how to get a SOAP server with php5.

PHP5 has a SOAP library that will handle your client and server, but you have to create your own WSDL. I’ve seen a few SOAP servers implemented using NuSOAP, a php library, but there is rarely a wsdl provided, and it can only be connected to by other NuSOAP clients from what I can tell.

Ruby on Rails has SOAP client and server capabilities, and doesn’t make you write your wsdl by hand. BenRobb has written a post on how to set up your web service in Rails.

I don’t have any experience creating SOAP servers in .NET, but I hear it is super slick. Java has a plethora of tools for doing SOAP. While I am comfortable programming in Java and I understand basic J2EE principles, I don’t have much experience with Java web services either.

Side By Side – Flickr gives some good examples of what a request and response would look like for each of these web services.

Resources to help you connect to web services

The first time I was required to connect to a web service, I had no idea where to start, and ended up doing things the hard way. I needed to connect to a REST like web service in order to process a credit card transaction. I was coding in php at the time, and created all of my http headers from scratch and sent my request using fsockopen and very low-level function calls. Well, there’s a much easier way to do things. Here are some resources to help you connect to web services.

If you’re programming in Ruby, php, C#, VB.NET, ActionScript, JavaScript, or Python, then the Yahoo! Developer Network has some great resources for you. The gurus at Yahoo! have provided code samples and links to libraries that will help you connect to RESTful and SOAP web services.

If you’ve never connected to a web service before, it is really easy! Yahoo has some cool web services to try out, so have at it!

Web Services Help for: