Promoting Properties Inside an Orchestration

This is one of those BizTalk tricks that I don’t use very often so I have to look it up. This post will serve to document it for future reference.

Recently I was working on a project where we needed to set the promoted properties used by the ESB Toolkit from inside an orchestration. While it is easy to set the value of a context property inside an expression shape, the property won’t be promoted for routing once the message leaves the orchestration. The workaround is to use a correlation set to promote the newly set values to the message context. This can be accomplished in three easy steps.

First, create a correlation type with the context values you want promoted. (You may need to add a reference to an assembly if you don’t see the properties in the list) Next, create a correlation set using the previously created correlation type. Finally choose the correlation set as the initializing correlation set on the send port for your message. Now the message context values you set will be promoted and can be used for routing.

For a longer explanation of how this works, check out “Property Promotion inside Orchestration” by Saravana Kumar.

Posted in BizTalk, Technology | No Comments »

Using WCF MessageContract to Model A Composite Schema

I have been working on a project where I needed to take an XML Schema Definition created in BizTalk and create a WCF service that would receive messages using the same schema. In this particular case, the XSD was a composite schema and was too complex for svcutil to automatically generate a data contract. (I prefer to create data contracts by hand anyway. The code generated by svcutil is a mess.) Attempting to model this using just the DataContract and DataMember attributes proved to be impossible because you can’t specify an XML namespace on a DataMember.

For example, if you have an XSD that imports elements from another XSD with a different target namespace, when you try and model it as a data contract there is no way to specify the that the imported schema elements belong to a different namespace. So if you have an XML document like this:

Sample XML message

<?xml version="1.0" encoding="utf-8" ?>
<ns0:MyMessage xmlns:ns0="http://someservice.example.com/">
    <ns0:Message>
        <ns0:MyElement>Data</ns0:MyElement>
    </ns0:Message>
    <ns1:OtherMessage xmlns:ns1="http://someservice.example.com/other/">
        <ns1:MyOtherElement>OtherData</ns1:MyOtherElement>
    </ns1:OtherMessage>
</ns0:MyMessage>

You might try and model that schema with something that looks like this:

DataContract

[DataContract(Namespace = @"http://someservice.example.com/")]
public class MyMessage
{
    [DataMember(Order = 1)]
    public Message Message { get; set; }
 
    [DataMember(Order = 2)]
    public OtherMessage OtherMessage { get; set; }
}
 
[DataContract(Namespace = @"http://someservice.example.com/")]
public class Message
{
    [DataMember(Order = 1)]
    public string MyElement { get; set; }
}
 
[DataContract(Namespace = @"http://someservice.example.com/other/")]
public class OtherMessage
{
    [DataMember(Order = 1)]
    public string MyOtherElement { get; set; }
}

If you use DataContract and DataMember attributes exclusively WCF will produce XML that looks like this:

WCF DataContract XML

<?xml version="1.0" encoding="utf-8" ?>
<ns0:MyMessage xmlns:ns0="http://someservice.example.com/">
    <ns0:Message>
        <ns0:MyElement>Data</ns0:MyElement>
    </ns0:Message>
    <ns0:OtherMessage xmlns:ns1="http://someservice.example.com/other/">
        <ns1:MyOtherElement>OtherData</ns1:MyOtherElement>
    </ns0:OtherMessage>
</ns0:MyMessage>

Note the incorrect namespace prefix on the OtherMessage elements. To get WCF to model the XSD correctly we need to turn the MessageContract attribute to get the job done. The MessageContract attribute along with the MessageBodyMember attribute (among others) gives you full control over the structure off your XML. So now we take the data contract we defined earlier for the Message and OtherMessage elements and combine it with a message contract to get the final result we are looking for.

MessageContract

[MessageContract(WrapperNamespace = @"http://someservice.example.com/")]
public class MyMessage
{
    [MessageBodyMember(Namespace = @"http://someservice.example.com/")]
    public Message Message { get; set; }
 
    [MessageBodyMember(Namespace = @"http://someservice.example.com/other/")]
    public OtherMessage { get; set; }
}

Posted in .NET, Technology | 1 Comment »

WCF Service Namespaces

Since I seem to forget this on a regular basis, I wanted to leave a note to myself for future reference. I also want to post a link to the Tech Talk blog where I found most of this information.

By default, WCF assigns the namespace “http://tempuri.org/” to all web service operations. (For more information on tempuri.org, see the W3C Note on WSDL.) Typically you will want to assign your own namespace to uniquely identify any web services you have deployed. Since WCF has a (seemingly) infinite number of configuration options, there are no fewer than three places you need to set the namespace to keep WCF from using the default tempuri.org.

The first is on the ServiceContract attribute. Set the namespace in the attribute constructor.

[ServiceContract(Namespace = @"http://someservice.example.com/")]
public interface ISomeService

The second place it needs to be set is on the ServiceBehavior attribute. Again, set the namespace in the attribute constructor.

[ServiceBehavior(Namespace = @"http://someservice.example.com/")]
public class SomeService : ISomeService

The third and final place it needs to be set is the bindingNamespace attribute on the endpoint element in the App.config file.

<endpoint bindingNamespace="http://someservice.example.com/"...>
...
</endpoint>

Posted in .NET, Technology | Comments Off

RESTful Integration: Part Two

In our last episode, we added a custom media type to our service. This time around we want to add links to our schema to form a basic API. The links will be used to tell the client what the next steps in the workflow should be. Along with the links, we will also be adding basic support to Rails for the HTTP OPTIONS verb.

First thing we will do is add a link field to our model. There is not a good reason to keep links in the database, so we will add them directly to our model.

class Quotation < ActiveRecord::Base
  def link
    read_attribute("link")
  end
 
  def link=(uri)
    write_attribute("link", uri)
  end
 
  def after_initialize
    write_attribute("link", "http://localhost:3000/quotations/" + read_attribute("id").to_s)
  end
end

Next we will add the link field to the new method in our quotations controller.

format.rogue {
        quote = params[:quotation]
        @quotation.customer = quote[:customer]
        @quotation.quote_amount = quote[:quote_amount]
        @quotation.status = quote[:status]
        @quotation.save
        @quotation.link = "http://localhost:3000/quotations/" + @quotation.id.to_s
        render :xml => @quotation, :status => :created
      }

This is the same basic code as last time with two additions. The first is the line that sets the link value after the quotation object has been stored in the database. The second is the addition of the “:status => :created” parameter to the render method. This instructs Rails to return the HTTP code 201 (which means resource created) instead of the standard code 200. (Which means OK) Now when we post a quotation to our service, it will respond with the 201 code, which tells us the resource was successfully created, and a link to our newly created resource. If this were a real service, we would include a link to the next step in our workflow/process.

Now that we have a resource, it might be nice to have a way to find out what we can do with it. For that we need add support for the HTTP OPTIONS verb to our service. We will start by adding an “options” method to our controller. The basic idea is while the quotation has a status of “New”, we will permit the client to update and delete the resource. Once the status changes, we will only permit the client to view the current state of the resource.

def options
    @quotation = Quotation.find(params[:id])
 
    respond_to do |format|
      format.rogue {
        if @quotation.status == "New"
          return head(:allow => "GET, PUT, DELETE")
        else
          return head(:allow => "GET")
        end
      }
    end
  end

Now we also need to add a line to config/routes.rb to map the OPTIONS verb to our new method.

map.connect '/quotations/:id', :controller => "quotations", :action => :options, :conditions => { :method => :options }

Once you add the new route, be sure to bounce your Rails application for it to take effect. Now we can use cURL to query the options on our resource.

/usr/bin/curl -i -X OPTIONS -H "Accept: application/vnd.rogue+xml" -H "Content-Type: application/vnd.rogue+xml" http://localhost:3000/quotations/22

If your resource has a status of “New”, you should see an “Allow: GET, PUT, DELETE” header value. Otherwise you should only see the “Allow: GET” header.

That will wrap up this installment. We now have the building blocks for a basic service interface. The next logical step would be to use these building blocks to put together a complete process. It would be nice to take the quotation idea and put together a service that takes quotation requests, approves them and then turns them into sales orders. Sounds like something to keep me busy now that the weather is starting to get colder.

Posted in Integration, Rails, Ruby | Comments Off

Resizing a Windows qemu Image

This is for the crazy few who, like me, run Windows in KVM on a Linux host. If you ever need to resize one of your Windows disk images, these are the steps necessary to accomplish this feat. (Please note the steps for actually changing the size of the disk image were taken from this linuxwave blog post) This works for Windows 2008 Server, and most likely Windows 2008 Server R2. I do not know if it works with any other versions of Windows.

• Convert qemu image to raw
  • # qemu-img convert -f qcow2 imagename -O raw imagename.raw
• Extend the image size using dd (In this example, to the size of 10G)
  • # dd bs=1 if=/dev/zero of=/path/to/imagename.raw seek=10G count=1 conv=notrunc
• Convert back to qemu image
  • # qemu-img convert -f raw imagename.raw -O qcow2 imagename
• Boot into Windows and use Storage Manager to resize your partition

Posted in Linux | Comments Off

RESTful Integration: Part One

Last time, we setup a simple controller in Rails that accepted POX data. Now lets go the next step and add a hypermedia format to our service and message. Our original service used a media type of “application/xml” but we really should use something different. The media type “application/xml” has a well defined processing model and semantics. The deal breaker here is “application/xml” does not define a semantic meaning for links. (We will cover the importance of links in a future post.)  What we really want is a custom media type so we can assign our own semantics to the message.

Our media type for this example will be “application/vnd.rogue+xml”. (The “vnd” namespace is reserved for creating proprietary media types.) The first thing we need to do is configure our Rails application to recognize our new media type. We do that by adding the following two lines to the mime_types.rb file located in config/initializers.

Mime::Type.register "application/vnd.rogue+xml", :rogue
 
ActionController::Base.param_parsers[Mime::Type.lookup("application/vnd.rogue+xml")] = :xml_simple

The first line registers our new media type and assigns a symbol to it. The second line configures a parser for our new media type. Since we are still using XML to represent our data, I decided to use the xml_simple parser. Next, we need to add some new code to the respond_to block of our controller. This works the same way it did in our original example.

format.rogue {
        quote = params[:quotation]
        @quotation.customer = quote[:customer]
        @quotation.quote_amount = quote[:quote_amount]
        @quotation.status = quote[:status]
        @quotation.save
        render :xml => @quotation
      }

Finally, in order to test this we change our curl script to use our new media type.

curl -H "Accept: application/vnd.rogue+xml" -H "Content-Type: application/vnd.rogue+xml" -d @quote.xml http://localhost:3000/quotations/new

Next time we will take a look at adding additional semantic information to our data and look at how we can use that information to represent application state.

Posted in Integration, Rails, Ruby | 1 Comment »

RESTful Integration: Part Zero

To kick this series off, lets start with something simple. The least-common denominator for a RESTful service is one that sends and receives POX. Obviously this is not where we want to be when we finish this series, but it does give us a place to start. I will be using Ruby on Rails as the platform for the code examples since it has excellent support for REST concepts built into the framework. I want to be able to concentrate on the problem of integration without getting bogged down by technical minutiae in trying to build these services using .NET or Java. Once I get some of this working with Rails, maybe I will do a series on implementing this integration architecture using other platforms. I am also going to assume a working knowledge of Ruby and Rails. I will be focusing on integration and will not be providing step-by-step directions on using Rails. So without further ado, lets get started.

This first application will consist of a basic service that will accept job quotations. So using Rails’ scaffolding generator, I created a simple quotation service with three fields. (Customer, Status and Quote Amount) As we continue through this series we will add more features to this service, but this will be enough to get started. With our service generated, we can proceed to set it up to receive an XML message using an HTTP POST command. First we need to add a little bit of code to the new method in the quotations controller.

def new
    @quotation = Quotation.new
 
    respond_to do |format|
        format.html # new.html.erb
        format.xml {
            quote = params[:quotation]
            @quotation.customer = quote[:customer]
            @quotation.quote_amount = quote[:quote_amount]
            @quotation.status = quote[:status]
            @quotation.save
            render :xml => @quotation
        }
    end
end

This code snippet will parse the XML message and store using whatever storage mechanism you have configured Rails to use. (On my system, it is SQLite3) Our next step is to then test this. To do so, we will first create an XML message with some test data.

< ?xml version="1.0" encoding="UTF-8"?>
<quotation>
	<customer>Rogue Technology</customer>
	<quote -amount>50000.00</quote>
	<status>New</status>
</quotation>

Next we will use curl to send the message to the service.

curl -H "Accept: text/xml" -H "Content-Type: text/xml" -d @quote.xml http://localhost:3000/quotations/new

Now that we have this much working lets take a look at where we are. We have built a basic service that will accept an XML message and thanks to the Rails framework, will happily return XML messages in a RESTful manner. At the same time, we have some issues that will need to be addressed. Our service does not use HATEOAS to present any sort of context information to clients. Plus, using text/xml or application/xml does nothing to tell the client about the format of the XML message our service is expecting to receive. Next time we will look at setting up our service to use a custom media type that better identifies what our service will be sending and receiving.

Posted in Integration, Rails, Ruby | 1 Comment »

REST-*: Blessing or Bane?

Recently, JBoss has announced a new architecture initiative called REST-*. The purpose of this initiative is to create an open source project dedicated to the goal of applying web architecture to integration and middleware. They intend to take traditional integration concepts and find which ones fit into the REST model. The hope is to publish the specifications that will be created from this project as IETF RFCs.

From the website, the stated architecture goals are:

Low barrier to entry – Clients that use the specification should have a very low barrier to entry. They shouldn’t need to install a library or large stack of software to use a specification. An HTTP client or web server provided by the language or platform should be enough to implement or use implementations of the specification.

Edge Cases should be Extensions - Edge cases that complicate the main specification should be defined in a separate sub-specification. Extensions should strive to be layered on top of the main specification by using facilities like HATEOAS and HTTP conneg to provided their features.

Pragmatic REST – While a specification should strive to follow RESTful principles, simplicity should never take a back seat to being a pure RESTafarian. If you need to bend the rules of REST to create a simpler design, then that’s the path that should be taken.

80/20 Rule – Specifications should remain simple. Many times in specification efforts, edge cases cause a lot of bloat and complexity within the specificaiton making it difficult to use, understand, and implement. Specifications should cover 80% of the most common use cases. Edge cases should not be in the main specifications. REST should be able to provide the facilities (HATEOAS, HTTP conneg) to abstract away edge cases.

Avoid Envelope formats - Whenever possible, avoid envelope formats. Examples of envelope formats are SOAP and Atom. Envelope formats encourage tunneling over HTTP instead of leveraging HTTP. They also require additional complexities on both the client and the server. In most cases HTTP headers should be enough to transfer metadata about the request, response, or resource.

Isolate data formats to extensions - If possible, specifications should try not to define new data formats.

I will be following the developments of this project very closely since this is an area where I am dabbling a bit myself. I am very interested in seeing how they approach some of the issues I have blogged about recently. My only concern is that they will attempt to build specifications around some of the worst ideas in distributed computing. That is my primary complaint with the way the WS-* specifications have evolved. Back when the “S” in SOAP still stood for “Simple”, SOAP seemed like a promising way to build distributed systems based on open standards. Unfortunately, the specification committee promptly took the opportunity to add everything including the kitchen sink. In the process they made it nearly impossible to build anything with SOAP-based services and wrecked the promise of interoperability between different vendor implementations. I just hope the REST-* project does not repeat the same mistakes. (Distributed transactions and reliable messaging. I am looking at you.)

Posted in Technology, Web | Comments Off

WOA: The Missing Link

Since the last time I wrote something about WOA, I have spent some more time researching this topic. (Though I still have not had a chance to sit down and build something.) Two articles in particular, “How to GET a Cup of Coffee” and “HATEOAS – The Confusing Bit from REST” do an excellent job of explaining what HATEOAS is and how workflow is handled in a RESTful fashion as opposed to how things are typically done with SOAP or POX. I find the idea of using standard HTTP infrastructure and features to handle the sending and receiving of data, and placing the next steps of the workflow process in the data to be absolutely brilliant. This gives us routing and workflow (aka. orchestration) which are arguably the two most important features of any integration implementation.

With that said, there are some additional integration concepts that I need to translate into a WOA/REST world before I would be comfortable selling this idea to a client.

• Discovery – If I am the new kid on the development team, how do I find out what REST services are already available on the network? How do I find out what their data contracts are? This could be something as simple as a web or wiki page, but I would like to know if there are any automated ways of discovering services and their data contracts.
• Transformation – Each service is going to expect data in a certain format. If I am getting data from another source, say, a packaged application how to I convert the data from the packaged application’s format into the format expected by the service? Without an intermediary handling the data, transformation will need to occur on the client. I can envision this being a part of a REST framework so the client will not be forced into implementing custom transformation logic.
• BAM - Most traditional integration platforms have a method of gathering data as messages are being processed for reporting purposes. How do I monitor business activity using REST? Maybe this is where enterprise mashups come into play. Instead of collecting data with an intermediary, each service will collect it’s own data and provide a query interface for retrieval. These different query interfaces could then be used to extract and combine the data into meaningful reports.
• Legacy Systems – Many existing legacy systems do not have support for RESTful communication. How do you incorporate a legacy system or application into a RESTful workflow? I figure this is where the developer will need to create a service either by writing code or leveraging an EAI tool to create a REST interface to the legacy system.

These are a few of the issues that need to be resolved before I can consider using WOA/REST for a real integration project. In order to find answers to these questions, I will need to build something so I can see what works. As I continue to work through these issues I will try and record more of my thoughts here.

Posted in Technology | 1 Comment »

Ohio LinuxFest 2009

Another Ohio LinuxFest has come and gone and another wonderful time was had by all. This year I did something a little different. Instead of attending the more technical sessions, I spent my time in the sessions on business and legal issues as they relate to FOSS. Here are some of the highlights from the sessions I attended.

• Open Source Business  101 For Hackers – This session provided an overview of the different business models for open source companies and the pros and cons of each model. I found this very interesting as a business owner who would like to incorporate open source into our product and service offerings.
• Fedora, OLPC, lessons learned and where we go from here – This was an interesting review of the OLPC project and where they plan on going.
• Legalities of FOSS from a Hacker’s Perspective – This session discussed the legal issues around licensing your code in an open source project. Discussion included a look at some of the most popular licenses and how to choose one to fit your project, what it means to put something into the public domain and some of the special cases like dual-licensing. From a business standpoint, it is critical to pick a proper license the first time to avoid problems down the road.
• Building a Community Around Your Project – This was a look at what to do and what not to do in order to build a community around your project.
• Be A Wonk! Open Source, Government Policy, and You – This was a look at some of the technical issues like net neutrality that are being discussed and examined by our government. It also provided tips on how to get involved at the local, state and federal levels.

Once again the OLF committee put together another fabulous community event. I really look forward to seeing what they put together for next year.

Posted in Linux, Technology | Comments Off