What’s New In BizTalk Server 2013 R2

This is the first in a series of posts exploring What’s New in BizTalk Server 2013 R2. It will also serve as the index of the series, and contain links to all of the posts to come.


This is a listing of all of the posts within the series:

  1. What’s New In BizTalk Server 2013 R2
    Shows Shared Access Signature (SAS) Authentication for Service Bus
  2. Getting Started with BizTalk Server 2013 R2’s Built-in Health Monitoring
    Demonstrates the installation and use of the BizTalk Health Monitor
  3. JSON Schemas in BizTalk Server 2013 R2 [Code Sample]
    Shows how to generate a JSON schema and write unit tests to validate instances
  4. Decoding JSON Data Using the BizTalk Server 2013 R2 JsonDecode Pipeline Component [Code Sample]
    Shows how to receive JSON messages and write integration tests to validate a configurable pipeline

We’ve been pretty busy over here at QuickLearn over the past few months, as many of you may have noticed. We’ve released our BizTalk Server 2013 Administrator Deep Dive class, and have been hard at work on our Azure BizTalk Services Fundamentals class (coming as soon as September 2014). Meanwhile, Microsoft has released BizTalk Server 2013 R2.

As a result, I am starting a series in a similar vein as my What’s New in BizTalk Server 2013 series, to uncover those new features in 2013 R2 that will make your life just a little bit easier. However, this time around it will be a weekly series that will occasionally take breaks to share time with posts about Azure BizTalk Services.

All of that having been said, I’m going to get upgraded, and then jump right in to check out one of the things I’m most excited about.

image

I Love Microsoft Azure Service Bus

I’ve got to admit that I’m a huge fan of Microsoft Azure Service Bus. Not only that, but I’m a big fan of the .NET API which really feels oh-so-right and makes allowances for multiple patterns for synchronous vs. asynchronous code.

That being said, a big pain point with Service Bus has been using the Access Control Service for fine-grained security – which really can be the opposite of intuitive – especially when the concept of an identified user isn’t really needed or important to your integration scenario.

Thankfully, ACS isn’t the only security model that Service Bus supports. We actually can also use Shared Access Signatures for authentication. SAS authentication allows you to generate a key for a specific fine-grained entity within Service Bus for which you want to authorize access (e.g., Listen access for a given Queue), and then from that key you can generate a signed token that expires after a period of time. This token need not be borne by a specific user, it need only be presented to be accepted.

While all of that could be filed under mildly interesting, the important thing to note is that unless you have BizTalk Server 2013 R2 installed, you will be limited to using the ACS model for Service Bus authentication and authorization.

SAS-SY SB-Messaging Adapter

Thankfully, after upgrading to BizTalk Server 2013 R2, if you visit the Authentication tab of the SB-Messaging Transport Properties dialog, you will find the following option available:

image

Knowing that you can use Shared Access Signatures is one thing, being able to put that into practice is another. If you haven’t used SAS authentication/authorization before, you’re in for a treat.

Configuring SAS Authentication for Service Bus Entities

If you head over to the Microsoft Azure Management Portal, create some Service Bus entities, and then head to the Configure tab for the same, you will find a section titled Shared Access Policies

image

This section allows you to define access policies (e.g., read-only access, or Listen permission for the queue shown in the screenshot), and then generate signing keys that can be used to generate access tokens granting that level of permission.

image

It’s nice to know that this can all be done through the web UI if needed, but nothing here seems to relate back to the first property that you may have noticed when examining the settings for the SB-Messaging adapter (i.e., the Shared Access Key Name property). In reality, it’s asking for what the UI calls the Policy Name.

So what would the adapter configuration look like for this Shared Access Policy?

image

Putting it to the Test

So let’s put the updated adapter to the test and see what we get out the other end. First, let’s whip up a quick console app that will give us a message that is simply the string “Hello World”

image

Yes, the purists will note that I did not use a MesagingFactory. Why? Because this is not going to be a long-lived client, and it felt nicer to type. However, given a real world example, MessagingFactory will usually be the correct choice.

So let’s run down what I have while that message sits in the queue. I have a one-way Receive Port with a single Receive Location. This Receive Location uses the SB-Messaging adapter pointed at myqueue and using SAS for authentication (per the settings in the screenshot above). I have a Send Port subscribing to all messages from this Receive Port. This Send Port is using the FILE adapter, because I’m writing this late at night and am running out of creativity.

With everything in place, you will see this glorious sight…

image

And opening the file reveals…

image

Am I impressed that this traveled through a Service Bus Queue to arrive on my file system? No. I’m just happy that it authenticated using SAS token along the way, and I didn’t have to touch ACS at all during the process.

One hope that I have for this new functionality, is that it will see people moving beyond using owner for everything. Even though it’s something that I would find myself doing for demonstration purposes, it is also something that made me cringe to see in real life. It’s a lazy and even dangerous thing to do in some cases.

Just a Taste

This is really just a small flavor of what’s to come. There are some pretty big changes that aren’t getting a lot of fanfare at the moment, but I hope that will change as more and more people upgrade and discover what is waiting for them under the covers.

Until next week, take care!

Windows Azure Service Bus Queues + BizTalk = Out of Body Experiences

This post is the third in a weekly series that will highlight things that you need to know about new features in BizTalk Server 2013. One of the major new features of BizTalk Server 2013, is the native ability to integrate with Windows Azure Service Bus through the new SB-Messaging adapter. This week we will be covering what can happen if you’re trying to integrate with applications that are loading up your Windows Azure Service Bus Queues with Brokered Messages made purely of properties.

Out of Context

Context properties are not a foreign concept to BizTalk developers. Indeed, BizTalk messages themselves are made up of both a message context and body parts. The same is true to some extent of Service Bus Brokered Messages – though the implementation differs greatly.

To deal with context properties of Service Bus Messages, the SB-Messaging adapter provides us the ability to promote these properties to the context of the BizTalk message generated upon receipt of a Brokered Message. The adapter configuration that controls this behavior looks like this:

sb-promote-props

One major difference between the properties of a Brokered Message and the context properties of a BizTalk message, is that BizTalk message context properties have a namespace to disambiguate properties that share the same name. Brokered messages have no such thing, and no concept of a property schema for that matter (which allows BizTalk to have an awareness of all possible properties that can exist in the context).

The adapter configuration setting pictured above is a bridge between those worlds that allows the Brokered Messages properties to be understood by BizTalk Server. Of course, you will have to define a property schema for those properties on the BizTalk side. Here’s an example of a really simple one that defines two custom properties:

propertyschema

All of this will function pretty perfectly – assuming that the message sent to the Queue has a body.

Out of Body

At last year’s TechEd NA conference in Orlando, one of the presenters (I believe it was Clemens Vasters) was discussing what Brokered Messages were, and how they were exchanged. In the talk, he mentioned that it was perfectly fine, and even preferable in some cases, to create a Brokered Message that was purely properties without any body content. Certainly this was a light-weight operation that relied only on HTTP headers, and required no overhead for serialization of body content. And at the time, I accepted this, and was happy.

As a result, a lot of my code interacting with the Service Bus involved snippets like this:

var client = Microsoft.ServiceBus.Messaging.QueueClient.CreateFromConnectionString(connString, queueName);
var message = new Microsoft.ServiceBus.Messaging.BrokeredMessage();
message.Properties.Add("CustomProperty", "Value");
message.Properties.Add("CustomProperty2", "Value2");
client.Send(message);

Notice here, we’re just passing around properties, and not dealing with a message body. That’s fine when I’m receiving the message in .NET code living somewhere else, but when moving into the BizTalk world, it became problematic. When attempting to receive such a message (even with the property schema deployed, and everything else otherwise happy), I would receive this (using the PassThruReceive pipeline):

errormsg

For those trying to use Google-fu to solve the same problem, the error message reads:

“There was a failure executing the receive pipeline: ‘Microsoft.BizTalk.DefaultPipelines.PassThruReceive, Microsoft.BizTalk.DefaultPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35” Source: “Pipeline” Receive Port: “WontMatchWhatYouHaveAnyway” URI: “sb://namespacehere-ns.servicebus.windows.net/somequeuenamehere” Reason: The Messaging Engine encountered an error whlie reading the message stream.

If you came to this article because this error message appeared, and you’re not trying to receive an empty message from Azure Service Bus, just head right back to the Bing results page, and check the next result. Everyone else, stick with me.

Let’s take a step back for a second. BizTalk Server 2013 is designed for integration between systems. I typically should not be adjusting my behavior outside of BizTalk to work with BizTalk. So here I want to take the same approach.

What are our options here? Well clearly we have a failure in the pipeline while processing an entirely empty message body, which doesn’t allow us to get the point where we are processing a map. That removes our ability to use the excellent ContextAccessor functoid to pull the properties into the body of the message upon receipt.

Instead we are left with building a custom solution.

PropertyMessageDecoder Pipeline component

What is this custom solution I speak of? For now, I’m calling it the PropertyMessageDecoder pipeline component. It is a custom pipeline component that creates a message body based on the context properties of a BizTalk message. If you already have a message body, this pipeline component will not help you, since it blindly overwrites the body if one exists. Instead, this is a purpose-built pipeline component for the specific scenario addressed by this blog post.

I started development of this component by using Martijn Hoogendoorn’s BizTalk Server Pipeline Component Wizard. Unfortunately, it hasn’t yet been updated to work with BizTalk Server 2013, so I had to first create/upload a patch for BizTalk 2013.

Once I was able to create a custom pipeline component project, I created a simple pipeline component that took the input message, and dumped the context properties into the message bodies. I also included some configurable properties that allowed one to opt-out of properties from the BizTalk system properties namespace, or to filter out all properties except those from a specific namespace (especially helpful in this scenario).

Here’s the bulk of the Execute method for that component:

public Microsoft.BizTalk.Message.Interop.IBaseMessage Execute(Microsoft.BizTalk.Component.Interop.IPipelineContext pc, Microsoft.BizTalk.Message.Interop.IBaseMessage inmsg)
{
    Stream dataStream = new VirtualStream(VirtualStream.MemoryFlag.AutoOverFlowToDisk);
    pc.ResourceTracker.AddResource(dataStream);

    using (XmlWriter writer = XmlWriter.Create(dataStream))
    {
        // Start creating the message body
        writer.WriteStartDocument();
        writer.WriteStartElement("ns0", ROOT_NODE_NAME, TARGET_NAMESPACE);

        for (int i = 0; i < inmsg.Context.CountProperties; i++)
        {
            // Read in current property information
            string propName = null;
            string propNamespace = null;
            object propValue = inmsg.Context.ReadAt(i, out propName, out propNamespace);

            // Skip properties that we don't care about due to configuration (default is to allow all properties)
            if (ExcludeSystemProperties && propNamespace == SYSTEM_NAMESPACE) continue;
            if (!String.IsNullOrWhiteSpace(CustomPropertyNamespace)
                    && propNamespace != CustomPropertyNamespace) continue;

            // Create Property element
            writer.WriteStartElement(PROPERTY_NODE_NAME);

            // Create attributes on Property element
            writer.WriteStartAttribute(NAMESPACE_ATTRIBUTE);
            writer.WriteString(propNamespace);
            writer.WriteEndAttribute();

            writer.WriteStartAttribute(NAME_ATTRIBUTE);
            writer.WriteString(propName);
            writer.WriteEndAttribute();

            // Write value inside property element
            writer.WriteString(Convert.ToString(propValue));
            writer.WriteEndElement();
        }

        // Finish out the message
        writer.WriteEndElement();
        writer.WriteEndDocument();
        writer.Flush();

    }

    dataStream.Seek(0, SeekOrigin.Begin);
    var outmsg = Utility.CloneMessage(inmsg, pc);
    outmsg.BodyPart.Data = dataStream;

    return outmsg;
} 

The output will end up conforming to this schema:

propertymessageschema

And will look a little something like this:

<?xml version="1.0" encoding="utf-8"?>
    <ns0:PropertyMessage xmlns:ns0="http://schemas.quicklearn.com/PropertyMessage/2013/06/">
        <Property Namespace="http://schemas.example.org/" Name="CustomProperty1">Value1</Property>
        <Property Namespace="http://schemas.example.org/" Name="CustomProperty2">Value2</Property>
    </ns0:PropertyMessage>

When included in a pipeline alongside the XML disassembler component, we are finally able to take those pesky “empty” messages from Azure Service Bus and process them as XML documents. The message context properties are still retained in the context for routing, but our custom properties are also made available for use directly in maps (through the liberal use of Value Mapping (Flattening) and Logical functoids), and further processing.

To follow this through to the end, I created a quick proof-of-concept map:

samplemap

Essentially we’re looping through the properties from the context (hence using the Value mapping flattening functoid) and only mapping the value stored inside the Property node whenever the namespace and name of the property match those that we care about. Since the namespace for all of our custom properties is the same, we have a single Equals functoid that we’re sharing across all (two) properties.

After processing a message all the way through the bus, custom pipeline, and map, this should result in this beauty as opposed to a nasty error message:

sampleoutput

un-Borked brokered messages

Remember, that none of this stuff is necessary if you’re working with Brokered Messages that have a message body. You only need to go this deep when you’re dealing with Brokered Messages made purely of properties (which is a recommended practice when possible).

By bringing out-of-the-box first class support for Windows Azure Service Bus Queues, BizTalk Server 2013 continues to prove that with a solid and extensible architecture in place, any type of integration can be made possible.

If you would like to learn more about extending BizTalk Server 2013 to meet your integration challenges, check out one of our upcoming BizTalk 2013 Developer Deep Dive classes.

If you would like to access sample code for this blog post, you can find it on github.