Integrate 2014 Starting Up

Tomorrow morning, the Integrate 2014 conference revs up here in Redmond, WA with Microsoft presenting the vision for the next versions of BizTalk Server and Microsoft Azure BizTalk Services.

I’m getting pretty excited as I’m looking over the first day agenda to see what to expect from the talks and sessions like Business Process Management and Enterprise Application Integration placed in between talks about BizTalk Services and On-Premise BizTalk Server. I’m even more excited seeing the second day agenda and seeing talks like Business Rules Improvements, and especially Building Connectors and Activities.

What’s Keeping Me Busy

In anticipation of this event, I have been building out a hybrid cloud integration sample application that leverages some of the ideas laid out in my previous post regarding cloud integration patterns while also providing the content for the long overdue next piece in my new features series for BizTalk Server 2013 R2.

In the sample, I’m running requests through MABS for content-based routing based on an external SQL lookup, with the requests targeting either an on-premise SQL Server database or a BizTalk Server instance for further processing that can’t otherwise be done currently with BizTalk Services (namely Business Rules Evaluation and Process Orchestration).

I’m hoping that after the conference this week, I will be able to tear the sample apart and build out most (if not all) of the elements in the cloud.

Best Bridge Between Azure and On-Prem?

Along the way, I’ve been further playing with using Service Bus Topics/Subscriptions as the Message Box of the cloud. At the same time, it represents a nice bridge between BizTalk Services and On-Premise BizTalk Server.

Consider the itinerary below;

image

This was actually the first draft of prototyping out the application. What this represents is an application that is receiving requests from client applications that have been published to a service bus topic. As they are received, they are routed based on content and sent either to a database to be recorded (as an example of an on-premise storage target), or to BizTalk Server 2013 R2 for further processing through a Service Bus Relay.

Given the scenario, maybe a relay is appropriate – lower latency requirement, no requirement for durability (which we have already sacrificed by running it through an initial bridge).

However, maybe we want to take a more holistic approach, and assume that the cloud is giving us a free public endpoint and some quite powerful content-based routing, translation, and even publish-subscribe capability when we bring Azure Service Bus to the mix. Let’s further assume that we view these capabilities as simply items in our toolbox alongside everything that BizTalk Server 2013 R2 is already providing us.

Let’s make it more concrete. What if the on-premise processing ultimately ends up sending the message back to the TradesDb? Is there a potential waste in building out that portion of the process in both locations?

Service Bus is the New Hybrid Integration MessageBox

Let’s try this instead:

image

Here, instead of using a relay, we’re using a Service Bus Topic to represent the call out to BizTalk Server 2013 R2.

Why would we do  this? While it introduces slightly more latency (in theory – though I haven’t properly tested that theory), it becomes pure loosely coupled pub-sub. I’m going to go out on a limb here (and risk being called an architecture astronaut), and say that not only is that not a bad thing, but it might even be a good idea. By feeding a topic rather than directly submitting to BizTalk Server via Relay allows us to easily swap out the processing of the rules with any mechanism we want, at any time – even if it means that we will have to transform the message and further submit by a completely different transport. Maybe one day we will be able to replace this with a call to a Rules Engine capability within MABS (crossing my fingers here) if we see such a capability come.

Further, we have broken out the logging of trades to the database into it’s own separate miniature process alongside the rest. This one might be fed by messages generated by BizTalk Server 2013 R2 on-premise or the earlier processing in MABS – providing only a single implementation of the interface to manage and change.

Is It The Right Way™?

I don’t know. It feels kind of right. I can see the benefits there, but again, we are potentially making a sacrifice by introducing latency to pay for a loosely coupled architecture. Call me out on it in the comments if that makes you unhappy. Ultimately, this would have to simply become a consideration that is weighed in choosing or not choosing to do things this way.

What Could Make it Even Better?

Imagine a time when we could build-out a hybrid integration using a design surface that makes it seamless. One where I could quickly discover the rest of the story. Right now there’s quite a bit happening on-premise into which we have no visibility via the itinerary – and very limited visibility within the orchestration since most logic is in Business Rules and my maps happen just before/after port processing.

Tomorrow

Tomorrow I will be writing a follow-up blog with a re-cap of the first day of the Integrate 2014 conference. Additionally, I will be playing with this application in my mind and seeing where the things announced this week change the possibilities.

If you’re going to be there tomorrow, be sure to stop by the QuickLearn Training table to sign-up for a chance to win some fun prizes. You can also just stop by to talk about our classes, or about any of the ideas I’ve thrown out here – I welcome both the positive and negative/nitpicky feedback.

Also, make sure you’re following @QuickLearnTrain on Twitter. We’ll be announcing an event that you won’t want to miss sometime in the next few days.

See you at the conference!

– Nick Hauenstein

What’s New In BizTalk Server 2013 R2

This is the first in a series of posts exploring What’s New in BizTalk Server 2013 R2. It will also serve as the index of the series, and contain links to all of the posts to come.


This is a listing of all of the posts within the series:

  1. What’s New In BizTalk Server 2013 R2
    Shows Shared Access Signature (SAS) Authentication for Service Bus
  2. Getting Started with BizTalk Server 2013 R2’s Built-in Health Monitoring
    Demonstrates the installation and use of the BizTalk Health Monitor
  3. JSON Schemas in BizTalk Server 2013 R2 [Code Sample]
    Shows how to generate a JSON schema and write unit tests to validate instances
  4. Decoding JSON Data Using the BizTalk Server 2013 R2 JsonDecode Pipeline Component [Code Sample]
    Shows how to receive JSON messages and write integration tests to validate a configurable pipeline

We’ve been pretty busy over here at QuickLearn over the past few months, as many of you may have noticed. We’ve released our BizTalk Server 2013 Administrator Deep Dive class, and have been hard at work on our Azure BizTalk Services Fundamentals class (coming as soon as September 2014). Meanwhile, Microsoft has released BizTalk Server 2013 R2.

As a result, I am starting a series in a similar vein as my What’s New in BizTalk Server 2013 series, to uncover those new features in 2013 R2 that will make your life just a little bit easier. However, this time around it will be a weekly series that will occasionally take breaks to share time with posts about Azure BizTalk Services.

All of that having been said, I’m going to get upgraded, and then jump right in to check out one of the things I’m most excited about.

image

I Love Microsoft Azure Service Bus

I’ve got to admit that I’m a huge fan of Microsoft Azure Service Bus. Not only that, but I’m a big fan of the .NET API which really feels oh-so-right and makes allowances for multiple patterns for synchronous vs. asynchronous code.

That being said, a big pain point with Service Bus has been using the Access Control Service for fine-grained security – which really can be the opposite of intuitive – especially when the concept of an identified user isn’t really needed or important to your integration scenario.

Thankfully, ACS isn’t the only security model that Service Bus supports. We actually can also use Shared Access Signatures for authentication. SAS authentication allows you to generate a key for a specific fine-grained entity within Service Bus for which you want to authorize access (e.g., Listen access for a given Queue), and then from that key you can generate a signed token that expires after a period of time. This token need not be borne by a specific user, it need only be presented to be accepted.

While all of that could be filed under mildly interesting, the important thing to note is that unless you have BizTalk Server 2013 R2 installed, you will be limited to using the ACS model for Service Bus authentication and authorization.

SAS-SY SB-Messaging Adapter

Thankfully, after upgrading to BizTalk Server 2013 R2, if you visit the Authentication tab of the SB-Messaging Transport Properties dialog, you will find the following option available:

image

Knowing that you can use Shared Access Signatures is one thing, being able to put that into practice is another. If you haven’t used SAS authentication/authorization before, you’re in for a treat.

Configuring SAS Authentication for Service Bus Entities

If you head over to the Microsoft Azure Management Portal, create some Service Bus entities, and then head to the Configure tab for the same, you will find a section titled Shared Access Policies

image

This section allows you to define access policies (e.g., read-only access, or Listen permission for the queue shown in the screenshot), and then generate signing keys that can be used to generate access tokens granting that level of permission.

image

It’s nice to know that this can all be done through the web UI if needed, but nothing here seems to relate back to the first property that you may have noticed when examining the settings for the SB-Messaging adapter (i.e., the Shared Access Key Name property). In reality, it’s asking for what the UI calls the Policy Name.

So what would the adapter configuration look like for this Shared Access Policy?

image

Putting it to the Test

So let’s put the updated adapter to the test and see what we get out the other end. First, let’s whip up a quick console app that will give us a message that is simply the string “Hello World”

image

Yes, the purists will note that I did not use a MesagingFactory. Why? Because this is not going to be a long-lived client, and it felt nicer to type. However, given a real world example, MessagingFactory will usually be the correct choice.

So let’s run down what I have while that message sits in the queue. I have a one-way Receive Port with a single Receive Location. This Receive Location uses the SB-Messaging adapter pointed at myqueue and using SAS for authentication (per the settings in the screenshot above). I have a Send Port subscribing to all messages from this Receive Port. This Send Port is using the FILE adapter, because I’m writing this late at night and am running out of creativity.

With everything in place, you will see this glorious sight…

image

And opening the file reveals…

image

Am I impressed that this traveled through a Service Bus Queue to arrive on my file system? No. I’m just happy that it authenticated using SAS token along the way, and I didn’t have to touch ACS at all during the process.

One hope that I have for this new functionality, is that it will see people moving beyond using owner for everything. Even though it’s something that I would find myself doing for demonstration purposes, it is also something that made me cringe to see in real life. It’s a lazy and even dangerous thing to do in some cases.

Just a Taste

This is really just a small flavor of what’s to come. There are some pretty big changes that aren’t getting a lot of fanfare at the moment, but I hope that will change as more and more people upgrade and discover what is waiting for them under the covers.

Until next week, take care!

BizTalk Integration Summit 2013 Recap

The week before this last holiday week was quite a time to be in the BizTalk world. The BizTalk Integration Summit 2013 kicked off at the swanky Fairmont Olympic Hotel in Seattle, Washington bringing with it a defined release cadence and roadmap, new cloud-based product announcements, and even a peek into the plans for the upcoming on-premise version of BizTalk Server.

The First Peek

The first peek into what the Thursday-Friday event would hold is captured in this agenda from the first day. The first day contained a single track of content, where all attendees would be able to experience the same messaging and presentations. The second day provided two tracks for those who wanted to chose their own adventure.

agenda

Familiar Faces & New Faces

The man in the Red Polo himself – Scott Guthrie kicked off the event – and also took the opportunity to announce the general availability of Windows Azure BizTalk Services. To clarify, it’s real and released – no longer in preview – but also not a static unchanging thing (as we will see later).

scottgu

We also had a new face bringing the initial welcome – Mark Moritmore. He’s the man behind the event, having received the torch passed by Kent Brown (who is now working with NY-based integration powerhouse Nimbo). This guy is passionate about the product, and is a long-term thinker. He’s exactly what BizTalk needs as organizations continue to choose BizTalk Server and Windows Azure BizTalk Services for mission critical integration applications that are expected to have long lifespans.

mark

Updated WABS SDK

The announcement of the general availability of Windows Azure BizTalk Services left me itching to get the bits, and “play” with them for lack of a better term. However, all the turkey from Thanksgiving put me a week behind the times and I just had a chance to download/install the freshly released SDK last night. One of the changes I noticed right away (from the last time I had installed one of the preview SDKs), is that I wasn’t having to hunt for miscellaneous MSIs to install the pre-requisite components. The new streamlined setup just makes it happen (though you’d better have a SQL Server or SQL Server Express instance handy if you’re planning on using the BizTalk Adapter Service – it’s not going to do that for you).

auto_install_dependencies

One of the other things demoed at the summit was the BizTalk Service Explorer Visual Studio Extension. This is a nice little plug-in for the Server Explorer window that allows you to interact with BizTalk Services artifacts in really cool ways (e.g., submitting a test message without leaving Visual Studio). While the UI looks/feels clunky at times, it definitely gets the job done. After seeing it, this will be on my list of extensions to install by default.

biztalkservice_explorer

Misc Thoughts About WABS GA

Looking at WABS from the perspective of migrating existing on-premise solutions to the cloud, I’m not sure the story is really complete there (and that’s not some huge secret – WABS is still in active development, and upcoming changes will erase most of the concerns I’m about to mention).

First, as noted in an earlier post, I did have some expectations around the map upgrade tool that still weren’t really met with this release. I had at least hoped the limitations in the preview would not be seen in the final release, or that I would get happier results than I was seeing before out of the tool at the very least.

Another tool that ships with the SDK is a TPM migration tool (essentially to move parties/profiles/agreements to the cloud. While this is actually done pretty well, there is still some mismatch in capabilities (e.g., EDIFACT is available on-premise, but is still a future delivery for WABS) that will prevent moving all solutions.

I took a look at the X12 schemas while thinking through some integrations I’ve worked on in the past, and noticed an absence of the schemas used for HIPAA sub-document splitting. To be fair, I haven’t actually had an opportunity to test to see if that functionality is present (for the most part the schemas are 100% clones of the on-premise schemas, so maybe those extensions are implemented in the WABS version of the EDI Disassembler), but it’s not looking promising. So those integrations would potentially stay on-premise as well.

Another thing that makes me more than a little bit uncomfortable is the direct deployment from Visual Studio. I’m hoping we will see more solutions like Michael Stephenson’s AppFx.ServiceBus.Build that will make deployment a happier experience (and hopefully a fully automated one that doesn’t live in Visual Studio).

That’s not the end of the story though. I don’t intend to write this in such a way that it seems I’m saying that WABS is somehow unsuitable. Instead these are the places where there is still work to be done – and I’ll emphasize again – work is being done. EDIFACT support was confirmed as coming, along with a new rules engine (with new tooling), support for Windows Workflow Foundation workflows, AS2 support, and other fun extensibility points (e.g., BAM-like tracking, easy to build/lightweight adapters). So we’re getting a better B2B story, and also starting to see the seeds of a BPM story emerge within WABS.

Essentially WABS is doing great things already (for straight-forward X12 integrations built fresh for WABS), and will be doing even better things in the future.

BizTalk 2013 Updates Since Release

Another point of discussion was the present state of BizTalk Server 2013 on-premise, as well as a road map for the future. Since the release of BizTalk Server 2013, the BizTalk team has been hard at work, and shipping updates and new functionality (e.g., HIS 2013, which had been delayed due to close work between the product team and TAP customers to get everything happy and ready for release).

2013updates

BizTalk Futures (2013 R2 and Beyond)

Looking to the roadmap ahead, the team made a commitment to a release cadence (that in a way is a continuation of what we have already seen, but which hadn’t so much been publicly formalized). The release cadence going forward will see:

  • Major BizTalk Server release every other year
  • R2 BizTalk Server release every year there is no major release
  • Cumulative update package each quarter
  • WABS refresh each quarter

release_cadence

Looking at this slide, I feel like I’m looking at a description of the release cadence for Visual Studio. Which is nice, because in a way this provides some level of guarantee that I’m going to be able to develop BizTalk Server artifacts in vLatest of Visual Studio (give or take a few months in this case).

As part of the commitment to this release cadence, an announcement was made for BizTalk Server 2013 R2, which will bring:

  • Platform alignment
  • JSON Support (so much for all that work writing pipeline components!)
  • Proxy support for the SFTP adapter (as well as SSO support)
  • Authentication improvements for Service Bus
  • Healthcare Accelerator Improvements (slides to follow with specifics)

2013r2features2013r2features2

Exciting Road Ahead

When you get down to the bottom of it, it really is an exciting road ahead for BizTalk Server. I had a lot of fun this year at the BizTalk Integration Summit, and I’m hoping that this becomes an annual tradition and opportunity for the BizTalk community to come together in-person, and have a dialog with the team that’s building the product and service.

Before I close this post out, I want to give a shout out and thanks to everyone that stopped by our table and has found themselves sporting a new tablet cover with the QuickLearn Training logo (spread the love!), and a thanks to everyone who made it to our party at the Rock Bottom Brewery – lots of good times and good conversations were had there!

If you want to get some other perspectives on the summit, Saravana and Steef Jan (among others) beat me to the punch and have offered their thoughts here:

Well, it looks like I don’t have an excuse to sit in this restaurant any longer (one of the upper floors of a seemingly endless concourse at DEN). I’m going to go catch the rest of my flight out to Houston (completely different part of Houston than last time) to teach some TFS this week, and I’ll be back in short order. Until then, take care!

Mapping Code Pairs in the WABS Mapper

This post is the twenty-third in a weekly series intended to briefly spotlight those things that you need to know about new features in BizTalk Server 2013.

Assuming that one of the major benefits of using Windows Azure BizTalk Services is being able to offload some of our EDI processing to the cloud, then a major use case for the new WABS mapper becomes EDI mapping. One of the fun tasks that comes out of that is dealing with code pairs (if you’re interested in the on-premise side of this problem, the best coverage to date is found here). In this post, I’m going to examine how one might approach this problem using the new tooling available in the WABS SDK.

Give Me an Example of Code Pairs

Maybe you’re reading this and thinking to yourself, “I don’t really care about EDI, but WABS looks interesting,” and you still want to be able to gain something from this discussion without getting buried in the esoterica that is EDI. If that’s the case for anyone out there, here’s an example of the problem I’m going to try to solve using the WABS mapper.

We have an input that is a repeating record of items where there is a qualifier that describes that type of item we’re looking at, and a value that provides the value of the item. Consider this schema:

image

Here we have (at some point in the schema) a list of phone numbers, identified by a Type field element (our qualifier) and the Number field element (our value). The Type field is defined as follows:

image

We have an output that is a flat listing of items where we have a set of fields representing a few of these different types of numbers:

image

So what we are trying to do is iterate through the list of numbers in the input, map over the value for the item with the qualifier Mobile for the CellNumber output, and grab the value of the Home qualified number and toss it into the HomeNumber output field. It’s something that looks trivial, and yet is annoying to do en masse using the BizTalk Mapper:

image

In a Land Without FLATTENING Value MAPPERS

So what does this look like when brought over to the land of WABS using .trfm style maps? Well, looking into the toolbox, we do have an item called Conditional Assignemnt, which is essentially a Value Mapper:

image

If we run with that (combined with some Logical Expression map operations, we end up with something like this:

image

Unfortunately, this isn’t a Value Mapper (Flattening) type map operation. As a result, this actually will only work if we are interested only in a single qualifier, and that qualifier happens to show up in the first code pair that appears in the document.

So what do we do now? Do we reach to XSLT and code it up by hand (after-all, it is super easy to pull off)? Let’s take a look at how the migration tool that comes with the SDK handles this scenario:

image

It adds two MapEach operations to the map, that are configured something like this:

image

As you can see, the MapEach was made to be conditional upon the Type field element being compared to a constant string input. However, this conditional check is repeated inside the MapEach operation within the chained Logical Expression and Conditional Assignment map operations. While the map technically works as generated by the tool, it looks horrible, and it really feels like there ought to be a better way.

Another Approach

If you don’t want to use the MapEach approach (which really wouldn’t be so bad if our two nodes weren’t hanging out at the root as siblings), then we can take advantage of the ability of the new mapper to generate lists of items in memory while working through the mapping (really this is kind of similar to the Table Looping and Extracting functoid).

In order to make that happen, we use the Create List map operation, and then at some point inside include the Add Item operation to add items to the list. The Create List operation is configured to include the names of the fields of each item in the list. It then includes the ForEach operation with a nested Add Items. The ForEach is pointed to by the parent repeating node of our code pairs. The Add Items operation is pointed to by each value we want to include for each source item:

image

Here’s what the configuration of the Create List map operation looks like:

image

Once we have the list created, we can select values out of the list using the Select Value operation (in fact we’ve used two of them):

image

Does it Work?

Technically, both approaches worked, and took this input, and generated the output shown below it:

image

image

I’m going to be honest here, it still feels comparatively clunky, and I’m probably going to be reaching for that XSLT override pretty frequently. That is, unless there’s yet another way that’s cleaner and more simple than those presented thus far (this is one of those posts that I hope to revisit in the future with a correction showing an even better way to approach the problem).

Next Week

Next week, I will be wrapping up this series and shifting focus back to day-to-day BizTalk Development and all the fun things that we’re actively working on there.

Until then, happy mapping with shiny new tools!

Building Blocks of Windows Azure BizTalk Services

This post is the twenty-second in a weekly series intended to briefly spotlight those things that you need to know about new features in BizTalk Server 2013.

Since the last BizTalk Summit, there has been a lot of coverage within the greater BizTalk Community of Windows Azure BizTalk Services – the Azure-backed offering that brings pieces of the core BizTalk Server functionality to the cloud. This is something that QuickLearn had early opportunity to get hands-on with, and also something for which we were able to create labs (even before public availability). At the same time, it’s something that we haven’t really written about until now.

This week, rather than going through specific how-to guidance for pieces of the offering, I instead want to take a step back and look at the offering as a whole, so that we all understand the pieces that are there (and can see the forest through the trees). If you want to follow along, go ahead and download the Windows Azure BizTalk Services Preview SDK.

Once installed, you’ll find that Visual Studio has been juiced up with two additional project templates found under the BizTalk Services heading:

image

The first project type BizTalk Service allows you to define Itineraries (not in the ESB sense, though they do display a complete message flow, much like ESB), and the configuration for Bridges (much like a BizTalk Pipeline).

The second project type BizTalk Service Artifacts allows you to define schemas and maps (.trfm files, though you can convert your existing .btm files with some limitations using a tool provided at the SDK download link). The mapper is something I’m not going to be discussing any further, and as a result I will instead recommend this excellent blog post on the subject by Glenn Colpaert, and then also the official documentation on the conversion process/limitations.

So we can create these Itineraries, configure these Bridges, and deal with Schemas and Maps, but how does this all fit together within the context of Windows Azure BizTalk Services? To understand the answer to that, we have to first address the three things that Windows Azure BizTalk Services is trying to do:

  • Rich Messaging Endpoints (Itineraries + Bridges + Maps)
  • BizTalk Adapter Service (Service Bus Relay + Local Helper Service + BIzTalk Adapter Pack)
  • Business to Business Messaging (EDI Schemas + All of the Above)

Rich Messaging Endpoints

image

I’m going to start with the weirdest of the bunch – Rich Messaging Endpoints. The developer experience of this is similar (to a point) to defining an ESB Toolkit itinerary, but only inasmuch as you’re seeing the entire message flow within a single diagram. Here you can define certain sources (FTP/SFTP) for messages, and have them routed through Bridges (similar to Pipelines), and then routed out to certain destinations (for which CBR can be used to route the messages to one or multiple destinations).

In order to specify the settings necessary to connect to those destinations, you often find yourself within a configuration file defining WCF related binding settings directly:

image

Double-clicking on a bridge within the itinerary will bring up a semi-clone of the Pipeline Designer with a fixed set of components in place (as well as a location to specify maps to execute directly as part of the pipeline):

image

In terms of integration patterns, the bridges here are giving us a subset of the VETRO pattern (namely the VETR part of it): Validate, Enrich, Transform and Route. Where is the Route part of the equation you might ask? Well, you’ll actually find it in the Properties window for the bridge, by clicking the button in the Route Ordering Table property:

image

Here, we can do context-based routing that helps determine the destination for each message coming from the destination (though here, I have only connected a single destination endpoint).

BizTalk Adapter Service

image

What happens if the list of destinations doesn’t suit me? What if I want to take information from an SFTP drop, transform it into something that I could use to generate a record within a table in SQL, and then directly insert that record into my on-premise system? In that case, I’ll find myself reaching to the BizTalk Adapter Service feature.

This one has a nice list of dependencies that need to be in place before you can even use it (e.g., Windows Server AppFabric, BizTalk Server Adapter Pack), but once you have those, it’s fairly straight-forward to setup.

What it’s really providing is a single WCF endpoint that is exposed over a Service Bus Relay endpoint (and thus accessible from anywhere in the world – even if hosted behind a fairly strict firewall). This single endpoint can be passed messages destined for any number of internal systems that you can setup through the Server Explorer interface within Visual Studio. The BizTalk Team Blog actually had a pretty decent article on the topic back in June that sadly generated 0 comments.

Essentially, this is allowing you to bring your LOB systems into the mix to play along with everything else already mentioned (assuming those LOB systems have WCF adapters included in the BizTalk Adapter Pack).

Business to Business Messaging

The final capability that WABS is bringing to the table is B2B messaging (i.e., EDI). We have the ability through a special portal (still in preview) within the Windows Azure Management interface to create Parties and Agreements a la BizTalk Server on-prem. In fact, the same exact schemas are used to define the X12 messages, so if you’ve already had to do some schema customizations on a per-partner basis for in-house integrations, those same changes can now be brought to the cloud.

Pulling it All Together

Is this replacing BizTalk Server on-premise as we know it? Not quite. We have a lot of the same pieces: Transport (Limited)/Translation (Limited)/Transformation and Context-based Routing (Content-based Routing when using the Enrich stage of the Bridge). We are missing more complete process orchestration with exception handling / transactions / compensation (Orchestrations). We are missing rule-driven injectable logic (Business Rule Engine), among other things (though I’m stopping the list here to avoid debate about similar (but not quite) functionality in Azure in general).

So what do we do with this? Use it to solve integration problems that use a hybrid between cloud-based and on-premise resources (e.g., http://msdn.microsoft.com/en-us/library/windowsazure/hh859742.aspx). Use it when an on-premise BizTalk installation would be overkill for the task, but it can be done happily using the tools available in WABS. Use it to take over the most taxing parts of your B2B processing, leaving BizTalk Server on-premise to do those things it’s great at.

Ultimately, it’s up to you. BizTalk Server 2013 and Windows Azure BizTalk Services together give you the power to decide how to break up your integration work, and how you approach the integration challenges that are thrown your way.

If you haven’t yet given the SDK a download yet, go do it. Though the service is still in preview, the SDK has been available for quite some time now, and great things are right around the corner – so keep your skills sharp.

That’s all for now!

BizTalk Server 2013 New Features Series Index

I thought it would be wise to create a post that can serve as a continually updated index of the articles that I have written within the BizTalk Server 2013 New Features Series here on the QuickLearn Team Blog. This post is that listing, and it also includes links to the code developed for each post where applicable.

Without further ado, here’s where we have been so far:

  1. What the BizTalk Server 2013 Mapper Updates Mean for You
    Introducing XslCompiledTransform and returning null values from functoids
  2. Acknowledging a Subtle Change in BizTalk Server 2013
    Examining X12 999 Acknowledgments
  3. Windows Azure Service Bus Queues + BizTalk = Out of Body Experiences
    How to handle messages that are only properties [Code Sample]
  4. SharePoint Integration without Cheating in BizTalk 2013
    Introducing the Client-side Object Model support in the improved SharePoint adapter.
  5. Why Updated Platform Support Matters
    Using PipelineTesting library in concert with the Fakes/Shims support of Visual Studio 2012 [Code Sample]
  6. A Brief Tour of the BizTalk Server 2013 Evaluation Windows Azure Virtual Machine
    How to quickly setup the BizTalk 2013 IaaS offering
  7. Publishing a WCF Service via Service Bus Relay in BizTalk Server 2013 (Part 1)
    Using the installed-by-default AppFabric connect functionality to expose/consume a NetTcp endpoint in IIS via Service Bus Relay — for a client that understands Service Bus Relay
  8. Publishing a WCF Service via Service Bus Relay in BizTalk Server 2013 (Part 2)
    Using the new adapters to expose/consume a BasicHttp endpoint hosted In-Process via Service Bus Relay — so that it is transparent to the client
  9. Automating and Managing BizTalk Server 2013 with PowerShell
    Shows off the soafactory PowerShell Provider for BizTalk while also trying to give PowerShell 101
  10. Exploring the Out-of-the-Box Support for SFTP in BizTalk Server 2013
    Using the SFTP adapter to talk to an Ubuntu Linux box on Azure
  11. Dynamic Send Port Improvements in BizTalk Server 2013
    Shows off the new Send Handler Mapping dialog for Dynamic Send Ports
  12. BizTalk Server 2013 and TFS 2012 – Why Can’t I Lock the Files
    Shows how to configure your TFS workspace to be BizTalk ready
  13. BizTalk Server 2013 Support for RESTful Services (Part 1/5)
    GETing Data + Context Property URL Mapping
  14. BizTalk Server 2013 Support for RESTful Services (Part 2/5)
    Receiving POSTed XML + URL Rewriting [Code Sample]
  15. BizTalk Server 2013 Support for RESTful Services (Part 3/5)
    POSTing UrlEncoded Data to ASP.NET Web API Service [Code Sample]
  16. BizTalk Server 2013 Support for RESTful Services (Part 4/5)
    JSON Decoding + Cross-domain communication with JSONP and CORS [Code Sample]
  17. BizTalk Server 2013 Support for RESTful Services (Part 5/5)
    Using OAuth to authenticate with cloud service providers’ APIs
  18. BizTalk Server 2013: Discovering Dependencies
    Exploring the Dependencies Statistics Panel
  19. Getting Started with the ESB Toolkit 2.2 Extensibility Samples in BizTalk Server 2013
    Examining ESB Install/Configuration Process for Development / Troubleshooting the Designer Extensibility Samples
  20. BizTalk Server 2013 Ordered Delivery Improvements
    Discussing the specific improvements that are seen in how BizTalk Server 2013 handles Ordered Delivery.
  21. BizTalk Server 2013: Eliminating the Dependency on SQL Server 2005 Notification Services
    Configuring BAM Alerts without touching SQL Server 2005 Notification Services.
  22. Building Blocks of Windows Azure BizTalk Services
    Overview of Windows Azure BizTalk Services Features.
  23. Mapping Code Pairs in the WABS Mapper
    Demonstrates attempted approaches for dealing with EDI code pairs using the new WABS mapper.
  24. Top 5 Indicators of the BizTalk Server 2013 Community’s Vitality
    Survey of the BizTalk Server 2013 ecosystem

Exploring the Out-of-the-box Support for SFTP in BizTalk Server 2013

This post is the tenth in a weekly series intended to briefly spotlight those things that you need to know about new features in BizTalk Server 2013.

This week we’re going to examine the new SFTP adapter included with BizTalk Server 2013. You read that correctly, we finally have an SFTP adapter out of the box with BizTalk Server 2013!

Background Information

In the 2010 release, we saw the addition of the FTPS adapter (or rather the enhancement of the existing FTP adapter to support SSL), which was great for those that wanted to be able to transfer files between systems in an encrypted fashion. Since the SSL support was built into the FTP adapter, we were also able to take advantage of other enhancements (e.g., read-only FTP mode) at the same time.

The SFTP adapter is designed to support a completely different scenario. Here we’re not doing regular FTP with SSL encryption; instead we’re using an SSH tunnel to access a remote file system (with a protocol that is most definitely not FTP).

As a result, SFTP functionality isn’t something that could be easily hacked onto an existing adapter. Instead, there is a brand new adapter built from the ground up for inclusion in BizTalk Server 2013.

Putting it to the Test

While I have a few Linux boxes at home of various flavors, and even a few running on EC2, I have not yet spun-up a non-windows VM on WIndows Azure (it just doesn’t feel right). Thus, I felt that this would be a fitting opportunity to do so.

Unlike with the Windows Virtual Machines, the Quick Create experience does not allow you to specify a custom user name, so the user name I will be using (along with everyone else on the service) is azureuser. As part of the provisioning process, Windows Azure will configure sshd and generate an RSA 2048-bit key pair. After provisioning, the key thumbprint will be visible in the Windows Azure Management Portal:

image

From there I fired up PuTTY to login to the VM. The only details that you will need are the host name (your-vm-name-here.cloudapp.net), the user name (azureuser), and the password that you chose when creating the VM.

From there, I used nano1 to create a quick text file for testing purposes:

image

Great. Now I have a file. Let’s see if BizTalk 2013 can now access and do something meaningful with it.

Creating a Receive Location

For the purposes of this blog post, all I really want to do is get the file into BizTalk, examine the properties that the adapter is promoting, see how it responds in read-only situations, and then route it out to a file location to get it out of my message box and placed somewhere meaningful.

To that end, I created a one-way receive location and selected the SFTP adapter. Upon configuring the adapter, I was greeted with the following properties:

image

It is interesting to see that Microsoft ditched their regular language used to refer to public key fingerprints (the “thumbprint”), and went with the more widely used terminology. Other than that, there is not a lot of surprises here for those who have worked with the plain-old FTP adapter.

A few notable things are missing here that I would have expected, though. I would have liked to see integration with SSO (i.e., an SSO Affiliate property like we have in the FTP adapter), and I would like to have seen some read-only support as well (i.e., a Delete After Download property, and an Enable Timestamp comparison property)2.

Filling in the property grid with the information from my Azure VM, I ended up with:

image

In the Security section, I changed the AcceptAnySSHServerHostKey because I wanted to be certain that it was indeed my VM that was accepting the connection (and I knew the fingerprint to expect). I set the SSHServerHostKeyFingerPrint property to the finger print listed on the Window Azure Management Portal (after adding a colon ‘:’ every two characters to put it in a more standard format).

Hitting a Wall

Well, it always happens at some point in the day – a road block that leaves me scratching my head until after a cup of coffee (hopefully it’s not just me, eh?).

In this case, the road block was an error message in the Event Log that took the following form (reproduced for those exercising Bing-fu3 to resolve a problem):

The Messaging Engine failed to add a receive location “OrdersIn_SFTP” with URL “sftp://my-server-url-is-not-your-url.cloudapp.net:22/home/azureuser/*.txt” to the adapter “SFTP”. Reason: “Microsoft.BizTalk.Adapter.SftpInvoker.SftpException: Open SFTP connection error.
   at Microsoft.BizTalk.Adapter.SftpInvoker.SftpInvoker.Open()
   at Microsoft.BizTalk.Adapter.Sftp.SftpConnection.OpenUnderlyingConnection(SftpConnectionProperties connectionProperties)
   at Microsoft.BizTalk.Adapters.CommonHelpers.Connection`3.ConnectionPool`3.GetConnection(T1 connectionProperties, TimeSpan timeout)
   at Microsoft.BizTalk.Adapters.CommonHelpers.Connection`3.GetConnection(T connectionProperties, TimeSpan timeout)
   at Microsoft.BizTalk.Adapter.Sftp.SftpRLConfig.ValidateConfiguration(SftpReceivePropertyBag receivePropertyBag)
   at Microsoft.BizTalk.Adapter.Sftp.SftpRLConfig.CreateBinding(RHConfig rhConfig)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.BtsServiceHostBase.InitializeRuntime()
   at System.ServiceModel.ServiceHostBase.OnBeginOpen()
   at System.ServiceModel.ServiceHostBase.OnOpen(TimeSpan timeout)
   at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfReceiveEndpoint.Enable()
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfReceiveEndpoint..ctor(BizTalkEndpointContext endpointContext, IBTTransportProxy transportProxy, ControlledTermination control)
   at Microsoft.BizTalk.Adapter.Wcf.Runtime.WcfReceiver`2.AddReceiveEndpoint(String url, IPropertyBag adapterConfig, IPropertyBag bizTalkConfig)”.

Say what? An error that goes all the way down to a WcfReceiver class? This just got crazy.

After spending some time changing settings to see which configuration option might be the culprit, I found that setting the AcceptAnySSHServerHotKey to True resulted in the error disappearing and the Receive Location staying alive. Unfortunately, I actually do care about that setting being set to False – I don’t want someone to impersonate my box and get away with it!

So then, what must we do here? Well, it turns out if one will simply RTFM, they will find happiness. So I decided to do that and found this article on MSDN which indicated that I had specified the host key fingerprint in the wrong format. It must include the type of key, the size of the key (in bits) and then the fingerprint.

Azure VMs use a 2048-bit RSA key, so that will need to be reflected in that setting. After some adjustments, my properties were updated to look like this:

image

Suspending the File for Inspection

Since I really wanted to get to the bottom of what this adapter could do for me, I decided to not create a Send Port initially (but rather to just have it suspend the message so that I could inspect the context, and then decide what to do with it later – if anything).

So I enabled the receive location and waited. The file appeared in the message box within the 5 second polling interval as configured, and yielded the following look into its context:

image

From the image above, you will find that there are quite a few properties from the WCF-properties namespace. In addition, you will see that the ReceivedFileName property from both the file-properties namespace, and the new sftp-properties namespace (dated 2012) has been written to the context (but not promoted). Beyond that, there wasn’t really anything super interesting :-/.

Checking on the server side of things, the file was indeed removed as expected:

image

What about Read Only?

I decided to disable the receive location for a few minutes so that I could setup a test wherein there was a file that a user would not have permissions to delete (this also required adding another user which I affectionately named nickh):

image

After I setup a new locked down directory, with a new locked down file, preventing all but read-only access to a new locked down user, I decided to update the Receive Location properties to see how BizTalk would react to the situation.

Guess what? The file isn’t deleted even after leaving the Receive Location running for some time, and the Event Log isn’t full of errors either. However, my output directory (yes, I did finally setup a Send Port to route out these suckers), is full:

image

Now about the Event Log, it is definitely not full of errors, but it is fairly full of warnings at this point:

image

The error reads:

The adapter “SFTP” raised an error message. Details “System.Exception: Message: Delete file error.. Sftp Error Code: ‘3221488222’. Sftp Error message: ‘The system cannot find the file specified.
‘. —> Microsoft.BizTalk.Adapter.SftpInvoker.SftpException: Delete file error.
   at Microsoft.BizTalk.Adapter.SftpInvoker.SftpInvoker.DeleteRemoteFile(String filename)
   at Microsoft.BizTalk.Adapter.Sftp.SftpFileReceiver.DownloadComplete(String fileName, Boolean deleteFile)
   — End of inner exception stack trace —
   at Microsoft.BizTalk.Adapter.Sftp.SftpFileReceiver.DownloadComplete(String fileName, Boolean deleteFile)
   at Microsoft.BizTalk.Adapter.Sftp.SftpRequestContext.Reply(Message message, TimeSpan timeout)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.Reply(MessageRpc& rpc)”.

Essentially, it will fail to delete the file, but still process the contents. If I need to receive the same nightly batch (with the same file name every day), that’s not a problem – I can set my polling interval to 1 day, or even mess with service windows to help out. But if the nightly batch has a new name every night (and the others remain in the directory), my life will be somewhat painful as I work to deal with the duplicate receives.

AN ASIDE: Peeling Back the Layers

After seeing WCF in the error message I received at the beginning of this exploration, I had to do some digging around to satisfy my curiosity.

Snooping around a bit inside the BizTalk Server 2013 installation direction, you will find how it all fits together. There’s an sshlib.dll and sshmessages.dll inside the Bins32 & Bins64 directories – these appear to be unmanaged code that implements the raw connection. Alongside those, we see a managed library named Microsoft.Biztalk.Adapter.SftpInvoker.dll. Finally, at the root of the installation directory, there is a Microsoft.BizTalk.Adapter.Sftp.dll. Visual Studio’s object browser reveals that this is, indeed, like the Service Bus adapters, yet another new WCF adapter (likely using a custom binding that somehow relies on the aforementioned components):image

Unfortunately, it does not appear to publicly expose whatever binding it is using, so it is unlikely that the implementation here will be re-usable outside of BizTalk.

Bringing it to a Close

I hope this article helped you out in one way or another. Maybe you found out that there is an SFTP adapter out of the box in BizTalk Server 2013 and that was big news for you. Maybe you were curious about how it would handle read-only situations, and now you have some idea of that behavior. Or maybe you were struggling with the configuration, and something here has shed light on an issue you were encountering. Either way, best of luck to you on your integration projects.

If you want to learn more about the new features in BizTalk Server 2013, or even want to learn BizTalk Server for the first time, check out one of our BizTalk Server 2013 Developer Immersion of BizTalk Server 2013 Developer Deep Dive classes running now from your home, at your workplace, or right here in Kirkland, WA.

1I choose to be Switzerland in the great editor war – though you will find on my VM that vi is installed while emacs is not.

2Listen, I’m not going to be that guy and not tell you that you can still achieve your goal here. Check out the AfterGetActions enum in this bad boy. That adapter still exists, and still lets you do what you want to do.

3Yes, that is me shamelessly selling out. They give you a free Redbox rental after 110 rewards points. I’m a sucker for points and free movies.

Publishing a WCF Service via Service Bus Relay in BizTalk Server 2013 (Part 1)

This post is the seventh in a weekly series intended to briefly spotlight those things that you need to know about new features in BizTalk Server 2013.

This post will focus on a feature that is technically present in BizTalk Server 2010, but was not installed by default. This feature was a subtle enhancement to the BizTalk WCF Service Publishing Wizard – namely the ability to publish a Windows Azure Service Bus Relay Endpoint without leaving the wizard (and also without using one of the new adapters that has Relay in its title).

Even if you didn’t know that this feature existed, you will find it the first time that you attempt to publish any BizTalk artifact (schema/orchestration) as a WCF service in BizTalk Server 2013. It takes the form of the following screen within the BizTalk WCF Service Publishing Wizard, which appears immediately after choosing to publish a service (rather than simply a metadata description):

publish_sbendpoint

If you check this box and proceed through the wizard (regardless of adapter selected on the first screen), you will see the following screen at the end of the typical wizard:

publish_sbendpoint_part2

The wizard assumes that you have already gone to the Windows Azure Management Portal and created a Service Bus namespace (effectively reserving yourself a sub-domain of servicebus.windows.net, and setting up an access control namespace for claims resolution and authorization purposes). If you’ve never done that before, it can be done through the New menu as shown below:

servicebus_namespace

The next page of the wizard will require information necessary for BizTalk to authenticate with Windows Azure and prove that it is indeed the destination endpoint for the relay. This information is also obtainable only from the Windows Azure Management Portal. In order to access this information, you will need to click on your newly created namespace in the list of Service Bus namespaces, and then click the Connection Information button at the bottom of the page:

connection_info_button

This will bring up the following listing, of which you really only need to worry about the Default Issuer (Issuer Name in the wizard), and Default Key (Issuer Key in the wizard):

connection_info_listing 

Once you gather this information, you’re ready to have a copy/paste party and fill out the last page of the wizard (unless you really want to live life in hard mode, don’t enable client authentication for metadata exchange – if you feel the need to do that, you may as well pass around the raw WSDL to whomever needs it and just forget about a MEX endpoint):

publish_sbendpoint_part3

Once you’re up to this point, you can sprint to the finish by clicking Next, Create (after reviewing the wonderful WSDL and making sure it’s something that you can be proud of), and then finally Finish.

Tackling Common Issues

Outside of the typical issues one might encounter hosting a WCF Service in a BizTalk Isolated Host (e.g., App Pool Identity needs permissions to Message Box database, correct .NET Framework version needs to be selected for App Pool, Receive Location must be started), you may also encounter a new one:

image 

Again, for those using Google-fu in attempt to resolve an error, the error message reads:

Invalid element in configuration. The extension name ‘transportClientEndpointBehavior’ is not registered in the collection at system.serviceModel/extensions/behaviorExtensions

And it’s highlighting the portion of the configuration file that includes the key for Service Bus authentication.

Before getting right to the resolution, let’s recap what we’re looking at. Going through the wizard, I had selected to host a WCF-WSHttp endpoint internally in a BizTalk Isolated Host (i.e., running in an IIS App Pool rather than a BizTalk Host Instance process). I then indicated that I wanted to also expose the service through a NetTcp relay endpoint hosted externally (on Windows Azure).

My local IIS instance has been provided all the configuration information that it needs to coordinate with Windows Azure and make the relay live, but it currently doesn’t know what to do with it – which is why I have the error, and why my relay won’t be alive yet.

In reality, this all could have been avoided by reading. Specifically reading the last page of the BizTalk WCF Service Publishing Wizard, which should tell you that you need to install the AppFabric 1.0 SDK before any of the relays will work. Again, this is functionality that was technically available in the previous generation, hence the older SDK version number.

How to Know That It’s Working

If you have made it this far successfully, hitting the local service endpoint should give you a page that looks something like this:

workingservice

If you go to the Windows Azure Management Portal and dig into the namespace you created, you should see something like this:

relay_azure

It’s Not Real Until I Can Consume It

If you’re anything like me, this is an unsatisfactory ending point – the service doesn’t really exist until I can consume it from some client. To make it as fair as possible, I am hosting the service inside a fairly locked down Windows Azure Virtual Machine, and I will be consuming it from my laptop connected ultimately via a microwave connection somewhere in the shadow of Mt. Pilchuk.

To consume this beast, we need to know our namespace (e.g., unique-name.servicebus.windows.net), and the mex endpoint exposed via relay (seen in the screenshot above). From there I can construct the address I would use in the Add Service Reference dialog in Visual Studio. In this case, that URL will be: https://unique-name.servicebus.windows.net/GetItemServiceDescription/GetItemService.svc_mex

Upon consuming the service, you may notice that the WSDL importer chokes hard on the WSDL it’s finding. It tries to build up a custom binding to make the call, and it’s finding that it doesn’t really know anything about some of the binding elements required:

choked_importer

Again for those searching for this specific error, the message reads:

WsdlImporter encountered unrecognized policy assertions in ServiceDescription ‘YOUR NAMESPACE HERE’

In reality, we don’t need to build a custom binding to make this all happen. Instead, we can use the netTcpRelayBinding to make the connection. An easy way to make that happen is to install the Windows Azure Service Bus NuGet package to our project. We can start out by using the Manage NuGet Packages context menu item in Solution Explorer:

manage_nuget_packages

Then search for the Windows Azure Service Bus package, and click the Install button:

manage_nuget_packages_part2

This should update the App.config file of your application to include the following WCF extensions:

appconfig

From there, you will want to update your endpoint to reference the netTcpRelayBinding (instead of the custom binding that the WSDL importer failed to generate properly):

    <client>
      <endpoint address="sb://YOUR-NAMESPACE.servicebus.windows.net/YOUR-SERVICE-LOCATION/YOUR-SERVICE-HERE.svc"
        behaviorConfiguration="sharedSecretClientCredentials" binding="netTcpRelayBinding"
        contract="RelayedItemService.ItemService" name="RelayEndpoint" />
    </client>

You will also notice above that we have assigned a behaviorConfiguration — one that currently does not yet exist. So next, we will need to add an endpoint behavior (inside the system.serviceModel section of the App.config) to perform client authentication (if you don’t want to re-use the same credentials as before, make that visit over to the ACS Management Portal that the Connection Information page is begging you to do):

    <behaviors>
      <endpointBehaviors>
        <behavior name="sharedSecretClientCredentials">
          <transportClientEndpointBehavior>
            <tokenProvider>
              <sharedSecret issuerName="owner" issuerSecret="YOUR SHARED SECRET VALUE HERE" />
            </tokenProvider>
          </transportClientEndpointBehavior>
        </behavior>
      </endpointBehaviors>
    </behaviors>

This is something that has changed more recently. In fact, I couldn’t find a single place documenting what this behavior should actually look like right now. Hopefully that will change.

Invoking the Service

Once we get through all the configuration craziness, we’re ready to make quick work of calling the service with two lines of code (backed by a billion lines of configuration):

YourServiceDotNetNamespace.YourServiceClient client = new YourServiceDotNetNamespace.YourServiceClient("RelayEndpoint");
client.GetItem(new YourServiceRequest() { Id = "test" });

That’s all it takes.

Final Thoughts

I hope you found some of this helpful. Like I said, this is all technically possible in BizTalk Server 2010, but required additional installation. In BizTalk Server 2013, it is there by default and ready to go (with one tiny SDK install) out of the box.

If you enjoyed this week’s post, then stay tuned. Next week, we will be accomplishing the same tasks with a service hosted in-process in BizTalk Server 2013 using one of the new WCF Relay adapters.

A Brief Tour of the BizTalk Server 2013 Evaluation Windows Azure Virtual Machine

This post is the sixth in a weekly series intended to briefly spotlight those things that you need to know about new features in BizTalk Server 2013.

In this blog post, I’m going to take a little bit of a departure (for this week only) from the tone of my earlier posts – which focused on scenarios you will encounter after using the product – and look instead at things that would be of interest to people who are new to product, or are still evaluating BizTalk Server 2013 as an option for enterprise integration.

By far the quickest way to get started with BizTalk Server 2013 is to be handed a machine that has it pre-installed. With the Infrastructure-as-a-Service offering in BizTalk Server 2013, we have just that. In fact, we have it both for production as well as evaluation use.

Just Give Me a VM

While attending “//build/” this week (quick shout-out to those also here), I spent some time in the big room next to the Windows Azure booths. They had 3 challenges for attendees that involved completing certain tasks using the platform, and were offering a prize for each challenge completed. They were designed to sound somewhat difficult, yet prove to the participant just how easy life can be with Windows Azure. The first challenge was to provision a virtual machine running Windows Server 2012 — a trivial task (like it should be).

Well guess what? Provisioning a BizTalk Server 2013 instance is just as easy. You can start by logging into the Windows Azure Management Portal. From there, I can show you all the requisite clicks in a single screenshot:

ProvisionVMClicks 

Coffee Break #1

That should kick-off the provisioning process for the Virtual Machine, and also starts off your first coffee break within this process.

provisioning

Get Connected

So you have a virtual machine running BizTalk Server in the cloud, now what? Well you probably want to get connected, and remote in so that you can make use of it. That can be done by clicking the Connect button in the toolbar while the virtual machine is highlighted in the list of Virtual Machines.

ConnectVM

This will cause a .rdp file to download with all of the settings (DNS name/port) required to make an Remote Desktop Protocol (RDP) connection, and fire up the Microsoft Terminal Services Client (still named mstsc.exe) to make it all happen. Make sure your internal firewall allows this, or you will not have much fun after this point.

You will be connecting using the user name that you entered while creating the virtual machine (not your internal domain account). The virtual machine starts out in workgroup mode, and will not be joined to a domain.

Make IE Usable

I think we all know that secure != usable, and sometimes there is an inverse relationship between the two. You will find this to be the case with the virtual machine that you have just provisioned. It is configured to be a headless server by default, and to have a really small attack surface. That’s fantastic, and I love security by default. However, for our purposes of evaluating the product, and doing some development within the Virtual Machine, it’s probably not our first choice.

As a result, you will probably want to head straight over to Server Manager first thing (which will open up automatically). From there, you will click Local Server, and then proceed to toggle IE Enhanced Security Configuration to the Off state:

IE_ESC

What will this do for us? It will allow us to use the Internet in a more reasonable fashion. But it will also potentially open you up to make poor choices. Do this with care:

IE_ESC_P2

Write Some Code

If you’re evaluating BizTalk Server 2013, it’s likely that you will want to have a development environment available to you – Visual Studio 2012 would be our environment of choice in this case. If we want to do anything interesting with BAM activities and views, we will also need Microsoft Office installed.

So you will probably want to get those installed. With double-click mounting of disk images, it shouldn’t be a problem to install once you have the download links to the ISO files. Where do those live? Right here (but your best bet is to go to the product sites directly and walk through the registration process which will give you a longer evaluation period without persistent nag screens):

Configure This Beast

At this point you will have quite a few tools installed, but a BizTalk Group hasn’t yet been setup. Luckily, since you’re using the Evaluation image, you already have SQL Server installed and ready to serve. At this point you could run through the configuration wizard, or you could go with some happy defaults and just get it done.

So let’s go with the happy defaults. Go ahead and download this file.

Open it up, and then replace the following values, and then save it out to D:\config.xml:

Special String Replace With
@@ADMINUSER@@ Name of user that you created when you created the virtual machine
@@MACHINENAME@@ Name of your virtual machine
@@PASSWORD@@ Password for the user that you created when you created the virtual machine

Once you have that all put together, you can automate the configuration by executing the following commands at the command prompt:

cd “C:\Program Files (x86)\Microsoft BizTalk Server 2013\”
.\Configuration.exe /s “D:\config.xml” /l “D:\config.log”

This should bring you around to Coffee Break #2

What About the Adapter Pack / ESB Toolkit?

At this point you have a usable installation of BizTalk which will give you hours of fun. But what if you want to do something with one of the adapters in the Adapter Pack, or even install the ESB Toolkit? In that case, you will find the binaries for the BizTalk 2013 Evaluation edition at: C:\BizTalkServer2013Evaluation\

Now What?

Now that you have a fully configured BizTalk environment. Kick the tires, play around, look through this blog for some of the capabilities you can experiment with. After that, get some training to find out what BizTalk Server is really capable of. I recommend you check out one of our upcoming BizTalk Server 2013 classes. We have classes for Administrators and classes for Developers now available on the QuickLearn Training calendar.

That’s all from me at //build/, see you next week!

Windows Azure Service Bus Queues + BizTalk = Out of Body Experiences

This post is the third in a weekly series that will highlight things that you need to know about new features in BizTalk Server 2013. One of the major new features of BizTalk Server 2013, is the native ability to integrate with Windows Azure Service Bus through the new SB-Messaging adapter. This week we will be covering what can happen if you’re trying to integrate with applications that are loading up your Windows Azure Service Bus Queues with Brokered Messages made purely of properties.

Out of Context

Context properties are not a foreign concept to BizTalk developers. Indeed, BizTalk messages themselves are made up of both a message context and body parts. The same is true to some extent of Service Bus Brokered Messages – though the implementation differs greatly.

To deal with context properties of Service Bus Messages, the SB-Messaging adapter provides us the ability to promote these properties to the context of the BizTalk message generated upon receipt of a Brokered Message. The adapter configuration that controls this behavior looks like this:

sb-promote-props

One major difference between the properties of a Brokered Message and the context properties of a BizTalk message, is that BizTalk message context properties have a namespace to disambiguate properties that share the same name. Brokered messages have no such thing, and no concept of a property schema for that matter (which allows BizTalk to have an awareness of all possible properties that can exist in the context).

The adapter configuration setting pictured above is a bridge between those worlds that allows the Brokered Messages properties to be understood by BizTalk Server. Of course, you will have to define a property schema for those properties on the BizTalk side. Here’s an example of a really simple one that defines two custom properties:

propertyschema

All of this will function pretty perfectly – assuming that the message sent to the Queue has a body.

Out of Body

At last year’s TechEd NA conference in Orlando, one of the presenters (I believe it was Clemens Vasters) was discussing what Brokered Messages were, and how they were exchanged. In the talk, he mentioned that it was perfectly fine, and even preferable in some cases, to create a Brokered Message that was purely properties without any body content. Certainly this was a light-weight operation that relied only on HTTP headers, and required no overhead for serialization of body content. And at the time, I accepted this, and was happy.

As a result, a lot of my code interacting with the Service Bus involved snippets like this:

var client = Microsoft.ServiceBus.Messaging.QueueClient.CreateFromConnectionString(connString, queueName);
var message = new Microsoft.ServiceBus.Messaging.BrokeredMessage();
message.Properties.Add("CustomProperty", "Value");
message.Properties.Add("CustomProperty2", "Value2");
client.Send(message);

Notice here, we’re just passing around properties, and not dealing with a message body. That’s fine when I’m receiving the message in .NET code living somewhere else, but when moving into the BizTalk world, it became problematic. When attempting to receive such a message (even with the property schema deployed, and everything else otherwise happy), I would receive this (using the PassThruReceive pipeline):

errormsg

For those trying to use Google-fu to solve the same problem, the error message reads:

“There was a failure executing the receive pipeline: ‘Microsoft.BizTalk.DefaultPipelines.PassThruReceive, Microsoft.BizTalk.DefaultPipelines, Version=3.0.1.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35” Source: “Pipeline” Receive Port: “WontMatchWhatYouHaveAnyway” URI: “sb://namespacehere-ns.servicebus.windows.net/somequeuenamehere” Reason: The Messaging Engine encountered an error whlie reading the message stream.

If you came to this article because this error message appeared, and you’re not trying to receive an empty message from Azure Service Bus, just head right back to the Bing results page, and check the next result. Everyone else, stick with me.

Let’s take a step back for a second. BizTalk Server 2013 is designed for integration between systems. I typically should not be adjusting my behavior outside of BizTalk to work with BizTalk. So here I want to take the same approach.

What are our options here? Well clearly we have a failure in the pipeline while processing an entirely empty message body, which doesn’t allow us to get the point where we are processing a map. That removes our ability to use the excellent ContextAccessor functoid to pull the properties into the body of the message upon receipt.

Instead we are left with building a custom solution.

PropertyMessageDecoder Pipeline component

What is this custom solution I speak of? For now, I’m calling it the PropertyMessageDecoder pipeline component. It is a custom pipeline component that creates a message body based on the context properties of a BizTalk message. If you already have a message body, this pipeline component will not help you, since it blindly overwrites the body if one exists. Instead, this is a purpose-built pipeline component for the specific scenario addressed by this blog post.

I started development of this component by using Martijn Hoogendoorn’s BizTalk Server Pipeline Component Wizard. Unfortunately, it hasn’t yet been updated to work with BizTalk Server 2013, so I had to first create/upload a patch for BizTalk 2013.

Once I was able to create a custom pipeline component project, I created a simple pipeline component that took the input message, and dumped the context properties into the message bodies. I also included some configurable properties that allowed one to opt-out of properties from the BizTalk system properties namespace, or to filter out all properties except those from a specific namespace (especially helpful in this scenario).

Here’s the bulk of the Execute method for that component:

public Microsoft.BizTalk.Message.Interop.IBaseMessage Execute(Microsoft.BizTalk.Component.Interop.IPipelineContext pc, Microsoft.BizTalk.Message.Interop.IBaseMessage inmsg)
{
    Stream dataStream = new VirtualStream(VirtualStream.MemoryFlag.AutoOverFlowToDisk);
    pc.ResourceTracker.AddResource(dataStream);

    using (XmlWriter writer = XmlWriter.Create(dataStream))
    {
        // Start creating the message body
        writer.WriteStartDocument();
        writer.WriteStartElement("ns0", ROOT_NODE_NAME, TARGET_NAMESPACE);

        for (int i = 0; i < inmsg.Context.CountProperties; i++)
        {
            // Read in current property information
            string propName = null;
            string propNamespace = null;
            object propValue = inmsg.Context.ReadAt(i, out propName, out propNamespace);

            // Skip properties that we don't care about due to configuration (default is to allow all properties)
            if (ExcludeSystemProperties && propNamespace == SYSTEM_NAMESPACE) continue;
            if (!String.IsNullOrWhiteSpace(CustomPropertyNamespace)
                    && propNamespace != CustomPropertyNamespace) continue;

            // Create Property element
            writer.WriteStartElement(PROPERTY_NODE_NAME);

            // Create attributes on Property element
            writer.WriteStartAttribute(NAMESPACE_ATTRIBUTE);
            writer.WriteString(propNamespace);
            writer.WriteEndAttribute();

            writer.WriteStartAttribute(NAME_ATTRIBUTE);
            writer.WriteString(propName);
            writer.WriteEndAttribute();

            // Write value inside property element
            writer.WriteString(Convert.ToString(propValue));
            writer.WriteEndElement();
        }

        // Finish out the message
        writer.WriteEndElement();
        writer.WriteEndDocument();
        writer.Flush();

    }

    dataStream.Seek(0, SeekOrigin.Begin);
    var outmsg = Utility.CloneMessage(inmsg, pc);
    outmsg.BodyPart.Data = dataStream;

    return outmsg;
} 

The output will end up conforming to this schema:

propertymessageschema

And will look a little something like this:

<?xml version="1.0" encoding="utf-8"?>
    <ns0:PropertyMessage xmlns:ns0="http://schemas.quicklearn.com/PropertyMessage/2013/06/">
        <Property Namespace="http://schemas.example.org/" Name="CustomProperty1">Value1</Property>
        <Property Namespace="http://schemas.example.org/" Name="CustomProperty2">Value2</Property>
    </ns0:PropertyMessage>

When included in a pipeline alongside the XML disassembler component, we are finally able to take those pesky “empty” messages from Azure Service Bus and process them as XML documents. The message context properties are still retained in the context for routing, but our custom properties are also made available for use directly in maps (through the liberal use of Value Mapping (Flattening) and Logical functoids), and further processing.

To follow this through to the end, I created a quick proof-of-concept map:

samplemap

Essentially we’re looping through the properties from the context (hence using the Value mapping flattening functoid) and only mapping the value stored inside the Property node whenever the namespace and name of the property match those that we care about. Since the namespace for all of our custom properties is the same, we have a single Equals functoid that we’re sharing across all (two) properties.

After processing a message all the way through the bus, custom pipeline, and map, this should result in this beauty as opposed to a nasty error message:

sampleoutput

un-Borked brokered messages

Remember, that none of this stuff is necessary if you’re working with Brokered Messages that have a message body. You only need to go this deep when you’re dealing with Brokered Messages made purely of properties (which is a recommended practice when possible).

By bringing out-of-the-box first class support for Windows Azure Service Bus Queues, BizTalk Server 2013 continues to prove that with a solid and extensible architecture in place, any type of integration can be made possible.

If you would like to learn more about extending BizTalk Server 2013 to meet your integration challenges, check out one of our upcoming BizTalk 2013 Developer Deep Dive classes.

If you would like to access sample code for this blog post, you can find it on github.