Friday, February 18, 2011

Generating Flat Files With BizTalk 2009

Again, I was asked to produce a simple comma delimited file with BizTalk.  Easy, right? Well, it is once you know what you're doing, but for those of us (myself included) who rarely are asked to produce these flat files, it can be a bit challenging.  For one, we're used to XML formatting and generally writing to a database or simply producing an XML file.

So I can't stress enough how convenient it is leverage the Flat File Schema wizard.  Only catch is, you need a sample file to get the wizard rolling along. Just create a quick sample file with one row of data and the columns you need, and don't forget to add a carriage return after the last column.

What you get next is another window asking for a little clarification.  Little tricky here.  Ultimately, what you would like is a csv file that applies a carriage return after each row of data.  So select the entire row including the carriage return. Following that, make sure you are defining the record by a delimiter.  A screen will appear asking such, so populate the "child delimiter" field with a carriage return/line feed delimiter.

Moving right along. The next screen is very important. Here, you're defining the name of that root element (not that important) and the element type (important).  Select the Record element type, click next.

You'll go through the same screens for the next set of elements. Only difference here is that you won't be selecting the carriage return when asked to "Select Document Data," and your delimiter will be something else (usually a comma for comma delimited files). And of course your "Child Elements" will be defined as "Field element."  One other thing to note, ensure that your root element has the "child order" property set to "postfix."

From here your only other requirement is to define a send pipeline.  Why you ask? Because the send pipelines format your data according to the specifications defined within your schema.  But not to worry, all you have to know is that in your pipeline file, you simply need to drag a "Flat file assembler" on to the Assemble shape.  Once there, define the "Document schema" to that of what was defined by the Flat File Schema wizard.
Voila! You're ready to generate csv files....Unless of course you wanted to include column headers.  I can't stress enough how difficult it was to find resources that explained how these header schemas work.  Oh sure, define another schema with your column names and your done, right?  Not for all of us.  What I didn't understand was that even after I defined a header schema, 2 things were occurring; empty string values for column names and no carriage return.  
So I set off to find what was wrong with my definition.  What I found was that, I needed to define either the "Fixed" or "Default Value"  property for each element field.   But before I get into that, let me note that defining a header schema is as simple as taking a copy of your document schema and manipulating the values for Fixed or Default and ensuring that a PostFix value is defined for the child order of the root record element.  With that, your header values will print and your carriage return will push the first row of data below the column headers.  Now you're done. Well, almost.  Go back to your Send Pipeline and define the Header Schema property with the newly created Header Schema and now you can deploy and enjoy your BizTalk application.

Tuesday, February 15, 2011

BizTalk Orchestrations and their respective ports

It seems that every time I have a new BizTalk project land on my desk, my brain turns to mush.  My most recent project is a case in point.

So I was asked to create a simple project that would take data from two disparate systems and merge/clean the data, before finally outputting said data to a simple XML file.  "No problem!" or so I thought.  Recalling my BizTalk acronym SOME (Schemas - Orchestrations - Maps - ESB Itineraries), I was able to quickly produce a project that would perform the task required.  Everything compiled ok.  I was able to deploy successfully (or so I thought).

I only incurred one minor issue, I'd forgotten how to configure my send ports, leveraging "Specify later" binding, to produce an xml file to a directory.  "This can't be" I thought, "I've created several other send ports similar to what I need now."  Thus, panic set in once again.  So I did what any other would do in this event...I reviewed my past to uncover the secret behind properly configuring my send port.  Only problem was that the answer was not as blatantly obvious as I would have preferred. 

"Oh my god! How do you configure a simple send port?"  Well folks, the answer is rather trivial, but annoying nonetheless.  Within your orchestration, you will recall having to define a Port Type.  Some important properties to configure within this type is the "Type Modifier."  Type Modifier should be set to public so that it becomes viewable via Administration Console.  Next you should take note of the Port identifier property for the Port, not to be confused with the Port Type.  This property will prove useful when configuring your send port via Administration Console. 

So you've set your "Type Modifier" to public and you've successfully deployed your solution.  Great!  Now all you need is to configure a send port on the Administration Console.  So select the Application, and create a new send port of Transport Type "File."  Beautiful!  Now BizTalk should produce files for me to the directory specified....right?  Not quite, and here's where I panicked a bit.  At this point, you've declared a send port at the BizTalk level, and you've declared a send port at the project level.  So how do you tell BizTalk to pair the two?  Well, it just so happens that when you right-click the Application in Administration Console, and select "Configure," you are presented with a window displaying the artifacts available within the app.  You'll find both the orchestrations, and send/receive ports created.  Your final step here is to select the orchestration that produces the send port and pair it with the send port you configured via Administration Console.

Voila! 

Friday, January 14, 2011

First Annoyance Solved For Sharepoint 2010

So I was asked to begin the process of converting our Sharepoint 2007 apps to 2010 solutions.  My first thought was, "Oh Great! I forgot to study."   Once I got over the initial shock, I began tackling the assignment with clueless abandon.  I must have attempted to convert 7 to 10 different solutions with the out of the box Sharepoint conversion wizard.  Attempted, but never realized.

Needless to say, it didn't go well.  With little to no help via the web, I scrapped the idea and was momentarily defeated.  But it wouldn't be my style to simply give up.  So off I went, on an adventure to the center of the Sharepoint universe (or at least the edge of my seat).

Long story short, I created an empty sharepoint project with Visual Studio 2010, selected Sandboxed solution and added all my solution files to the project.  All well and good, my project compiled and I was able to deploy to Sharepoint...except.  I couldn't add my webpart.  I'll spare you with the troubleshooting effort and ambiguous exception messages.  To get around my issue, I discovered a few articles stating that certain solutions need to be Farmed, not Sandboxed.

Easy enough I thought, so I began looking through every VS menu item and right-clicking every conceivable object in site to convert my solution to Farm.  No luck, there was a suggestion to open the project file and hand enter the xml node, but I ran into trouble here, plus I was confident that MS had an easy method for converting these silly Sharepoint projects, I just had to dig.  Well, wouldn't you know it, I found the solution.  Just view the project properties by selecting it in Solution Explorer and setting the "Sandboxed Solution" property to False.  So there you have it.  Simple solutions that could leave you feeling silly once you find them or worse yet, feeling like an idiot when your non-technical wife points it out to you.

Wednesday, October 27, 2010

BizTalk for beginners

It's been some time since my last post, for no other reason than I've gone from SharePoint development to BizTalk related activities in recent months.  In any case, let's get to it.

For this first BizTalk related post, I just want to create a beginner's level guide to developing BizTalk projects.  So what are the terms floating around BizTalk; Orchestrations, Itineraries, Maps, Schemas, Send/Receive Ports, Messages, etc.  Now, I can go over the terms for everyone, but it's not necessary to review what's been published hundreds of times already. My focus is to help developers adjust their current mindset to become open to the idea of project compartmentalization.  I have found that it is easiest to consider every aspect of a BizTalk project as an object.  And as a rule of thumb, work from the ground up, meaning that the development effort should begin with the most basic/independent "objects" first.

From a developers standpoint, it's important to become familiar with Schemas first and foremost.  Why? For no other reason than it is how your message content will be defined for specific operations and they are the absolutely most basic aspect of the BizTalk project.  Schemas are defined in Xml format.  Generally, the top elements define the message operations, such as select or delete operations, and their children define the data processed by these operations.  So it is possible to have a field defined several times within one schema.

A big problem I first ran into during development was that I had no idea how to structure my incoming message for unit testing.  Luckily, (and I found this out after many trials) Visual Studio has the ability to generate an instance of a message based on a supporting schema (Just right-click the schema and select Generate Instance).  This comes in handy when running tests againsts maps....

So how do we transfer information from one Message/Schema to another? Maps (There are other methods, but this is just a beginner's guide).  When you create a new mapping file, you are asked for a source and destination schema.  This is where the Schemas you generated come in handy.  Define a source and destination and leverage the mapper as much as possible to map fields from one schema to the other.  There may be fields that require some massaging before transmitting to their respective destination schema fields, and this is where it becomes important to familiarize yourself with functoids.

Functoids are the tools leveraged within a map to translate data.  They come in many forms; String functoids, Mathematical functoids, Date/Time functiods, etc.  Drop these on your map and define their input/output fields, as well as any other information required of them (such as actual C# code for instance).

In review, create your schemas first, then create your Maps.  Leverage Visual Studio to generate an instance of your source message/schema so you can test your maps more precisely.  Don't forget the functoids to translate your data for the destination system.

Ok. Ports.  You have send and receive ports.  As you may have guessed, ports define what data is transmitted into or out of the BizTalk environment.  But it's not enough to discuss ports alone, because ports have several working parts; Port Types, Operations, Communication Patterns, Message Types, Message Directions, Binding.   Where do we start? Start with Port Types.

To discuss Port Types we should first discuss Communication Patterns.  Communication Patterns define the direction in which the data is to be transmitted by the port.  There are two options; One way or Request-Response.  A one way communication pattern is best defined as a pattern that does not require a formal handshake, whereas a Request-Response pattern does.  To best illustrate this, think of the task of running a select query against a database.  The handshake in this instance occurs when the database receives the select query and returns a resultset back.  Conversely, the one way pattern is illustrated as a push event.

Operations are simply actions that define what activity can occur over a port.  So let's say you have a port that connects a SQL database to BizTalk, the operations can be select, insert, and update operations.

Message Directions define the how the message content is used. It can be either as a request or a response, much like you have requests and responses over http.

Message Types are simply the operations defined within the schemas.  As mentioned earlier, schemas are defined in Xml format.  Their top level elements are generally used to define specific operations and their children define the data processed by these operations.

So Ports need a Port Type assigned to them.  Once a port type is selected for a port, the communication direction is automatically selected for it.  For one way ports, this is fine, but on Request-Response ports, you must be cautious that the comm direction is the one you intended, otherwise you may have to manually select the comm direction.  To help you decide, you must determine whether the port will send data and expect to receive data in return, or vice-versa. Bindings on the other hand can be defined either through Visual Studio or the BizTalk Administration console.  In either case, the bindings are basically the system defined connection protocol.

With all of these defined, the Ports then only allow messages that match the message type defined within their Port type. Not much left to be said about that, so on to the next item...

Messages are simple as well. Consider them the data packages defined by an operation instance within a schema.  Thus, a Message Type must be selected for each message.  Again, not much left to say about messages.

Next up are orchestrations.  With Schemas, Maps, Messages and Ports defined, the orchestration binds these objects all together.  Now within the Orchestrations, you setup your Messages, Ports, Port Types, etc. Basically, everything can be setup from within an orchestration file, but you're better off creating your Maps and Schemas first and adding references to them from within the orchestration file.  All other objects are fair game to create within the Orchestration.

Now, tips.  I've found that it is best to have two separate orchestrations.  One to act as a container for your Port Types, and the other to handle the rest.  Why, because you may have several orchestrations within a project file, so it becomes easier to maintain your port types from one source file and reference those types from the other orchestration files.  Also, I've found that my orchestrations generally leverage similar port types when contained within a project.

Well,  that's it for now.  Hopefully, I'll pick up a few more tips before Microsoft buries BizTalk, in which case, I may have to turn to AppFabric.

Friday, December 18, 2009

FullTextSqlQuery People search annoyance resolved

Ok, here's an easy one. A lot of us leverage the FullTextSqlQuery class to perform our People searches. Well, one annoyance I find is revealing the proper syntax to return the property values. There's generally no good references since People search varies from farm to farm. So what do you do? Well, you can look through all the Managed Properties in SSP to find them, or you could implement a method that could return the properties you need based on Display name. I prefer the latter since it takes the grunt work out of sifting through hundreds of records.

On to the solution. So by leveraging the UserProfileManager, you can get an instance of the PropertyCollection class and voila! A quick loop through the collection will get you the property you need. Here goes...


static string GetProfileMatchingProperty(string propertyDisplayName)
{
UserProfileManager oSPManager = new UserProfileManager(ServerContext.Default, true);

Microsoft.Office.Server.UserProfiles.PropertyCollection profileProperties =
oSPManager.Properties;

foreach (Property profProperty in profileProperties)
{
if (profProperty.DisplayName.ToLower(System.Globalization.CultureInfo.InvariantCulture) ==
propertyDisplayName.ToLower(System.Globalization.CultureInfo.InvariantCulture))
return profProperty.ManagedPropertyName;
}

throw new Exception("Couldn't find it. Sorry.");
}

Friday, December 11, 2009

Working with SPFieldChoice type

This may come as a no-brainer for some, but I thought I'd share it in any case. So, there may be times in a developers life that they need to write some functionality that fiddles around with SPList objects. Within those SPList objects, you may have a field derived from SPFieldChoice type. I'm not going to make this a note about how to derive the current value since that is ultimately a trivial task, but what I will do is talk about defaulting values programmatically (again, probably something trivial for most, but then again, maybe something you don't think about).

Ok, so let's say that you wrote a webpart that has to apply default values to an SPListItem. And within that SPListItem, say you have an SPFieldChoice type. Of course, within SharePoint, you can declare you're own default values, but say you want to buck the trend and set one of the other choices as a defaulted value.

Now you have a few decisions to make: hard code the value (bad design), always take field default value, or loop through choices. I'll show you how to get the latter 2 choices (I'm not about to champion the hardcoding movement).

To get the default value for SPFieldChoice:
  • SPListItem.Fields[Guid, Index, or Name of Column].DefaultValue
To loop through the SPFieldChoice choices:

foreach (string oneChoice in ((SPFieldChoice)SPListItem.Fields[Guid, Index, or Name of Column].Choices)
{
switch (oneChoice)
{
case value1:
{some code}
break;
case value2:
{some code}
break;
default:
{some code}
break;
}
}

Friday, December 4, 2009

MOSS Page Timeout Issues

So I was asked to write a simple webpart that would tally survey responses. Everything was moving along just fine and the code was behaving beautifully. Wouldn't you know it, once the code reached User Acceptance, issues started creeping along. I won't get into all the issues since most were minor. The one biggie was a timeout issue that appears as a nasty little default error message that reads "An unexpected error has occurred."

This was mind-boggling for me at first. How could my code fail after it passed unit testing and QA? Digging a little further, I copied masses of data over to my dev box and started debugging my code. Again, no problem while I was stepping through the code. But a funny thing happened when I switched my web.config settings on my dev machine from debug="true" to debug="false." The page threw this error at me. Finally, I was making head-way!

So I started stepping through the code once again, but nothing happened! How could this be? I thought my box was off debug mode? Turns out, when stepping through code, the box interprets those actions as having set debug="true" in web.config.

It finally dawned on me that the code was timing out. So I shut off debugger and added a new timeout setting to web.config. Presto! The code ran fine, but I wasn't satisfied. I wanted to apply my timeout setting to run only against that page. So I setup the web.config under my Layouts folder and wouldn't you know it, the issue appeared again. Now this was ridiculous, I should have the ability to set a path in MOSS to increase my timeout settings.

Turns out, I had to apply my setting in web.config at the root level. So under the home directory found in IIS (for most it would be c:\Inetpub\wwwroot), I updated web.config with the following under the configuration node and presto, my timeout setting applied only to my particular page url:

<location path="SomePath/MyPage.aspx">
<system.web>
<httpRuntime executionTimeout="3600" />
</system.web>
</location>