Workaround for the "The part 'part' of message" BizTalk annoyance

Working with BizTalk can be a frustrating experience for many developers. At least, that's the experience I've had with the tool. In any case, my latest annoyance was the dreaded "The part 'part' of message '{Message}' contained a null value at the end of the construct block."

Scouring the web for solutions turned up virtually nothing. And so it began, my journey for a workable solution to this enigma that is BizTalk (again). My first attempts were a simple exercise of attrition. It encompassed recompiling, re-referencing, de-referencing and every other ridiculously simple attempt to resolve the problem. None produced the desired outcome; the message persisted and my patience grew increasing limited.

Thus, my next series of attempted fixes resulted in reassigning the message variable. I'll spare you from the multitude of failures incurred before succumbing to the eventual fix.

The resolution consisted of the following:
1. Declare the outgoing message of type System.Xml.XmlDocument
2. Create an xPath string that will populate said outgoing message with the entire contents of the message to be leveraged.
3. In a message construct/message assignment shape, assign the outgoing message by calling an xpath solution.

Here's a sample of the solution that worked for me:

xPathString = "/*[local-name()='Employees' and namespace-uri()='http://mySpecialSchema']";
MsgOutput = xpath(MsgInput,xPathString);

Leverage the Looping Shape in an Orchestration

So I was tasked with generating a new BizTalk project that would update an email field in a SQL database. Great I thought, a simple orchestration with straight forward mapping ought to do the trick. And there I went, pointing and clicking away until I was thrown a curve by management. I was asked to leverage a stored procedure to update the field. Ordinarily, this isn't a problem, but we leverage a canonical schema from which all people records are derived. To update the stored procedure, I needed to devise a way of updating each record contained within in the canonical, one by one. What I was able to produce was an orchestration that enacted the Looping shape.

The way our canonical is formed, it contains an instance of an employee inside an employees node. The task was to extract each employee node, one by one, and push them out to the stored procedure via a send shape. Unfortunately, my stored procedure schema could only accommodate one record at a time, hence the loop shape.
First item of business was to determine how many records my canonical contained. Well, I needed a variable to hold this value, so I created one called recordCount. Now populating the recordCount required an xpath query to return the value. Now, xpath in BizTalk, can be a little cumbersome, but possible none the less. Taking my canonical schema I created this expression in a standard expression shape:

Now, the fun part. So I created a new Loop shape and entered

recordCount > 0

The next step entailed creating the single Employee message. To do this, you need a Construct Message shape. Within the Construct Message shape, you need to apply an Expression shape and Message Assignment shape. Obviously, the Message Assignment shape will handle the mapping of data from one message to another, but the Expression does the real work here. For this exercise, I created two temporary variables of type System.Xml.XmlDocument. Within my Expression shape, I was able to populate the temp variables with one record from my grand Employees canonical schema as such:

And of course, my Message assignment shape took the single Employee message and created a new message for the send port of the Update Email stored procedure.
Looping within an orchestration can be a clunky process, so I'd recommend you leave it as a last resort. But never the less, it beats writing hundreds of lines of c# code.

Happy Coding!

How to generate a BizTalk request message

What appears to be an issue for many BizTalk developers or newbies, is how to initiate a BizTalk request message.  Most perplexing about this is that BizTalk is not an intuitive interface, which leaves many in the dark as to how or what exactly needs to be done in order to perform a simple select query against a database.  Well, I've wrestled with this issue many times before, and what I've found is that generating your request messages is best served in code.  This is the only means, I find, that permits flexibility even after promoting your application to a production environment.

At the base is the simple question of where does one start?  Well, that's simple enough, just go to your map in Visual Studio, and right-click to perform a "Test map" operation.  If you don't have a map, create a temporary map and make sure you apply the "Generate Instance" setting in the TestMap Input field.  In the Output window, you'll find a link to the sample input file leveraged to generate the test.  Open this sample and voila! You have the xml needed to help generate your request message.

With the sample xml available, you can then create a class that performs the generate request message code.  Modify the xml values such that you can modify them from outside of BizTalk (perhaps in a config file).  Below follows sample code outlining message creation process.

public static XmlDocument CreateMsg(string msgType)
{
     XmlDocument xmlDoc = new XmlDocument();
       StringBuilder strBuilder = new StringBuilder();
            string columnString = string.Empty; \\ ex "column1 as [Sales], column2 as [Pct]
            string filterString = string.Empty; \\ ex "WHERE DATEDIFF(Day, LAST_DATE_WORKED, getdate()) < 8"


            strBuilder.Append("");

            switch (msgType.ToUpper().Trim())
            {
                case "A Specific Message":
                    columnString = ConfigurationManager.AppSettings["myRunTimeColumns"];
                    filterString = ConfigurationManager.AppSettings["myRunTimeWhereClause"];

                    strBuilder.Append(string.Format("{0}",
    columnString));

                    strBuilder.Append(string.Format("{0}",
    filterString));

                    strBuilder.Append("");
                    break;
                default:
                    strBuilder.Append("FirstName, LastName, OfficeNumber, case Code when '850' then Dept else Region end as LocationCode, Email");
                    strBuilder.Append("");
                    break;
            }

            strBuilder.Append("");


            xmlDoc.LoadXml(strBuilder.ToString());

            return xmlDoc;
}

And of course, you would have to declare a message object in your orchestration and initialize it from within the orchestration as such:

MyRequestMsg = HelperNameSpace.MessageConstructorClass.CreateMsg("A Specific Message");

For this example, I populated the BizTalk service config file with new keys under the appSettings node (myRunTimeColumns and myRunTimeWhereClause) to hold the column and where clause values respectively.

OOB BizTalk FTP Adapter Alternative

We tend to get naive when working with BizTalk and its many adapters.  I, for one, admit to operating under such soundless circumstances.  My latest issue has been with the out of the box FTP Adapter Microsoft so graciously included.  My objective was to generate an application that would connect to two separate FTP servers and apply the same file to both.  It just so happens that one of the vendors in this instance, was uncooperative or unable to supply a test account. So, I decided to test my process with the more cooperative of the two vendors and configured my send ports to push both files towards that vendor.

All worked great, and I ran and reran my process dozens of times over.  I passed development, alpha, beta and UAT testing until finally pushing my application to production.  That's when the wheels fell off the wagon.  After I reconfigured my send port to send my files to both vendors, within an hour of post-deployment testing, I soon discovered that the FTP adapter was failing for one of my send ports.  Upon review of the event logs, I noticed that my uncooperative vendor was refusing connection after the initial FTP call.

Digging a little deeper, I turned on the FTP logs made available by the FTP Adapter and found that the Adapter never sends the QUIT command to sever the connection until the host instance was shut down or restarted.  This was insane, I thought.  I tried a number of solutions such as applying an "After Put" command in the FTP Adapter. Sadly these solutions did not work effectively without throwing exceptions and cluttering up BizTalk with failed orchestrations.

So, I eventually gave in and decided to leverage a dynamic port and incorporate C# into the project.  The code declared a new class that would perform folder monitoring based on a value specified from the BizTalk configuration file.  My monitor would listen to the folder and wait for a new file or file modification to occur like such:

public void initiateFolderMonitor()
{
string fullFilePath = ConfigurationManager.AppSettings["FolderPath"];

folderMonitor.Path = fullFilePath;
UploadFilePath = fullFilePath;
UploadFile = ConfigurationManager.AppSettings["FileName"];
FTPAddress = ConfigurationManager.AppSettings["FTPAddress"];
FTPUser = ConfigurationManager.AppSettings["FTPUser"];
FTPPassword = ConfigurationManager.AppSettings["FTPPassword"];
FailureCount = 0;

folderMonitor.NotifyFilter = System.IO.NotifyFilters.FileName;

folderMonitor.NotifyFilter = folderMonitor.NotifyFilter | System.IO.NotifyFilters.Attributes;

folderMonitor.Created += new FileSystemEventHandler(monitoringEventRaised);
folderMonitor.Changed += new FileSystemEventHandler(monitoringEventRaised);


 try
 {
   folderMonitor.EnableRaisingEvents = true;
 }
 catch (ArgumentException)
 {
   stopActivityMonitoring();
 }
}

public void stopActivityMonitoring()
{
 if (FailureCount == 0)
   retryFileUpload();

 if (folderMonitor.EnableRaisingEvents)
   folderMonitor.EnableRaisingEvents = false;

 folderMonitor.Dispose();
 folderMonitor = null;
}

private void monitoringEventRaised(object sender, System.IO.FileSystemEventArgs e)
{
 switch (e.ChangeType)
 {
   case WatcherChangeTypes.Changed:
   case WatcherChangeTypes.Created:
      if (e.Name == UploadFile)
      {
         FailureCount += 1;

         if (FtpUloadFile())
         {
            stopActivityMonitoring();
            removeFile();
         }
         else
         {
            retryFileUpload();
         }
      }
      break;
   default:
      break;
 }
}

private void retryFileUpload()
{
  Thread.Sleep(10000);

  if (FailureCount > 3)
  {
     stopActivityMonitoring();
     return;
  } 
  else if (FtpUloadFile())
  {
     stopActivityMonitoring();
     removeFile();
     return;
  }

  FailureCount += 1;

  retryFileUpload();
}

And naturally, I have code to perform the actual FTP upload like such:

private bool FtpUloadFile()
{
   bool ftpReturn = false;

   try
   {
     System.Net.FtpWebRequest ftpRequest = (System.Net.FtpWebRequest)System.Net.WebRequest.Create(string.Format("ftp://{0}/{1}",
       FTPAddress, 
       UploadFile));
     ftpRequest.Method = System.Net.WebRequestMethods.Ftp.UploadFile;
     ftpRequest.Credentials = new System.Net.NetworkCredential(FTPUser, FTPPassword);

     System.IO.StreamReader streamFR = new System.IO.StreamReader(string.Format(@"{0}\{1}", 
       UploadFilePath, 
       UploadFile));
     byte[] filecontents = Encoding.UTF8.GetBytes(streamFR.ReadToEnd());
     streamFR.Close();
     ftpRequest.ContentLength = filecontents.Length;

     System.IO.Stream requestStr = ftpRequest.GetRequestStream();
     requestStr.Write(filecontents, 0, filecontents.Length);
     requestStr.Close();

     System.Net.FtpWebResponse resp = (System.Net.FtpWebResponse)ftpRequest.GetResponse();

     switch (resp.StatusCode)
     {
       case System.Net.FtpStatusCode.ClosingData:
          ftpReturn = true;
          break;
       default:
          ftpReturn = false;
          break;
     }

     resp.Close();
   }
   catch (Exception) { }

   return ftpReturn;
}

So, the above code gets initiated from my orchestration before the send shape. You have to declare a variable to hold the new class. Then, within an expression shape, you set the Folder monitoring in motion like so:

//Initiate Orchestration variable
monitorObj = new my.Helpers.MonitoringClass();
monitorObj.initiateFolderMonitor();

//Initiate the dynamic Send port
DynamicSendFilePort(Microsoft.XLANGs.BaseTypes.Address) = @"file://" + ConfigurationManager.AppSettings["FolderPath"] + @"\" + ConfigurationManager.AppSettings["FileName"];

Finally, in the orchestration, you can either call the disable monitoring method, or include code in your class to timeout the monitoring piece to release those system resources.

How to clone a SharePoint 2010 Hyper-V VM

If anyone has ever used Microsoft's Hyper-V Server, you've discovered that the interface is a little difficult.  For one, you're presented with no GUI, which can be intimidating to some.  In any case, I was able to spend a little time to uncover a few steps that could help towards cloning a standalone SharePoint instance hosted on VM.  Let's begin...

Your first step involves logging into the Hyper-V base.  On the base, what we want to accomplish is to create a copy of the VHD file containing the SharePoint 2010 instance.  For some, you may log into the base and find that there are no visible windows.  This little obstacle can be overcome by calling the Task Manager window; simultaneously press the Control-Shift-Esc keys.  That should help ease the nerves a bit.

With the Task Manager open, click the File->New Task(Run) option from the file menu.  This will popup a "Create New Task" window similar to what you'd find when you recall the run command in a normal windows environment.  At this point, you want to enter the following command “c:\Windows\System32\sconfig.cmd” to bring up the Server Configuration User window.  You'll want to open the "Create New Task" window again to open an instance of the command prompt as well.

Now, you need to know where the VHD file is located that you wish to copy.  If you don't, a quick way of determining the location is by connecting to the base image from another machine with Windows Explorer and looking under the VM configuration file repository normally located under "C:\ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines."  Open the available XML files until you find the one containing a reference to the file you want to copy.  With this, you will have the path to the VHD file.

On the base machine, go to the DOS prompt and navigate to the location of the VHD file.  Perform a DOS command to copy the file to wherever you wish.  Once copied, you can paste the file to the Hyper-V base image of your choice.

With a copy of the VHD file on your target base image, you'll need to log into a server that hosts Hyper-V Manager.  From Hyper-V Manager, navigate to the target base machine and begin the process of creating a new "Virtual Machine."  Assign a name to the new "Virtual Machine," and allocate an adequate amount of memory.  Configure the networking credentials for the VM before finally selecting the location of the VHD file copied over.

With those steps, you will have created a new VM instance that is an exact clone of the prior VM instance.  But we're not done just yet.  Now we want to rename the new VM instance, or suffer through duplicate name conflicts.  However, because we're also working with SharePoint, we need to modify the Alternate Access Mappings in Central Administration before anything else occurs.

Connect to the new VM instance from Hyper-V Manager.  Log in under a local administrator account and open "Central Administration."  When and if you're prompted for credentials, use the local administrator credentials.  Navigate to "Configure Alternate Access Mappings" and configure the URLs, leveraging the prior VM instance name, to point to localhost.  

One other SharePoint centric step required before officially renaming the server is to run an stsadm command.  There may be a powershell command you can run, but I haven't performed enough research to determine what that would be.  So you need to open a command prompt as administrator by explicitly selecting the "run as administrator" option.  In your command prompt, change the directory to "c:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\bin" or wherever the bin folder for 14-hive is located.  Run the command "Stsadm -o renameserver -newservername {YourNewServerName} -oldservername {YourOldServerName}"  to tell SharePoint that a server name change is in effect.

Now, let's bring up Server Manager to change that pesky server name by selecting Start->Administrative Tools->Server Manager.  From the Server Summary section, select "Change System Properties."  Click the "Change" button from the "Computer Name" tab and change the server name in the available text field.  You'll be asked to restart the VM.  Allow the server to restart.

Not quite done, but we're almost there.  Log back into the VM instance and open "Central Administration" again.  Go to the "Manage the farm administrators group" under the Security section.  You'll more than likely see accounts that reflect the server name of the prior VM instance.  You'll need to edit those accounts so that their names display the appropriate server name.

Voila!  These steps should get you through cloning a VM instance with SharePoint 2010.  A few assumptions are made with this tutorial and that is that it is assumed that SQL Server resides on the same VM instance.  Also assumed is that you have knowledge of the basic DOS commands required of this tutorial.  And finally, it is assumed that you have full access to the Hyper-V environment.

Generating Flat Files With BizTalk 2009

Again, I was asked to produce a simple comma delimited file with BizTalk.  Easy, right? Well, it is once you know what you're doing, but for those of us (myself included) who rarely are asked to produce these flat files, it can be a bit challenging.  For one, we're used to XML formatting and generally writing to a database or simply producing an XML file.

So I can't stress enough how convenient it is leverage the Flat File Schema wizard.  Only catch is, you need a sample file to get the wizard rolling along. Just create a quick sample file with one row of data and the columns you need, and don't forget to add a carriage return after the last column.

What you get next is another window asking for a little clarification.  Little tricky here.  Ultimately, what you would like is a csv file that applies a carriage return after each row of data.  So select the entire row including the carriage return. Following that, make sure you are defining the record by a delimiter.  A screen will appear asking such, so populate the "child delimiter" field with a carriage return/line feed delimiter.

Moving right along. The next screen is very important. Here, you're defining the name of that root element (not that important) and the element type (important).  Select the Record element type, click next.

You'll go through the same screens for the next set of elements. Only difference here is that you won't be selecting the carriage return when asked to "Select Document Data," and your delimiter will be something else (usually a comma for comma delimited files). And of course your "Child Elements" will be defined as "Field element."  One other thing to note, ensure that your root element has the "child order" property set to "postfix."

From here your only other requirement is to define a send pipeline.  Why you ask? Because the send pipelines format your data according to the specifications defined within your schema.  But not to worry, all you have to know is that in your pipeline file, you simply need to drag a "Flat file assembler" on to the Assemble shape.  Once there, define the "Document schema" to that of what was defined by the Flat File Schema wizard.

Voila! You're ready to generate csv files....Unless of course you wanted to include column headers.  I can't stress enough how difficult it was to find resources that explained how these header schemas work.  Oh sure, define another schema with your column names and your done, right?  Not for all of us.  What I didn't understand was that even after I defined a header schema, 2 things were occurring; empty string values for column names and no carriage return.  

So I set off to find what was wrong with my definition.  What I found was that, I needed to define either the "Fixed" or "Default Value"  property for each element field.   But before I get into that, let me note that defining a header schema is as simple as taking a copy of your document schema and manipulating the values for Fixed or Default and ensuring that a PostFix value is defined for the child order of the root record element.  With that, your header values will print and your carriage return will push the first row of data below the column headers.  Now you're done. Well, almost.  Go back to your Send Pipeline and define the Header Schema property with the newly created Header Schema and now you can deploy and enjoy your BizTalk application.

BizTalk Orchestrations and their respective ports

It seems that every time I have a new BizTalk project land on my desk, my brain turns to mush.  My most recent project is a case in point.

So I was asked to create a simple project that would take data from two disparate systems and merge/clean the data, before finally outputting said data to a simple XML file.  "No problem!" or so I thought.  Recalling my BizTalk acronym SOME (Schemas - Orchestrations - Maps - ESB Itineraries), I was able to quickly produce a project that would perform the task required.  Everything compiled ok.  I was able to deploy successfully (or so I thought).

I only incurred one minor issue, I'd forgotten how to configure my send ports, leveraging "Specify later" binding, to produce an xml file to a directory.  "This can't be" I thought, "I've created several other send ports similar to what I need now."  Thus, panic set in once again.  So I did what any other would do in this event...I reviewed my past to uncover the secret behind properly configuring my send port.  Only problem was that the answer was not as blatantly obvious as I would have preferred. 

"Oh my god! How do you configure a simple send port?"  Well folks, the answer is rather trivial, but annoying nonetheless.  Within your orchestration, you will recall having to define a Port Type.  Some important properties to configure within this type is the "Type Modifier."  Type Modifier should be set to public so that it becomes viewable via Administration Console.  Next you should take note of the Port identifier property for the Port, not to be confused with the Port Type.  This property will prove useful when configuring your send port via Administration Console. 

So you've set your "Type Modifier" to public and you've successfully deployed your solution.  Great!  Now all you need is to configure a send port on the Administration Console.  So select the Application, and create a new send port of Transport Type "File."  Beautiful!  Now BizTalk should produce files for me to the directory specified....right?  Not quite, and here's where I panicked a bit.  At this point, you've declared a send port at the BizTalk level, and you've declared a send port at the project level.  So how do you tell BizTalk to pair the two?  Well, it just so happens that when you right-click the Application in Administration Console, and select "Configure," you are presented with a window displaying the artifacts available within the app.  You'll find both the orchestrations, and send/receive ports created.  Your final step here is to select the orchestration that produces the send port and pair it with the send port you configured via Administration Console.

Voila! 

First Annoyance Solved For Sharepoint 2010

So I was asked to begin the process of converting our Sharepoint 2007 apps to 2010 solutions.  My first thought was, "Oh Great! I forgot to study."   Once I got over the initial shock, I began tackling the assignment with clueless abandon.  I must have attempted to convert 7 to 10 different solutions with the out of the box Sharepoint conversion wizard.  Attempted, but never realized.

Needless to say, it didn't go well.  With little to no help via the web, I scrapped the idea and was momentarily defeated.  But it wouldn't be my style to simply give up.  So off I went, on an adventure to the center of the Sharepoint universe (or at least the edge of my seat).

Long story short, I created an empty sharepoint project with Visual Studio 2010, selected Sandboxed solution and added all my solution files to the project.  All well and good, my project compiled and I was able to deploy to Sharepoint...except.  I couldn't add my webpart.  I'll spare you with the troubleshooting effort and ambiguous exception messages.  To get around my issue, I discovered a few articles stating that certain solutions need to be Farmed, not Sandboxed.

Easy enough I thought, so I began looking through every VS menu item and right-clicking every conceivable object in site to convert my solution to Farm.  No luck, there was a suggestion to open the project file and hand enter the xml node, but I ran into trouble here, plus I was confident that MS had an easy method for converting these silly Sharepoint projects, I just had to dig.  Well, wouldn't you know it, I found the solution.  Just view the project properties by selecting it in Solution Explorer and setting the "Sandboxed Solution" property to False.  So there you have it.  Simple solutions that could leave you feeling silly once you find them or worse yet, feeling like an idiot when your non-technical wife points it out to you.