Coding, content and startups

Being new to the world of NServiceBus, I just thought I’d share a few gotcha’s as I experience them.

When everything’s up and running there’s no easy way to see what’s going on as messages appear and disappear from the normal message queue very quickly. You can use an audit queue to log all messages appearing on a queue. To do this, in your app config you simply need to use the ForwardReceivedMessagesTo attribute, like so:

<UnicastBusConfig ForwardReceivedMessagesTo="MyAuditQueue@MachineName">
....
</UnicastBusConfig>

NServiceBus won’t automatically create an audit queue, so when you do so manually.

You can do this in code using:

NServiceBus.Utils.MsmqUtilities.CreateQueueIfNecessary(QueueName)

Alternatively, you can create it using the admin interface, but you need to ensure it has the same settings and permissions as the NServiceBus queues. Notably, that SYSTEM has permissions on the queue, and that it is transactional (if your queue is) – otherwise your audit queue will remain empty!

Running MsDeploy is awesome for automated deployments of websites, but it’s also possible to use it to deploy other applications to the file system – such as associated windows services. You just need to jump through a few more hoops to get things up and running.

I’m using TeamCity for our integration server, but the basic steps will work regardless of the system you are using. I tend to set up TeamCity to have a general “Build entire solution” configuration. This builds the entire project in release mode, and performs any config transformations you need (check out my post here if you to transform app.config files for your service).

Next, for each component and configuration we want to deploy (ie website to staging, website to production, services to staging, services to production), I create a new build configuration, with a dependency on the “build entire solution” configuration. This means we can assume that the build has completed successfully.

After the build, there’s a few steps that need to complete:

• Stop the existing service and uninstall it
• Copy over the output from the build to the target deployment server
• Install the new service and start it

Stopping and starting the services

For the first and last steps, we can define two simple batch files for each, with a hard coded path of where we’ll install the service on the target server.

MyServiceName.PreSync.cmd

net stop MyServiceName
C:\Windows\Microsoft.NET\Framework\v4.0.30319\InstallUtil.exe /u /name=MyServiceName “C:\Program Files\PathTo\MyServiceName.exe”
sleep 20

MyServiceName.PostSync.cmd

C:\Windows\Microsoft.NET\Framework\v4.0.30319\InstallUtil.exe /name=MyServiceName "C:\Program Files\PathTo\MyServiceName.exe"
net start MyServiceName

These should be saved in source control as part of your project resources (I put them in a Deploy folder), and so accessible from the build server. These are very basic at the moment – they could equally be PowerShell scripts doing far more complicated things or accepting configurable parameters – but this will do us for our example scenario!

We will use MsDeploy’s preSync and postSync commands to execute these batch files before and after it performs the synchronization on the file system.

MsDeploy command

Let’s now take a look at the MsDeploy command needed:

"tools/deploy/msdeploy.exe" -verb:sync -preSync:runCommand="%system.teamcity.build.checkoutDir%\tools\deploy\MyServiceName.PreSync.cmd",waitInterval=30000 -source:dirPath="%system.teamcity.build.checkoutDir%\src\MyServiceName\bin\%env.Configuration%" -dest:computerName=https://stagingserver:8172/msdeploy.axd?site=DummyWebSiteName,userName=%env.UserName%,password=%env.Password%,authType=basic,dirPath="C:\Program Files\MyWindowsService\" -allowUntrusted -postSync:runCommand="%system.teamcity.build.checkoutDir%\tools\Deploy\CodeConversion-PostSyncCommand.cmd",waitInterval=30000

Let’s just break this down:

• verb:sync – we are syncing!
• preSync:runCommand – before we perform the deployment, we can pass the path to a batch file that will be streamed to the deployment server and executed. By default, this will be run under a restricted local service account (“The WMSvc uses a Local Service SID account that has fewer privileges than the Local Service account itself.” – from MSDN).
• source:dirPath – this sets the path we want to copy files from. We’re using a parametrized build template in TeamCity to pass in the full path to the source directory, and the current configuration)
• dest:computerName – this is actually several parameters combined. I tried various permutations, and this is what worked best for me. I’m not using NTLM authentication here (so authType=basic) because my staging and production servers are on an external network. The username and password are for an IIS Management Service user that we’ll set up in a minute (and are also parametrized by TeamCity – but you could hard code them here).
• allowUntrusted – allows MsDeploy to accept the unsigned certificate from our target server. You don’t need this if you’re using an SSL certificate from a trusted authority.
• postSync:runCommand – the command we run after a successful deployment.

There’s one gotcha with the preSync and postSync operations at the moment – any error codes returned by preSync or postSync (such as being unable to install the service or start it), the whole MsDeploy action still return success. I haven’t found a nice way round this yet – you’d have to write some powershell script to parse the output and detect errors. Microsoft know about the issue so hopefully it will be fixed in the next release.

Configuring MsDeploy

Before we try and run this command, we need to set up a few things on the target server we are deploying to. I’m assuming you’re already using MsDeploy to deploy websites, and so you can already see IIS Management Service, IIS Manager Permissions, IIS Manager Users, and Management Service Delegation appearing as options under “Management” in your main IIS server configuration screen.

• Create a new IIS user from the IIS Manager Users screen. Alternatively, you can create a Windows user and use that instead.
• Even though we’re installing a service, we still need a target IIS website to associate our credentials with. This could be a dedicated empty website (it doesn’t need to be running) or an existing one. Make sure you replace “DummyWebSiteName” in the command above with the name of the actual website you choose. The underlying path doesn’t matter, as we override the target path as part of our MsDeploy command.
• Go into “IIS Manager Permissions” for the dummy website you are using, click “Allow user” and select either the IIS or Windows user you created above.
• Next, go into “Management Service Delegation”. We need to create two permissions – one so we can deploy the files to the file system, and another so we can run the pre/post sync commands. For the first, click “Add Rule”, select “Blank Rule” and then type “contentPath” in the providers field, * in the actions, set the Path to the one where you are going to deploy the service to. Save that, and add another blank rule.
• For this second rule, type “runCommand” in the providers field, “*” in actions, and choose “SpecificUser” under the Run As… Identity Type field. We need to run under elevated permissions in order to stop/start services and install them. Choose a user account that has these credentials.

File and user account permissions

In order for everything to work, we need to ensure that MsDeploy can access the folder we’re deploying to. We also need to extend the Local Service account so that it can impersonate a more elevated user in order to run the console commands necessary to stop/start and install services (note there are security implications for this – see MSDN for more details.).

• Add read/write access to Local Service account to the target deployment folder
• Run the following command on the console

sc privs wmsvc SeChangeNotifyPrivilege/SeImpersonatePrivilege/SeAssignPrimaryTokenPrivilege/SeIncreaseQuotaPrivilege

• Finally, you need to restart the Web Management Service for this to take effect.

If all has been set up correctly, you should now be all good to go – services will automatically deploy and get started!

Ignoring/preserving files

In a similar fashion to when deploying websites, you may find you wish to preserve logging folders and similar during deployment. You can do this by adding some additional parameters to the MsDeploy command. For instance:

-skip:objectName=filePath,skipAction=Delete,absolutePath=\\Logs\\.*$

will preserve any files in the Logs directory.

Common error messages & troubleshooting

When starting out with MsDeploy it’s likely you’ll hit a fair number of permission denied errors – without too much more information. Logging is your friend.

Request logging - enabled through the Management Service configuration window in IIS, you will find requests logged to %SystemDrive%\Inetpub\logs\WMSvc

Failed request tracing – enabled through the Management Service Delegation configuration window, click “Edit Feature Settings” and “Enable failed request tracing logs”. You will find these at C:\inetpub\logs\wmsvc\TracingLogFiles\W3SVC1

Web Management Service Tracing - enabled through a registry key, described on MSDN.

Below I’ve included some common error messages and some possible causes.

“Connected to the destination computer (“xyz”) using the Web Management Service, but could not authorize. Make sure that you are using the correct user name and password, that the site you are connecting to exists, and that the credentials represent a user who has permissions to access the site.”

Probably because the username and password you are using are invalid (they haven’t been set up) or do not have permissions set for the particular “dummy” website you are targeting.

“Could not complete an operation with the specified provider (“runCommand”) when connecting using the Web Management Service. This can occur if the server administrator has not authorized the user for this operation.”

Most likely you have not set up the correct delegated services through the Management Service Delegation window – either no runCommand permissions have been set, or the delegated user doesn’t have permissions to run the command.

Could not complete an operation with the specified provider (“dirPath”) when connecting using the Web Management Service. This can occur if the server administrator has not authorized the user for this operation.

Either you haven’t set the dirPath permissions via the Management Service Delegation window, or the Local Service account does not have read/write access to the specified directory.

This occurred for me if you haven’t given the Web Management Service permissions to impersonate another user using the sc privs described above, or you have, but haven’t restarted the service yet.

Info: Updating runCommand. Warning: Access is denied. Warning: The process ‘C:\Windows\system32\cmd.exe’ (command line ‘/c “C:\Windows\ServiceProfiles\LocalService\AppData\Local\Temp\giz2t0kb.0ay.cmd”‘) exited with code ’0×1′.

This occurred for me if I had set the Management Service Delegation for runCommand, but left the service running as it’s built-in identity rather than “RunAs”… “Specific user”.

I hope this helps someone!

I tripped up on a strange quirk working with the Image and ImageFormat classes recently. The intention was simple – load an Image object from an existing graphic, generate a thumbnail, and save it out in the original format. The Image class in .NET includes a handy “RawFormat” property indicating the correct format to save out in. So far, so easy. Except the object that RawFormat was returning didn’t seem to match any supported ImageFormat, and the Guid was one character out. For example, when loading a JPEG, you got:

b96b3caa-0728-11d3-9d7b-0000f81ef32e

when the Guid for ImageFormat.Jpeg.Guid was in fact

b96b3cae-0728-11d3-9d7b-0000f81ef32e

It turns out that the “RawFormat” seems to change to an internal format the moment you start modifying the original image. So the simple trick is to save the value of the RawFormat property first, do your modifications, and then save out the image using the original RawFormat value.

Just a quick note – if you switch from Community Server to WordPress like I have, in order to keep your links working you can add a simple regex rewrite rule to IIS. I simply used the following:

^/james_crowley/archive/(.*).aspx$

http://www.jamescrowley.co.uk/$1/

and

^/james_crowley/(.*)$

http://www.jamescrowley.co.uk/$1

where /james_crowley/ was where my blog was installed previously (on weblogs.asp.net as it happens).

Google Analytics supports a nifty feature called “Events”, which is designed to allow you to track non-pageview type events. This is particularly helpful if you have an AJAX type interface on which you want to gather statistics, but another use I’ve found handy is to track clicks on external links to other sites. If you’re using the asyncronous version of the tags (if not, why not), then you should have some code that uses the window._gaq variable. In order to track events aside from the initial page view, you simply need to call the following each time you want to record an event:

window._gaq.push(['_trackEvent','Event Category', 'Event Value']);

Using your favourite javascript library (mine are jQuery and Mootools), it’s easy to hook this up to automatically fire Google event tracking for any external hyperlinks on the site. We simply look for all a tags with href attributes that begin with “http://”. Then, if the href doesn’t contain our current hostname, then we assume it’s an external link. Obviously the logic could be adjusted for your particular needs!

Here’s the mootools version:

document.getElements('a[href^="http://"]').addEvent('click',function(link) {
  var href = link.target.href;
  if(href.indexOf(window.location.host) < 0) {
    window._gaq.push(['_trackEvent','Outbound Links', href]);
  }
});


Or for the jQuery fans amongst you, just swap the first line for:

$('a[href^="http://"]').bind('click',function(link) {

Now, after 24 hours or so, if you check out the "Event Tracking" page under "Content" in Google Analytics, you'll see an outbound link category listing all the external links clicked on the site (assuming Javascript is turned on, of course).

Side note: previous versions recommended by Google included a window.timeout before allowing the redirect to take place - in order to ensure the request to Google to record the click goes out first - as far as I can establish, this is no longer necessary.

Side note 2: By tracking outbound clicks this will affect your bounce rate figures. Essentially an event counts as another activity, so if a visitor lands on one page, and then clicks an external link, that will not count as a "bounce". Whether this is a fair reflection of bounces or not depends on your viewpoint - but something to bear in mind. Unfortunately there's currently no way to log an event that doesn't affect the bounce rate.

An open letter to Google.

I’ve been doing online publishing since 1999, across various successful websites, and have never knowingly been affected by any of your previous algorithm changes – sticking to the motto of great content, and optimizing for our user experience had so far done us well.

However, the Google “panda” algorithm changes being made over the last few weeks seem to have be far more wide reaching than any previously. I’m totally behind the goals you’ve stated to reduce the number of spam sites and content farms, but I’m sure I speak for a large number of publishers that we feel let down in the way this has been handled.

We are now consistently seeing exactly the kind of sites you claimed to want to punish, ripping off our content and ranking for our articles, while our original content is nowhere to be seen. I have talked to your PR team, contacted your search team, and all been given a standard response. We’ve posted in forums, filed reconsideration requests, and reported the people ripping our content off as spam. All with no effect.

I’ve no doubt that it was well within the technical realms of Google to establish which large, well established, high quality editorial sites would benefit and suffer – and so can only assume that a decision was made to press ahead regardless. While the really big publishers will no doubt survive, at the same time you are no doubt making or breaking thousands of small businesses across the web, who like it or not are dependent on traffic from Google, and yet have made every effort to produce genuine high quality original content for their audience.

We recently re-designed Developer Fusion and as part of that we needed to ensure that any external links were not broken in the process. In order to monitor this, we used the awesome LogParser tool. All you need to do is open up a command prompt, navigate to the directory with your web site’s log files in, and run a query like this:

"c:\program files (x86)\log parser 2.2\logparser" "SELECT top 500 cs-uri-stem,COUNT(*) as Computed FROM u_ex*.log WHERE sc-status=404 GROUP BY cs-uri-stem order by COUNT(*) as Computed desc" -rtp:-1 > topMissingUrls.txt


And you’ve got a text file with the top 500 requested URLs that are returning 404. Simple!

If you want to post to a facebook page using the Facebook Graph API and the Facebook C# SDK, from an “offline” app, there’s a few steps you should be aware of.

First, you need to get an access token that your windows service or app can permanently use. You can get this by visiting the following url (all on one line), replacing [ApiKey] with your applications Facebook API key.

http://www.facebook.com/login.php?api_key=[ApiKey]&connect_display=popup&v=1.0

&next=http://www.facebook.com/connect/login_success.html&cancel_url=http://www.facebook.com/connect/login_failure.html
&fbconnect=true&return_session=true&req_perms=publish_stream,offline_access,manage_pages&return_session=1
&sdk=joey&session_version=3

In the parameters of the URL you get redirected to, this will give you an access key. Note however, that this only gives you an access key to post to your own profile page. Next, you need to get a separate access key to post to the specific page you want to access. To do this, go to

https://graph.facebook.com/[YourUserId]/accounts?access_token=[AccessTokenFromAbove]

You can find your user id in the URL when you click on your profile image. On this page, you will then see a list of page IDs and corresponding access tokens for each facebook page. Using the appropriate pair,you can then use code like this:

var app = new Facebook.FacebookApp(_accessToken);
var parameters = new Dictionary
{
    { "message",  promotionInfo.TagLine },
    { "name" ,  promotionInfo.Title },
    { "description" ,  promotionInfo.Description },
    { "picture", promotionInfo.ImageUrl.ToString() },
    { "caption" ,  promotionInfo.TargetUrl.Host },
    { "link" ,  promotionInfo.TargetUrl.ToString() },
    { "type" , "link" },
};
app.Post(_targetId + "/feed", parameters);

And you’re done!

Visual Studio 2010 doesn’t have the same support for app.config files in the way that their web projects do, in order to vary connection strings and other configuration settings for different release modes – a real shame. You can vote on the issue here. In the meantime though, the ASP.NET team have a fix, detailed here.

All you need to do is save their custom targets file, add an imports tag immediately before the closing tag:

  ...
  <Import Project="$(MSBuildExtensionsPath)\Custom\TransformFiles.targets" />
</Project>

And add a TransformOnBuild metadata property to each config file you want transformed. So

<None Include="app.config" />

becomes

<None Include="app.config">
  <TransformOnBuild>true</TransformOnBuild>
</None>

(note you don’t need to do this on the configuration specific config files such as app.release.config). Then you can write your app.Release.config and similar files in the same way you do for web.config files. Sweet!

If you’re trying to use web.config transformations in VS 2010 with nHibernate you might be hitting the same issue I’ve been getting the transformation to match the hibernate-configuration node. The reason is because you have to specify an xmlns="urn:nhibernate-configuration-2.2" attribute as part of the configuration:

<configuration>
<hibernate-configuration xmlns="urn:nhibernate-configuration-2.2">
    <session-factory>
      <property name="connection.connection_string">connection string</property>
      ...
    </session-factory>
  </hibernate-configuration>
</configuration>

To fix, just set the namespace on the target node in the transformation file like this:

<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform"
               xmlns:hib="urn:nhibernate-configuration-2.2">
  <hib:hibernate-configuration>
    <hib:session-factory>
      <hib:property xdt:Transform="Replace" xdt:Locator="Match(name)" name="connection.connection_string">your connection string</hib:property>
    </hib:session-factory>
  </hib:hibernate-configuration>
</configuration>