Search Webservice for SharePoint / Search Server with configurable timeout December 10, 2009

Anyone who has used Microsoft Search Server or the search.asmx Query service in Sharepoint to search a site of any size has probably run into the System.ServiceProcess.TimeoutException error. This is caused by the search failing to complete within 10 seconds at which point sharepoint throws this exception. Anyone who has then Googled this error will probably have found out that this 10 second limit is (amazingly) hard coded into the QueryService class and so cannot be extended. The only suggestion I have seen for getting round this is to write your own webservice which uses the same classes as QueryService to perform a search and so that is what I have done.

To make use of this code:
1. Download and unzip the files

2. Copy:

SearchWithTimeout.asmx
SearchWithTimeoutWsdl.aspx
SearchWithTimeoutDisco.aspx

To: c:\Program Files\Common Files\Microsoft Shared\web server extensions\12\ISAPI

3. Install Guardian.SearchServer.dll into the GAC. The easiest way to do this is to copy the dll to C:\Windows\Assembly

In the assembly in which you reference the webservice you need to change the Config setting in System.ServiceModel\bindings\basicHttpBinding\binding
to set the mode to “TransportCredentialOnly” and the Transport clientCredentialType to “Windows” or “NTLM” (this is the same as with the original Search Server webservice, more info here). This should look as follows:

  <system.serviceModel>
    <bindings>
      <basicHttpBinding>
        <binding name="SearchWithTimeoutSoap" closeTimeout="00:01:00"
          openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00"
          allowCookies="false" bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"
          maxBufferSize="65536" maxBufferPoolSize="524288" maxReceivedMessageSize="65536"
          messageEncoding="Text" textEncoding="utf-8" transferMode="Buffered"
          useDefaultWebProxy="true">
          <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
            maxBytesPerRead="4096" maxNameTableCharCount="16384" />
          <security mode="TransportCredentialOnly">
            <transport clientCredentialType="Windows" proxyCredentialType="None"
              realm="" />
            <message clientCredentialType="UserName" algorithmSuite="Default" />
          </security>
        </binding>
      </basicHttpBinding>
    </bindings>

You can then call this webservice with the same queryXml you would pass in to the Search Server webservice as well as the timeout in milliseconds

var searchServiceQuery = new SearchServer.SearchWithTimeoutSoapClient();

            searchServiceQuery.ClientCredentials.Windows.ClientCredential.Domain = ConfigurationManager.AppSettings["SearchServer.Domain"];
            searchServiceQuery.ClientCredentials.Windows.ClientCredential.UserName = ConfigurationManager.AppSettings["SearchServer.Username"];
            searchServiceQuery.ClientCredentials.Windows.ClientCredential.Password = ConfigurationManager.AppSettings["SearchServer.Password"];
            searchServiceQuery.ClientCredentials.Windows.AllowedImpersonationLevel = System.Security.Principal.TokenImpersonationLevel.Impersonation;
            searchServiceQuery.ClientCredentials.Windows.AllowNtlm = true;

            //Verify that the Search Server is up and running
            if (string.Compare(searchServiceQuery.Status(), "ONLINE", true) != 0)
                throw new Exception("The Search Server isn't online.");

            var timeout = int.Parse(ConfigurationManager.AppSettings["SearchServer.TimeoutInMilliseconds"]);
            var searchResults = searchServiceQuery.QueryEx(query, timeout);

            searchServiceQuery.Close();

Notes
My webservice only has a subset of the methods that can be found on search.asmx/QueryService, this is because the missing methods used classes which were internal to Microsoft.Office.Server.Search.dll so I couldn’t reproduce their functionality. If you need the other methods then reference both webservices.
The methods I have reproduced are:

• QueryEx
• Query
• Status
• GetSearchMetadata

I have only tested Status and QueryEx, so I would be keen to hear from anyone on whether the other methods work as expected.
I have also had to make the following changes (all of them because the original code used internal classes):

1. removed some logging code that was in the original webservice
2. changed an exception type
3. exceptions do not load messages from the resource file

Making IEnumerable Default Comparer work with your own types November 26, 2009

As part of the Open Platform client library I have created a type for the tags that are returned by the api:

	//Attributes omitted for brevity
	public class Tag
	{
		public string Name { get; set; }
		public string Type { get; set; }
		public string Filter { get; set; }
		public string ApiUrl { get; set; }
		public string WebUrl { get; set; }
	}

The Filter property acts like a primary key so if two Tag objects have the same Filter then they should be considered equal.
This test describes the behaviour I want:

		[Test]
		public void Test_tag_equality()
		{
			Tag t1 = new Tag { Filter = "/Rupert" };
			Tag t2 = new Tag { Filter = "/Rupert" };
			Tag t3 = new Tag { Filter = "/Harry" };
			Tag t4 = t3;
			Assert.AreEqual(t1, t2);
			Assert.AreNotEqual(t1, t3);
			Assert.AreNotSame(t1, t2);
			Assert.AreSame(t3, t4);
		}

So I overrode Equals

        public override bool Equals(object obj)
        {
            return Filter == ((Tag) obj).Filter;
        }

and my test passed.
However later on I wanted to use IEnumerable.Intersection() to find all the elements from one list that were also in another list as follows:

//Get the tags which match with music types
            var musicTypes = new[]
			{
				new Tag{Filter="/music/popandrock"},
                new Tag{Filter="/music/classicalmusicandopera"},
                new Tag{Filter="/music/electronicmusic"},
                new Tag{Filter="/music/urban"},
                new Tag{Filter="/music/folk"},
                new Tag{Filter="/music/worldmusic"}
            };

            //filter the results to get the ones which are tagged with the musicType tags
            var filtered = results.Results.Where(c => c.TaggedWith.Intersect(musicTypes).Count() > 0);

but my intersection always returned 0 matches, in other words the way that Intersect() was comparing the equality of the elements of my IEnumerable was not using my Equals() override (or not exclusively anyway). There is an overload for this function that takes a custom EqualityComparer, but I didn’t want to use that.
I investigated a bit more and found that IEnumerable uses the IEquatable interface to determine whether elements are equal so I implemented this and thought that would be that:

	public class Tag : IEquatable<Tag>
	{
		...

		#region IEquatable<Tag> Members

		public bool Equals(Tag other)
		{
			return Filter == other.Filter;
		}

		#endregion
	}

but my intersection still failed to return any results.
After quite a bit of head scratching I tried implementing GetHashCode() as well and finally it worked

	//The final implementation
	public class Tag : IEquatable<Tag>
	{
		...

		public override bool Equals(object obj)
		{
			return Filter == ((Tag)obj).Filter;
		}
		public override int GetHashCode()
		{
			return Filter.GetHashCode();
		}
		#region IEquatable<Tag> Members

		public bool Equals(Tag other)
		{
			return Filter == other.Filter;
		}
		#endregion
	}

Stepping through the code with the debugger it seems that all of these methods are called. It actually works correctly without the IEquatable implementation, but I have left it in for completeness.

New .Net Client Library for Guardian Open Platform November 26, 2009

I’ve recently created a .Net client library for the Guardian’s Open Platform API which allows you to query guardian.co.uk, pull back content and ‘Build applications with the Guardian’.

The project is open source and hosted on codeplex.

You can find out more about the Open Platform API here:

http://www.guardian.co.uk/open-platform

Use CacheProfile attribute with output caching in asp.net mvc November 22, 2009

I’ve recently been setting up some caching on a new Asp.Net MVC site by using the OutputCache attribute on my controllers:

//cache this for an hour
 [OutputCache(Duration=60 * 60, VaryByParam="")]

This works really well except that it means hard codeding the cache time into my app so if I want to change it I need to recompile and deploy my code which is obviously far from ideal. So I changed my code to load the value from the web.config and ran into this error:

‘An attribute argument must be a constant expression, typeof expression or array creation expression of an attribute parameter type’

Not very helpful.

But fortunately there is a better way of doing all this using cache profiles, I just set up a cache profile in my web.config:

<system.web>
<caching>
<outputCacheSettings>
<outputCacheProfiles>
<add name="HomePage" duration="3600" varyByParam="None" location="ServerAndClient"/>
</outputCacheProfiles>
</outputCacheSettings>

</caching>

</system.web>

And then reference this in my attribute:

 [OutputCache( CacheProfile="HomePage")]

And bingo, we’re sucking diesel as a friend of mine says!

Create an Asp.Net MVC HtmlHelper for use in unit tests November 17, 2009

Here’s a utility factory class to create an HtmlHelper instance so that you can unit test extension methods written on it. It will take a controller and a model for methods which rely on those. I adapted it from this post, and changed it to work with Asp.Net version 1.0 and to accept a model.

class HtmlHelperFactory
 {
 public static HtmlHelper CreateInstance(RouteData route, Controller controller)
 {
 return CreateInstance(route, controller, null);
 }
 public static HtmlHelper CreateInstance(RouteData route, Controller controller, object model)
 {
 HttpContextBase httpContext = new HttpContextDummy();

 var cc = new ControllerContext(httpContext, route, controller);
 var vd = new ViewDataDictionary(model);
 ViewContext vc = new ViewContext(cc, new ViewDummy(), vd, new TempDataDictionary());
 return new HtmlHelper(vc, new ViewDataContainerDummy(vd), new RouteCollection());
 }

 // Dummy classes needed to be able to create HtmlHelper

 private class HttpRequestDummy : HttpRequestBase
 {
 public override string ApplicationPath
 {
 get { return ""; }
 }

 public override string AppRelativeCurrentExecutionFilePath
 {
 // Any shorter string here gives exception:
 // index larger than length of string
 get { return "~/"; }
 }

 public override string PathInfo
 {
 get { return ""; }
 }
 }

 private class HttpResponseDummy : HttpResponseBase
 {
 public override string ApplyAppPathModifier(string virtualPath)
 {
 return virtualPath;
 }
 }

 private class HttpContextDummy : HttpContextBase
 {
 public override HttpRequestBase Request
 {
 get { return new HttpRequestDummy(); }
 }

 public override HttpResponseBase Response
 {
 get { return new HttpResponseDummy(); }
 }
 }

 private class ViewDummy : IView
 {
 public void Render(ViewContext viewContext, System.IO.TextWriter writer)
 {
 throw new NotImplementedException();
 }
 }

 private class ViewDataContainerDummy : IViewDataContainer
 {
 public ViewDataContainerDummy()
 {
 }

 public ViewDataContainerDummy(ViewDataDictionary dataDictionary)
 {
 _data = dataDictionary;
 }

 private ViewDataDictionary _data;
 public ViewDataDictionary ViewData
 {
 get { return _data; }
 set { _data = value; }
 }
 }
 }

 

Access RouteData for the current Route from a library class in asp.net mvc October 7, 2009

Sometimes it’s useful to be able to access the current RouteData from outside a controller, for instance in a library class. In webforms you could always do

HttpContext.Current

but in Asp.Net MVC it’s a bit less obvious how you get access to this data.

I’ve found the following code works:

RouteTable.Routes.GetRouteData(new HttpContextWrapper(HttpContext.Current));

Whether there is an easier way or not I don’t know, but this will do me for now…

Group By with Count in Linq September 29, 2009

To group a sequence getting a count for each element of the grouped sequence using Linq:

            var wordList = new List<String> { "test", "one", "test", "two" };
            var grouped = wordList
                .GroupBy(i => i) //Group the words
                .Select(i => new { Word = i.Key, Count = i.Count() }); //get a count for each

Which results in this sequence:

An Implementation of Fold for C# August 20, 2009

I expect there are a number of implementations of this around, but I couldn’t find one so I wrote this. It is an extension method which implements Fold on IEnumerable

public static class Extensions
{
  public delegate TDestination FoldOperation<TSource, TDestination>(TSource item, TDestination acc);
  public static TDestination Fold<TSource, TDestination>(this IEnumerable<TSource> list, FoldOperation<TSource, TDestination> foldOperation, TDestination acc)
  {
     foreach(TSource item in list)
     {
       acc = foldOperation(item, acc);
     }
     return acc;
  }
 }

I can then use this Fold function for something like this which takes an object with a number of properties and creates a querystring with the name/value pairs of all of those properties which are non-null:

public class UrlConstructor
{
  //Takes a RequestParameter object with a number of properties which
  //if their value is set should be translated into querystring parameters
  public static string ConstructUrl(RequestParameters rp)
  {
    //Get all the properties where the value is not null
    var values = typeof(RequestParameters).GetProperties().Where(p => (p.GetValue(rp, null) != null));

    //Fold these values up into a string
    var querystring = values.Fold(((p, acc) => acc + p.Name + "=" + p.GetValue(rp, null) + "&"), "").TrimEnd('&');

    return "http://mybaseurl.com/default.aspx?" + querystring;
  }
}

Enumeration model binder for asp.net mvc August 8, 2009

We were recently working on an asp.net MVC app and I noticed that in a lot of places actions were taking strings as parameters and then converting them to enum values with the action. The reason for doing this rather than just typing the parameter as the enum type was because they were optional and so when the value didn’t exist an error was thrown:

screenshot of error page

Because enums are not nullable types they can’t be used as optional parameters.

What was needed then was a way to provide a default parameter for when the parameter didn’t exist in the url. To do this I wrote a custom model binder and it solved the problem pretty well.

Here is the code for the binder

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.Mvc;

namespace EnumTest.Helpers
{
public class EnumBinder<T> : IModelBinder
{
private T DefaultValue { get; set; }
public EnumBinder(T defaultValue)
{
DefaultValue = defaultValue;
}

#region IModelBinder Members
public object BindModel(ControllerContext controllerContext, ModelBindingContext bindingContext)
{
return bindingContext.ValueProvider[bindingContext.ModelName] == null
? DefaultValue
: GetEnumValue(DefaultValue, bindingContext.ValueProvider[bindingContext.ModelName].AttemptedValue);
}

#endregion

public static T GetEnumValue<T>(T defaultValue, string value)
{
T enumType = defaultValue;

if ((!String.IsNullOrEmpty(value)) && (Contains(typeof(T), value)))
enumType = (T)Enum.Parse(typeof(T), value, true);

return enumType;
}
public static bool Contains(Type enumType, string value)
{
return Enum.GetNames(enumType).Contains(value, StringComparer.OrdinalIgnoreCase);
}
}
}

And to use it in your project you just register it in your Global.asax, specifying the type of your enum and the default value you want when there is no value available in the input.

protected void Application_Start()
        {
            RegisterRoutes(RouteTable.Routes);
            ModelBinders.Binders.Add(typeof(TestEnum), new EnumBinder<TestEnum>(TestEnum.rupert));
        }

Functional programming techniques in C# February 11, 2008

I’ve been reading quite a lot about functional programming recently, as well as playing around with F#, and I’ve found that it is starting to subtly change the way I write code. Recently I was working on an approach to using Linq to Sql with Asp.Net, and I found myself slightly dissatisfied with the following methods:

public static void Insert(TEntity entity)
{
    Insert(entity, false);
}
public static void Insert(TEntity entity, bool submitChanges)
{
    EntityTable.InsertOnSubmit(entity);
    if (submitChanges)
        DataContext.SubmitChanges();
}

public static void Update(TEntity entity)
{
    Update(entity, false);
}
public static void Update(TEntity entity, bool submitChanges)
{
    TEntity original = GetEntity(entity);
    UpdateOriginalFromChanged(ref original, entity);
    if (submitChanges)
        DataContext.SubmitChanges();
}

public static void Delete(TEntity entity)
{
    Delete(entity, false);
}
public static void Delete(TEntity entity, bool submitChanges)
{
    TEntity delete = GetEntity(entity);
    EntityTable.DeleteOnSubmit(delete);
    if (submitChanges)
        DataContext.SubmitChanges();
}

This is pretty straightforward stuff, we have 3 methods Insert, Update and Delete, each of which takes an entity of type TEntity (they are in a generic class) and an overload for each, which also takes a bool to allow client code to specify whether or not to call SubmitChanges on the DataContext immediately. What I dislike about this code is the repeated if statements in each of the overloads, surely there should be a way to refactor those out? This was when a post I’d been reading on Continuation Passing Style popped into my head and I realised that there is, if we use a more functional approach. What we need to do is to create a function ConditionalSubmit() that takes 3 parameters

• Our TEntity entity
• The boolean parameter that tells us whether to submit changes or not.
• A function which will be one our Insert, Update or Delete operations

ConditionalSubmit() can call the function which was passed in, with the entity and then submit the changes or not, depending on the value of the submitChanges parameter. Here is what it looks like:

protected static void ConditionalSubmit(TEntity entity, bool submitChanges, Action<TEntity> operation)
{
	operation(entity);
	if(submitChanges)
		DataContext.SubmitChanges();
}

Now I need to change my implementation of the Insert, Update and Delete methods as follows (I’ve just shown Update for the sake of brevity):

public static void Update(TEntity entity, bool submitChanges)
{
    ConditionalSubmit(entity, x => Update(x), submitChanges);
}

protected static void Update(TEntity entity)
{
    TEntity original = GetEntity(entity);
    UpdateOriginalFromChanged(ref original, entity);
}

The overload without the bool parameter now becomes the version that does all the work, and the overload with the bool passes this version into the ConditionalSubmit() function, using a lambda, along with the other parameters. I think this is a really elegant solution as it abstracts the submit changes functionality away into another function and allows us to get rid of all that repeated code.

Is this really a better solution though? I’m not sure, I think it’s probably just different. In fact in the final code I went back to the original version, purely because I thought it would be easier to understand for the majority of C# developers, and this is code which will need to be used and maintained by others. Having said that I am sure there will be times when a functional approach really does produce a better solution, so I found it very useful to work this through, to get myself to start thinking about coding in a more functional style.