Tidy IEqualityComparer with GenericEqualityComparer

Whilst looking through a codebase, I saw implementations of IEqualityComparer<>. After thinking to myself that the need to create an entire implementation of IEqualityComparer<> per use creates quite a bit of boilerplate for such a small amount of signal, I realised that creating a generic implementation of IEqualityComparer<> that takes a definition of equality in its constructor would be very simple.

public class GenericEqualityComparer<T> : IEqualityComparer<T>
{
	private readonly Func<T, T, bool> mEqualsFunc;
	private readonly Func<T, int> mGetHashCodeFunc;

	public GenericEqualityComparer(Func<T, T, bool> equalsFunc, 
		Func<T, int> getHashCodeFunc)
	{
		if (equalsFunc == null)
			throw new ArgumentNullException("equalsFunc");
		if (getHashCodeFunc == null)
			throw new ArgumentNullException("getHashCodeFunc");

		mEqualsFunc = equalsFunc;
		mGetHashCodeFunc = getHashCodeFunc;
	}

	public bool Equals(T x, T y)
	{
		return mEqualsFunc(x, y);
	}

	public int GetHashCode(T obj)
	{
		return mGetHashCodeFunc(obj);
	}
}

Creating and using an instance of this class is as simple as

public class TestClass
{
	private static readonly GeneralEqualityComparer<Foo> mFooComparer = 
		new GeneralEqualityComparer<Foo>(
			(x, y) => x.Id == y.Id,
			obj => obj.Id);

	public void GetDistinctFoos(IEnumerable<Foo> foos)
	{
		return foos.Distinct(mFooComparer);
	}
}

However, I was a bit embarrassed when I told my boss, Tony Beveridge, about this great use of generics and Funcs I had thought of, and he told me he had actually implemented exactly the same class some months ago.

Its worth noting that EqualityComparer<T>.Default provides a default implementation using the Equals() and GetHashCode() functions of T.

If you wanted to extend GenericEqualityComparer so you don’t have to provide an implementation for GetHashCode(), you can default mGetHashCodeFunc to always return zero. This will force the Equals function to always be called.

IEnumerable, ReadOnlyCollection, and the missing interface

I’ve been thinking on and off about the appropriate return signature for a method that returns an immutable list of objects, sparked off by reading Eric Lippert’s article, Arrays considered somewhat harmful, and my belief that the value of functional program and growth of parallelism means that immutability is desirable most of the time.

However, once you decide to return an immutable collection, what type do you return?

IEnumerable is not really appropriate. The problem is that an IEnumerable may possibly be only evaluable a single time, or may cost for every evaluation of it you perform. This means that you end up with consumers of your method having to use ToList() or ToArray() to flatten the IEnumerable before consuming it, which is wasteful when your method is returning a bounded collection.

So the only choice you have with .NET is ReadOnlyCollection. Which is okay, but not ideal, I believe.

Firstly this involves specifying a return signature as a concrete type. I prefer my method signatures to be interfaces when primitives are not being used, so they only specify behavour. This also means that you can’t return an object that doesn’t use ReadOnlyCollection as a base class.

The second issue is that ReadOnlyCollection implements ICollection and IList. Whilst the implementation of methods such as Add are explicit, the fact ReadOnlyCollection implements interfaces with methods that are invalid for it creates a class of bugs only findable at run time. Have a look at the following code.

public ReadOnlyCollection<object> GetReadOnly()
{
	ReadOnlyCollection<object> readOnly = new List<object>().AsReadOnly();
}

public void ShowIssue()
{
	ReadOnlyCollection<object> readOnly = GetReadOnly();
	// The next line prevented at compile time
	// readOnly.Add(new object());

	// However this code compiles, unfortunately
	IList<object> iList = GetReadOnly()
	iList.Add(new object()); // Fails with an exception at runtime
}

I think that it would have been sensible for .NET to have had an interface that inherits IEnumerable, that represents a readonly bounded collection, called something like IReadOnlyCollection. It would have a Count and allow read only access to the elements by index. ICollection and IList would both inherit this interface, and ReadOnlyCollection would be the implementation of it.

Update: Firstly, this article doesn’t really cover the differences between immutable and read only. The ReadOnlyCollection doesn’t provide any methods to change the collection membership. However ReadOnlyCollection is only a wrapper around the List, and it does not guarantee that the underlying list is not changed.

Links that may be of interest:

Using Policy Injection and Attributes to preempt calls to non-functioning systems

It’s a waste of processor cycles and user time to make web service calls to systems that are not currently functioning. I was involved in building a solution that allows code that depends on non-functioning systems to be skipped entirely. Code simply needs to be attributed with the systems it uses. Then a policy injection handler will throw an exception without even calling that code if a system is known to be unavailable.

I document the parts of the system I built in this article. The moving parts of involved in this solution are:

  • Agents. The agents are the classes that make web service calls to external systems.
  • FunctionalArea attributes. Agent interfaces are marked up with these attributes to indicate dependencies on external systems.
  • FunctionalAreaUnavailableException – thrown to indicate an agent method call has been made involving an unavailable system.
  • SystemStatusAgent. This service keeps track of the unavailability of systems, by receiving information from the application when a FunctionalAreaUnavailableException is thrown, and through its own monitoring. I don’t document it in this blog post.
  • The InterceptorBehavior. Policy injection causes this to be ran before each agent method call. It throws a FunctionalAreaUnavailableException when an attributed agent has a web service exception, or instead of a method call involving a system the SystemStatusAgent considers unavailable.
  • Global exception handling. Catches FunctionalAreaUnavailableExceptions, notifies the SystemStatusAgent of them, and shows the user an error indicating the system they are trying to work with is currently unavailable. I don’t document it in this blog post.

I was able to policy inject all Agents as they were constructed in our AgentFactory. I wished to use configuration based injection. But (if my memory serves me right) with Unity 2.0 policy injection, you can’t use configuration to generated an injected object that is of a concrete type. I had to specify interception behaviours (Unity synonym for policy handler) in code to use the required interceptor, TransparentProxyInterceptor.

PolicyInjectionHelper makes injection simple for us.

public class PolicyInjectionHelper
{
	private ReadOnlyCollection<Type> mInterceptorTypes;

	/// <summary>
	/// Construct a new interception helper that will provide objects with 
	/// policy injection, using interceptors of the given types, in the
	/// given order.
	/// </summary>
	/// <param name="interceptorTypes">
	/// Type of the interceptors - all must derive from IInterceptionBehavior
	/// </param>
	public PolicyInjectionHelper(IEnumerable<Type> interceptorTypes)
	{
		DBC.Assert(
			interceptorTypes.All(typeof(IInterceptionBehavior).IsAssignableFrom),
			"All interceptors must derive from IInterceptionBehavior");
		mInterceptorTypes = interceptorTypes.ToList().AsReadOnly();
	}

	/// <summary>
	/// Derive a policy inject object from the provided object
	/// </summary>
	public T PolicyInject<T>(T obj) where T : class
	{
		T result = obj;
		if (mInterceptorTypes.Any())
			result = Intercept.ThroughProxy<T>(
				result,
				new TransparentProxyInterceptor(),
				ConstructInterceptorInstances());
		return result;
	}

	private IEnumerable<IInterceptionBehavior> 
		ConstructInterceptorInstances()
	{
		return mInterceptorTypes.
			Select(type => Activator.CreateInstance(type) as IInterceptionBehavior);
	}
}

I utilised this in our AgentFactory as this simplified and abbreviated code shows. The PolicyInjectionHelper means that whilst the interception behaviours couldn’t be specified in config, they are easy to see in code (RequiresFunctionalAreaPolicyHandler below).

public class AgentFactory {
	private static readonly PolicyInjectionHelper mPolicyInjectionHelper = 
		mFunctionalAreaPolicyHandlerEnabled
		? new PolicyInjectionHelper(
			new List<Type> { typeof(RequiresFunctionalAreaPolicyHandler) })
		: new PolicyInjectionHelper(Enumerable.Empty<Type>());
				
	// Snipped singleton code

	// Real construction is more complex
	public T ConstructAgent<T>()
		where T : IBaseAgent
	{
		T agent = GetAgent<T>();
		return mPolicyInjectionHelper.PolicyInject(agent);
	}
}

A pair of FunctionalArea attributes allow agent interfaces and methods to be attributed to indicate dependencies

/// <summary>
/// Decorate agent interfaces and agent interface classes with this attribute to indicate
/// that they require a particular functional area to be available.
/// When decorated methods, or methods within a decorated interface are
/// called, a FunctionalAreaUnavailableException is thrown if
/// the area is unavailable, or if a SoapException occurs during
/// the method
/// </summary>
[AttributeUsage(AttributeTargets.Interface | AttributeTargets.Method, AllowMultiple = false, Inherited = true)]
public class FunctionalAreaRequiredAttribute : Attribute 
{
	public FunctionalAreaRequiredAttribute(FunctionalArea pArea)
	{
		Area = pArea;
	}

	public FunctionalArea Area { get; private set; }

	public bool Equals(FunctionalAreaRequiredAttribute other)
	{
		return !ReferenceEquals(null, other) && 
			   (ReferenceEquals(this, other) || Equals(other.Area, Area));
	}

	public override bool Equals(object obj)
	{
		return !ReferenceEquals(null, obj) &&
			   (ReferenceEquals(this, obj) || Equals(obj as FunctionalAreaRequiredAttribute));
	}

	public override int GetHashCode()
	{
		return Area.GetHashCode();
	}
}

/// <summary>
/// Decorate methods with this attribute to indicate that the functional
/// area requirements of their containing class should be ignored
/// </summary>
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = false)]
public class IgnoreRequiredFunctionalAreaAttribute : Attribute { }

Here is an example of how the attributes are used

[FunctionalAreaRequired(FunctionalArea.SystemX)]
public interface IAnAgent : IBaseAgent
{
	void AMethod();

	[IgnoreRequiredFunctionalArea]
	void NotThisMethod();
}
	
public interface IAnotherAgent : IBaseAgent
{
	[FunctionalAreaRequired(FunctionalArea.SystemY)]
	void AMethod();

	void NotThisMethod();
}

The FunctionalAreaUnavailableException itself is very simple.

public class FunctionalAreaUnavailableException : ApplicationException
{
	private const string MessageTemplate = 
		"Functional area {0} unavailable";

	public FunctionalAreaUnavailableException(FunctionalArea area, 
		Exception ex = null)
		: base(string.Format(MessageTemplate, area), ex)
	{
		Area = area;
	}

	public FunctionalArea Area { get; private set; }
}

I introduced a BaseInterceptionBehavior as I quickly found all my IInterceptionBehavior implementations had commonalities.

public abstract class BaseInterceptionBehavior : IInterceptionBehavior
{
	#region IInterceptionBehavior members

	/// <summary>
	/// Returns a flag indicating if this behavior will actually do anything when invoked.
	/// </summary>
	public bool WillExecute { get { return true; } }

	/// <summary>
	/// Check if an intercepted method invocation should be processed, and process it if
	/// interception is required.
	/// </summary>
	public IMethodReturn Invoke(IMethodInvocation input, 
		GetNextInterceptionBehaviorDelegate getNext)
	{
		return IsInterceptionRequired(input)
			? ProcessInvocation(input, () => getNext()(input, getNext))
			: getNext()(input, getNext);
	}

	/// <summary>
	/// Returns the interfaces required by the behavior for the objects it intercepts. 
	/// </summary>
	public virtual IEnumerable<Type> GetRequiredInterfaces()
	{
		return Enumerable.Empty<Type>();
	}

	#endregion

	/// <summary>
	/// Should this method invocation be intercepted?
	/// </summary>
	protected virtual bool IsInterceptionRequired(IMethodInvocation input)
	{
		// !input.MethodBase.IsSpecialName means properties 
		// aren't intercepted, and also ToString et. al.
		return input.MethodBase.IsPublic && 
			!input.MethodBase.IsSpecialName;
	}

	/// <summary>
	/// Process the intercepted method invocation
	/// </summary>
	protected abstract IMethodReturn ProcessInvocation(IMethodInvocation input, 
		Func<IMethodReturn> processNext);
}

This is the interception behaviour that throws FunctionalAreaUnavailableException as previously described.

/// <summary>
/// Provides agents with functional area enhancements. 
/// If an agent or method is attributed as requiring a functional area:
/// - Call to methods for which the System Status agent considers a 
///   functional area unavailable will throw a 
///   FunctionalAreaUnavailableException exception without the method 
///   being invoked.
/// - SoapExceptions in methods will be wrapped in a 
///   FunctionalAreaUnavailableException for the attributed functional area
/// </summary>
/// <remarks>
/// An assumption is made only methods should have this behaviour - 
/// it uses the standard BaseInterceptionBehavior.IsInterceptionRequired 
/// criteria
/// </remarks>
public class RequiresFunctionalAreaPolicyHandler 
	: BaseInterceptionBehavior
{
	protected override IMethodReturn ProcessInvocation(
		IMethodInvocation input, Func<IMethodReturn> processNext)
	{
		MethodBase method = input.MethodBase;
		FunctionalArea? area = GetFunctionalAreaRequired(method);
		IMethodReturn result;
		if (area.HasValue && !SystemStatus.IsOperational(area.Value))
		{
			// Don't even call the method if
			// the FunctionalArea is unavailable
			result = input.CreateExceptionMethodReturn(
				new FunctionalAreaUnavailableException(area.Value));
		}
		else
		{
			result = processNext();
            Exception ex = result.Exception;
			Type areaExType = typeof(FunctionalAreaUnavailableException);
			if (area.HasValue &&
				ContainsException(ex, typeof(SoapException)) &&
				!ContainsException(ex, areaExType))
			{
				result = input.CreateExceptionMethodReturn(
					new FunctionalAreaUnavailableException(area.Value, 
						ex));
			}
		}

		return result;
	}

	private static FunctionalArea? GetFunctionalAreaRequired(
		MethodBase pMethod)
	{
		return IgnoreRequiredAttributes(pMethod)
			 ? (FunctionalArea?)null
			 : GetAttrs(pMethod).
				 Select(attribute => attribute.Area).
				 SingleOrDefault();
	}
	
	private static IEnumerable<FunctionalAreaRequiredAttribute> GetAttrs(
		MethodBase pMethod)
	{
		var methodAttrs = pMethod.
			GetCustomAttributes<FunctionalAreaRequiredAttribute>();
		var typeAttrs = pMethod.DeclaringType.
			GetCustomAttributes<FunctionalAreaRequiredAttribute>(
				true, true);
		return methodAttrs.Union(typeAttrs);
	}

	private static bool IgnoreRequiredAttributes(MethodBase pMethod)
	{
		return pMethod.
			GetCustomAttributes<IgnoreRequiredFunctionalAreaAttribute>().
			FirstOrDefault() != null;
	}

	private static bool ContainsException(
		Exception pException, Type pSearch)
	{
		return pException != null && 
			(pSearch.IsAssignableFrom(pException.GetType()) ||
			ContainsException(pException.InnerException, pSearch));
	}
}

And that’s that.

Policy Injection has its costs. Apart from the additional complexity that policy injection introduces to your code, the usage of the proxy class introduces some overhead to every usage of a policy injected object. And you should be sure that a requirement is a cross cutting concern – do some reading about Aspect-oriented programming for some ideas about how policy injection should be used.

System availability is a cross-cutting concern for agents, which are defined as the set of classes that provide access to external systems in the application I’m dealing with. And the cost of accessing agents through a proxy is fractional compared to the cost of the actual web service calls involved.

This work resulted in a better user user experience when systems are down by providing fast failure, rather than making users wait for timeouts known to be inevitable. The SystemStatusAgent also provides monitoring of the health of the systems the application depends on. I found Unity policy injection wasn’t entirely intuitive. But I’m pleased with the outcome.

Parsing Event Logs using System.Diagnostics.Eventing.Reader

I’ve just had to analyse a bunch of Event Logs that contain exceptions, produced from a load testing exercise. I needed to turn them into a summary of the counts of each class of exception that occurred.

The exceptions belonging to each class of exception message didn’t generate exactly the same text in the event log data every time. So I decided the simplest way to categorise them was to have each category able to work out if it matches event log data using one of two criteria: either data contains a substring; or data matches a Regex. Flexible enough for the fairly simple requirements of this scenario.

After a little research, I found out about the existence of the System.Diagnostics.Eventing.Reader class, which will parse event logs from either the local or a remote computer, or from an EVTX file. The event logs I was parsing already needed to be saved into a file and archived after each run, so I made the parser use the archived files. I’m still keen to play with parsing logs directly off remote computers at some point.

Here’s some code. I haven’t included all the boring declaration of constants and the like in these code snippets, just the interesting bits. Some of it’s a wee bit hack-ish and not properly structured or parameterised; as this is a small utility for occasional use, the effort to make it really nice isn’t currently justified.

The EventLogMatcher class itself:

public class EventLogMatcher
{
  public EventLogMatcher(string dataContains, string dataRegex, string description)
  {
    if (!(string.IsNullOrEmpty(dataContains) ^ 
        string.IsNullOrEmpty(dataRegex)))
      throw new ArgumentException
        (&quot;One and only one of dataContains and dataRegex must be specified&quot;);
    DataContains = string.IsNullOrEmpty(dataContains) 
      ? null
      : dataContains;
    DataRegex = string.IsNullOrEmpty(dataRegex) 
      ? null 
      : new Regex(dataRegex, RegexOptions.Singleline);
    Description = description;
  }

  public string DataContains { get; private set; }

  public Regex DataRegex { get; private set; }

  public string Description { get; private set; }

  public bool IsDataMatch(string data)
  {
    return (DataContains != null &amp;&amp; data.Contains(DataContains)) || 
      (DataRegex != null &amp;&amp; DataRegex.IsMatch(data));
  }
}

The main loop of the program itself – it processes all the EVTX files in the SourcePath directory:

foreach (string sourceFile in Directory.EnumerateFiles(SourcePath, SourcePattern))
{
  string outputFile = DeriveOutputFilename(sourceFile);
  IDictionary&lt;EventLogMatcher, int&gt;; logMatchCounts = GetInitialisedLogTypeCounts();
  List&lt;UnmatchedEventLog&gt; unmatchedLogs = new List&lt;UnmatchedEventLog&gt;();

  EventLogReader logReader = new EventLogReader(new EventLogQuery(sourceFile, PathType.FilePath));
  for (EventRecord eventInstance = logReader.ReadEvent(); eventInstance != null; 
    eventInstance = logReader.ReadEvent())
  {
    EventLogMatcher matcher = logMatchCounts.Keys.
	  SingleOrDefault(key =&gt; key.IsDataMatch(GetData(eventInstance)));
    if (matcher == null)
      unmatchedLogs.Add(ToUnmatchedLog(eventInstance));
    else
      logMatchCounts[matcher]++;
  }
  WriteMatchedResults(outputFile, logMatchCounts, unmatchedLogs);
};

I love using System.Xml.Linq. It makes parsing XML files really simple, and quite readable. Especially for a utility like this where proper error handling isn’t important.

The code for getting the dictionary of EventLogMatchers and counts:

private static IDictionary&lt;EventLogMatcher, int&gt; GetInitialisedLogTypeCounts()
  {
    return LogTypes.ToDictionary(type =&gt; type, type =&gt; 0);
  } 

  private static IEnumerable&lt;EventLogMatcher&gt; GetParsedLogTypes()
  {
    XDocument source = XDocument.Load(KnownEventLogTypesFileName);
    return source.Elements(EventLogsTypesXName).
      Elements(EventLogsTypeXName).
      Select(ParseAsLogMatcher);
  }

  private static EventLogMatcher ParseAsLogMatcher(XElement element)
  {
    string dataContains = element.GetValueOfOptionalAttribute(DataContainsXName);
    string dataRegex = element.GetValueOfOptionalAttribute(DataRegexXName);
    string description = element.Attribute(DescriptionXName).Value; 
    return new EventLogMatcher(dataContains, dataRegex, description, ignored, defaultReason);
  }

internal static class XElementExtensions
{
  public static string GetValueOfOptionalAttribute(this XElement element, XName attributeName)
  {
    XAttribute attribute = element.Attribute(attributeName);
    return attribute == null ? null : attribute.Value;
  }
}

The actual XML for KnownEventLogTypes.xml is structured like this

&lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot; ?&gt;
&lt;EventLogsTypes&gt;
  &lt;EventLogsType 
     dataContains=&quot;System.Net.WebException: The operation has timed out&quot; 
     description=&quot;The operation has timed out&quot; /&gt;
  &lt;EventLogsType
     dataRegex=&quot;Internal failure Sorry, there was an error. Please try again later.*AnAgent.GetSomethingInteresting&quot;
     description=&quot;Internal failure in AnAgent. GetSomethingInteresting&quot; /&gt;
&lt;/EventLogsTypes&gt;

Not rocket science, but simple and effective.

Next time around I’d look at Powershell to do this: Get-EventLog also looks like a simple way to deal with event logs. But I’m glad to have had the opportunity to learn about System.Diagnostics.Eventing.Reader.