aspdotnetcodesonline.com

.NET Matters: Asynchronous HttpWebRequests, Interface Implementation, and More

james — Fri, 14 Oct 2011 21:44:39 +0000

Asynchronous HttpWebRequests, Interface Implementation, and More
Code download available at: NETMatters0412.exe (163 KB)
Browse the Code Online Q My client and I are working on a client-side application that entails making HttpWebRequests to a server application in order to submit data. We’re trying to limit the number of concurrent connections made by the client so we can limit the load placed on the server. Initially we tried making these requests from ThreadPool threads, but we continually received exceptions specifying that “There were not enough free threads in the ThreadPool object to complete the operation.” So I have two questions. First, why doesn’t the ThreadPool have enough threads; isn’t the whole point of the ThreadPool that it should cause queued work items to block until there are threads available? And second, if we can’t use the ThreadPool to do this, how can we throttle the number of concurrent connections requested?Q My client and I are working on a client-side application that entails making HttpWebRequests to a server application in order to submit data. We’re trying to limit the number of concurrent connections made by the client so we can limit the load placed on the server. Initially we tried making these requests from ThreadPool threads, but we continually received exceptions specifying that “There were not enough free threads in the ThreadPool object to complete the operation.” So I have two questions. First, why doesn’t the ThreadPool have enough threads; isn’t the whole point of the ThreadPool that it should cause queued work items to block until there are threads available? And second, if we can’t use the ThreadPool to do this, how can we throttle the number of concurrent connections requested?A Excellent set of questions, and based on a quick Web search one that has stymied many developers. The first thing to be aware of is that in version 1.x of the Microsoft®.NET Framework, HttpWebRequest never makes synchronous requests. What do I mean by that? Take a look at the code for HttpWebRequest.GetResponse as coded in the Shared Source CLI (SSCLI), shown here omitting the code that checks to see if the response was previously retrieved and that accounts for timeouts: public override WebResponse GetResponse() { ••• IAsyncResult asyncResult = BeginGetResponse(null, null); ••• return EndGetResponse(asyncResult);}A Excellent set of questions, and based on a quick Web search one that has stymied many developers. The first thing to be aware of is that in version 1.x of the Microsoft®.NET Framework, HttpWebRequest never makes synchronous requests. What do I mean by that? Take a look at the code for HttpWebRequest.GetResponse as coded in the Shared Source CLI (SSCLI), shown here omitting the code that checks to see if the response was previously retrieved and that accounts for timeouts: public override WebResponse GetResponse() { ••• IAsyncResult asyncResult = BeginGetResponse(null, null); ••• return EndGetResponse(asyncResult);}As you can see, HttpWebRequest.GetResponse is simply a wrapper around the pairing of BeginGetResponse and EndGetResponse. These operate asynchronously, meaning that BeginGetResponse makes the actual HTTP request from a different thread than the one from which it was called, and EndGetResponse blocks until the request has completed. The net result of this is that HttpWebRequest queues a work item to the ThreadPool for every outbound request. So, we know that HttpWebRequest uses threads from the ThreadPool, but why does that cause a problem if the call to GetResponse is made from a ThreadPool thread? Deadlocks.As you pointed out in your question, the ThreadPool will queue work items which will wait until threads are available to process them. For discussion’s sake, let’s say that the ThreadPool is a pool of only one thread (it’s of course much larger by default). You queue a method that calls HttpWebRequest.GetResponse, and this method starts executing on the sole thread in the ThreadPool. The GetResponse method calls BeginGetResponse, which in turn queues a work item to the ThreadPool, and then calls EndGetResponse to wait for that work item to be completed. Unfortunately, that’ll never happen. The work item BeginGetResponse queued won’t execute until a ThreadPool thread becomes available, but the only thread in the ThreadPool is currently executing the call to EndGetResponse waiting for the HTTP request to complete. Deadlock. “But,” you say, “this is a contrived example with only one thread in the pool, and as you said there are many more threads in the pool than that.” True, but let’s expand the example. What happens if there are two threads in the pool, and you quickly queue two methods that call GetResponse before either has a chance to queue their work items? Same problem, deadlock. What happens if there are 25 threads in the pool and you quickly queue 25 of these methods? Again, deadlock. See the problem?As a workaround for this, the Framework team implemented the exception you’re seeing. At the end of BeginGetResponse, just before the work item is queued, System.Net.Connection.IsThreadPoolLow is used to query how many worker threads are available in the pool. If the number is too low (less than two), an InvalidOperationException is thrown.The good news is that in the .NET Framework 2.0, synchronous requests made with HttpWebRequest.GetResponse are truly synchronous, and this problem goes away, allowing you to queue methods that call GetResponse to your heart’s content. The less-than-good news is that you’re stuck with this problem in version 1.x. One solution, as you point out, is to explicitly throttle the number of work items that you have queued to or are executing in the ThreadPool at any one time. To implement this approach, you need a way to keep track of how many work items are currently outstanding and block new requests when the predetermined limit has been reached.Semaphores are synchronization primitives that maintain a count between zero and some maximum value. Operations are provided that raise and lower the count, the trick being that attempting to lower the count when the count is already zero causes the calling thread to block until another thread comes along and increases the count. This makes semaphores great for a few different purposes. The first, and probably one of the most common scenarios discussed in a college operating systems class, is a producer/consumer model where producer threads generate products later used by consumer threads. The second, and arguably a superset of the first, is the controlling of access to shared resources that can support only a limited number of users. In such a scenario, the semaphore acts as a guard, blocking users’ attempts to access a resource while the maximum allowed number of users are currently accessing it. Sounds perfect for this scenario, right?In fact, it is. Unfortunately, version 1.x of the .NET Framework does not include a semaphore implementation. However, putting together a simple wrapper for the Win32® semaphore object requires only a handful of code, as shown in Figure 1 (note that the .NET Framework version 2.0 does include a Semaphore implementation, and a much more complete one than the class shown here). The DllImport attribute is used with P/Invoke to wrap CreateSemaphore (the Win32 function exposed from kernel32.dll and used to instantiate a semaphore object) and ReleaseSemaphore (also exposed from kernel32.dll and used to increase the count of the semaphore). A semaphore object is referred to using the handle returned by CreateSemaphore, which makes this a perfect opportunity to derive my Semaphore class from WaitHandle and to use the functionality it provides (WaitOne, for example) for waiting on synchronization objects. Using the base class’s WaitOne will cause the semaphore count to be decremented, blocking if the count is already at zero and until the count is increased or until a specified timeout elapses.public sealed class Semaphore : WaitHandle{ public Semaphore() : this(1, 1) {} public Semaphore(int initialCount, int maximumCount) { if (initialCount < 0 || initialCount > maximumCount) throw new ArgumentOutOfRangeException(“initialCount”); if (maximumCount < 1) throw new ArgumentOutOfRangeException("maximumCount"); IntPtr h = CreateSemaphore( IntPtr.Zero, initialCount, maximumCount, null); if (h == WaitHandle.InvalidHandle || h == IntPtr.Zero) throw new Win32Exception(); Handle = h; } public void ReleaseOne() { int previousCount; if (!ReleaseSemaphore(Handle, 1, out previousCount)) throw new Win32Exception(); } [DllImport("kernel32.dll", SetLastError=true)] private static extern IntPtr CreateSemaphore( IntPtr lpSemaphoreAttributes, int lInitialCount, int lMaximumCount, string lpName); [DllImport("kernel32.dll", SetLastError=true)] private static extern bool ReleaseSemaphore( IntPtr hSemaphore, int lReleaseCount, out int lpPreviousCount);}With this semaphore, it's now fairly straightforward to create a class that throttles items queued to the ThreadPool. Such an implementation is shown in Figure 2. The semaphore is initially configured to have an initial and maximum count equal to the maximum number of requests you want to allow to execute simultaneously. When ThreadPoolThrottle.QueueUserWorkItem is invoked, the semaphore’s WaitOne method is called to decrement the count and block if the maximum number of requests are currently executing. Just as with the ThreadPoolWait class from the October 2004 installment of .NET Matters, the user-specified WaitCallback and its associated state are packed into a new state object which is then queued to the ThreadPool as the state for a private method, HandleWorkItem. When invoked by the ThreadPool, HandleWorkItem invokes the user-specified delegate with the user-specified state. It then increments the semaphore to signal that this unit of work has completed execution, allowing a blocked thread to wake up and continue with its request. Assuming nothing else is using the pool at the same time (thus decreasing the number of available threads), a class like ThreadPoolThrottle can be used to successfully throttle the number of work items queued.public class ThreadPoolThrottle : IDisposable{ private Semaphore _throttle; public ThreadPoolThrottle(int maximumAllowed) { if (maximumAllowed < 1) throw new ArgumentOutOfRangeException("maximumAllowed"); _throttle = new Semaphore(maximumAllowed,maximumAllowed); } public void QueueUserWorkItem(WaitCallback callback) { QueueUserWorkItem(callback, null); } public void QueueUserWorkItem(WaitCallback callback, object state) { if (_throttle == null) throw new ObjectDisposedException(this.GetType().FullName); if (callback == null) throw new ArgumentNullException("callback"); _throttle.WaitOne(); try { QueuedCallback qc = new QueuedCallback(); qc.Callback = callback; qc.State = state; ThreadPool.QueueUserWorkItem( new WaitCallback(HandleWorkItem), qc); } catch { _throttle.ReleaseOne(); throw; } } private void HandleWorkItem(object state) { QueuedCallback qc = (QueuedCallback)state; try { qc.Callback(qc.State); } finally { _throttle.ReleaseOne(); } } private class QueuedCallback { public WaitCallback Callback; public object State; } public void Dispose() { if (_throttle != null) { ((IDisposable)_throttle).Dispose(); _throttle = null; } }}Q I'm writing a tool that uses reflection. Given a method on a class, I need to figure out which interface method that class method implements. If it doesn't implement an interface, I need to know that, too. Is this possible?Q I'm writing a tool that uses reflection. Given a method on a class, I need to figure out which interface method that class method implements. If it doesn't implement an interface, I need to know that, too. Is this possible?A First, the question is a bit misleading. You seem to be implying that a method on a class can only be used to implement one method on one interface, but that in fact is incorrect. A class method can be used to implement methods on multiple interfaces, or even multiple methods on the same interface, though not all languages support this functionality. Take the following C# sample as an example: interface SomeInterface1 { void Method1();}interface SomeInterface2 { void Method1();}class SomeClass : SomeInterface1, SomeInterface2 { public void Method1(){}}A First, the question is a bit misleading. You seem to be implying that a method on a class can only be used to implement one method on one interface, but that in fact is incorrect. A class method can be used to implement methods on multiple interfaces, or even multiple methods on the same interface, though not all languages support this functionality. Take the following C# sample as an example: interface SomeInterface1 { void Method1();}interface SomeInterface2 { void Method1();}class SomeClass : SomeInterface1, SomeInterface2 { public void Method1(){}}Here, the method SomeClass.Method1 implements two interface methods, SomeInterface1.Method1 and SomeInterface2.Method1, which both use implicit interface implementations. The C# language provides two ways in which a class method can be mapped to an interface method. One way is through explicit interface member implementation, indicated by naming the class member with both the name of the interface and the name of the interface method, as such: class SomeClass : SomeInterface1 { void SomeInterface1.Method1() {}}This causes the implementation to be exposed publicly but only through the interface. The second way, implicit implementation, is done by matching nonstatic public methods to interface methods by name, type, and formal parameter lists. As such, whereas in C# it's possible (as shown previously) to have one class method implement a method on two different interfaces, it isn't possible in C# to have one class method implement two different interface methods from the same interface. This is because it's invalid to have two methods on the same interface that have the same name, type, and formal parameters, a situation which would be required for a single class method to map to both. Visual Basic® .NET solves this problem by forcing the developer to explicitly state what interface methods a class method is implementing. Thus, in Visual Basic .NET it is possible to have one class method implement two different interface methods, both from the same interface, as shown in the following code snippet: Interface SomeInterface Sub Method1() Sub Method2()End InterfaceClass SomeClass Implements SomeInterface Public Sub SomeMethod() Implements _ SomeInterface.Method1, SomeInterface.Method2 Console.WriteLine("SomeMethod called.") End SubEnd ClassHere, SomeClass.SomeMethod actually implements both SomeInterface.Method1 and SomeInterface.Method2.With that said, back to the core of your question. Yes, it is possible using reflection to determine what interface methods a given class method implements, and one possible solution for doing so is shown in Figure 3.public static Type[] GetImplementedInterfaces(MethodInfo mi){ ArrayList list = new ArrayList(); Type reflectedType = mi.ReflectedType; foreach(Type ift in reflectedType.GetInterfaces()) { foreach(MethodInfo target in reflectedType.GetInterfaceMap(ift).TargetMethods) { if (target.Equals(mi)) { list.Add(ift); break; } } } return (Type[])list.ToArray(typeof(Type));}The Type class exposes the GetInterfaceMap method which, when passed a Type representing an interface, is used to return a mapping that denotes how that interface is mapped into the actual methods on the class that implements that interface. InterfaceMap, the type returned by GetInterfaceMap, exposes two important fields: TargetMethods and InterfaceMethods. These fields store arrays, the former being an array of MethodInfo objects representing the methods on the type that implement interface methods, and the latter being the interface methods that map to the former array. So for example, the method represented by the MethodInfo at TargetMethods[2] implements the interface method represented by the MethodInfo stored at InterfaceMethods[2].My GetImplementedInterfaces method shown in Figure 3 takes a MethodInfo representing the method in question, and it returns an array of Type instances, each of which represents one interface that method is used to implement. First, the method’s ReflectedType is retrieved, which is necessary in order to retrieve all of the interfaces that type implements as well as the interface mapping for that type’s methods. You might notice that the MemberInfo class (from which MethodInfo derives) exposes a DeclaringType property in addition to ReflectedType. It’s important that I not use DeclaringType for this purpose. The difference is that DeclaringType returns the type that declares this member, whereas ReflectedType returns the type that was used to retrieve the given MethodInfo. When are they different, and why does the distinction matter? For every interface implemented by the method’s container class, I need to loop through each of the target methods in the interface map and see if the specified method matches any of those. If it does match, then the specified method is in the interface map for the given interface, which means that the method is used to implement that interface. However, there are certain situations in which you can obtain a MethodInfo for an interface-implementing method but where that MethodInfo won’t match any MethodInfo in the TargetMethods array. Specifically, this can happen when the declaring type and the reflected type differ. Consider the following example: interface SomeInterface { void Method1();}class Base : SomeInterface { public void Method1() {}}class Derived : Base {}Derived.Method1 is actually implemented on the Base class. As a result, the MethodInfo retrieved using typeof(Derived).GetMethod(“Method1″)will be a different object than the MethodInfo retrieved using typeof(Base).GetMethod(“Method1″)If I were to use the GetInterfaceMap from the Base Type (which in this case is the type that declares Method1) to retrieve the interface mapping, but the MethodInfo had been retrieved using typeof(Derived), the MethodInfo won’t match any of the objects in the TargetMethods array. Thus, it’s important to call GetInterfaceMap on the same Type used to retrieve the specified MethodInfo, and that Type is the one returned from the MethodInfo’s ReflectedType property.Q I’ve been reading recently about COM threading models and how .NET fits into the picture. I know I can use the STAThreadAttribute and MTAThreadAttribute to initially set the threading model for the main thread of my application, and I know I can configure a thread using the Thread.ApartmentState property, but both of those techniques only seem to work before any calls to COM objects are made. After that, they appear to be immutable. Is that true? If so, what do I do if I need to use multiple COM objects, each of which uses a different threading model?Q I’ve been reading recently about COM threading models and how .NET fits into the picture. I know I can use the STAThreadAttribute and MTAThreadAttribute to initially set the threading model for the main thread of my application, and I know I can configure a thread using the Thread.ApartmentState property, but both of those techniques only seem to work before any calls to COM objects are made. After that, they appear to be immutable. Is that true? If so, what do I do if I need to use multiple COM objects, each of which uses a different threading model?A The Thread.ApartmentState property can be changed as often as you like until the first calls are made to a COM object. At that point, the threading model for the current thread becomes immutable and you won’t be able to change it. If you need to use COM objects that require a different threading model than that of your current thread, the standard solution is to spin up a new thread, set its ApartmentState appropriately, and then execute the code that uses that COM object on that new thread. (For a refresher in COM threading models, I suggest reading Larry Osterman’s blog post at What are these “Threading Models” and why do I care?.)A The Thread.ApartmentState property can be changed as often as you like until the first calls are made to a COM object. At that point, the threading model for the current thread becomes immutable and you won’t be able to change it. If you need to use COM objects that require a different threading model than that of your current thread, the standard solution is to spin up a new thread, set its ApartmentState appropriately, and then execute the code that uses that COM object on that new thread. (For a refresher in COM threading models, I suggest reading Larry Osterman’s blog post at What are these “Threading Models” and why do I care?.)To make this process a bit easier, I’ve implemented the class shown in Figure 4. ApartmentStateSwitcher exposes only one static method named Execute. This method takes a delegate to invoke, the parameters to pass to the delegate, and the ApartmentState required to execute this delegate. If the ApartmentState of the current thread matches the one requested by the user, the method simply invokes the delegate using the delegate’s DynamicInvoke method, which provides for late-bound method invocation. However, if the apartment states differ, it invokes the delegate on a new thread with the ApartmentState set appropriately. To do this, it creates a small object that it can use to pass the state back and forth between the current and new thread. Into this object it stores the delegate and the parameters with which to invoke it.public sealed class ApartmentStateSwitcher{ private Delegate _delegate; private object[] _parameters; private Exception _exc; private object _rv; private ApartmentStateSwitcher(){} private void Run() { try { _rv = _delegate.DynamicInvoke(_parameters); } catch(MemberAccessException exc) { _exc = exc; } catch(TargetException exc) { _exc = exc; } catch(TargetInvocationException exc) { _exc = exc; } } public static object Execute( Delegate d, object[] parameters, ApartmentState state) { if (d == null) throw new ArgumentNullException(“d”); if (state != ApartmentState.MTA && state != ApartmentState.STA) throw new ArgumentOutOfRangeException(“state”); if (Thread.CurrentThread.ApartmentState == state) { return d.DynamicInvoke(parameters); } else { ApartmentStateSwitcher switcher = new ApartmentStateSwitcher(); switcher._delegate = d; switcher._parameters = parameters; Thread t = new Thread(new ThreadStart(switcher.Run)); t.ApartmentState = state; t.IsBackground = Thread.CurrentThread.IsBackgound; t.Start(); t.Join(); if (switcher._exc != null) throw switcher._exc; return switcher._rv; } }}A new thread is then created with the appropriate ApartmentState, started, and immediately joined with the current thread, blocking the current thread until the new thread finishes execution. The main method for this new thread is Run, a private member of the class that executes the delegate with the supplied parameters. Any return value from the delegate’s invocation is stored to the state class, as is any exception thrown during the process (in version 1.x of the .NET Framework, exceptions thrown on worker threads are simply eaten by the runtime, and in version 2.0, worker thread exceptions cause the AppDomain to shut down; in neither version are exceptions passed back to the main thread, so I have to do that manually). Back in the Execute method, when the worker thread finishes, the Thread.Join call completes and the state class is examined for an exception and a return value. If an exception exists, it is rethrown. If not, the return value from the delegate is returned.Using this class is as easy as creating a delegate around whatever code you want executed (a process made much simpler with anonymous delegates in C# 2.0) and passing that delegate to ApartmentStateSwitcher.Execute, as shown here: [MTAThread]static void Main(string[] args) { PrintCurrentState(); ApartmentStateSwitcher.Execute( new ThreadStart(PrintCurrentState), null, ApartmentState.STA); PrintCurrentState();}static void PrintCurrentState() { Console.WriteLine(“Thread apartment state: ” + Thread.CurrentThread.ApartmentState);}This code prints to the console: Thread apartment state: MTAThread apartment state: STAThread apartment state: MTASend your questions and comments to netqa@microsoft.com.

Stephen Toub is the Technical Editor for MSDN Magazine.

View the original article here

Editor’s Note: New Technologies and a New Magazine

james — Fri, 14 Oct 2011 14:52:39 +0000

Here is the same content in en-us.

New Technologies and a New MagazineHey, remember the Web? Back around 1996 it was big news. People suddenly realized that with just a telephone line and Internet Explorer, they could go shopping without ever leaving the comfort of their home! Over the Web you could buy furniture, art, books, and appliances, all of which would make your home comfortable enough to stay there and do some more online shopping, accelerating the vicious cycle of consumerism. Cool!Over the years, however, the Internet has become an integrated part of the technology picture. Standalone client programs without Internet connectivity are a rarity today. More and more, it seems like innovation is happening at the fringes rather than at the core. Ideas like “breadcrumb” navigation and those annoying Flash ads that obscure the screen you’re trying to read until the “low mortgage prices” cube has finished spinning (before you can find the faint “Close X” button) are just a few examples. But don’t be fooled. There’s still a lot of new stuff to learn about Web programming.This month, we are highlighting a couple of these Web and Internet technologies. We’re talking about the new GridView component in ASP.NET 2.0. It’s a lot like the DataGrid, only better. Carl Franklin discusses peer-to-peer chats and how to design them using the .NET Framework. And Dino Esposito uses his Cutting Edge column to show you how to do partial page refreshes with either ASP.NET 2.0 or 1.x.We’ve received a flurry of letters from readers of late asking whether we can provide source code in more languages. As part of our { End Bracket } column, we’ve asked contributing editor John Robbins to take a look at this request and his solution was to whip up a tool that helps translate C# projects into Visual Basic .NET. To see it, flip to the back page right now!Back yet? Great. If you’re like many of our readers, your job takes you beyond mere development into the world of IT. Perhaps you’re a Windows power user, able to punch your way through arcane Control Panel settings and weird registry entries. Perhaps you know how to make a wireless network function properly without getting your building chalked for being an open access point. Who knows, maybe you even have a second career in the offing—driving in the South American racing circuit. (Probably not, but we need to set up the photo that’s on this page.)It’s become an MSDN Magazine tradition to produce an annual issue that discusses security. While we were planning this year’s version (due out in November), we realized something that in retrospect seems obvious: the people who are interested in security as developers are also looking for solutions for administration. From this simple idea comes a new project. This autumn, we will be publishing a special sister publication, designed for the IT professional—TechNet Magazine.The inaugural issue will provide gobs of technical information about security in the IT world, as well as other real, hands-on information that you (or your admin friends) can use right away. Just surf on over to http://www.technetmagsubs.com and sign up today. We’ll hold the rest of the Editor’s Note until you get back.While you were off signing up for TechNet Magazine, we took the liberty of wheeling our new MSDN Magazine race car into the Editor’s Note. We have been introducing MSDN Magazine to several countries worldwide, including a Portuguese edition in Brazil. So how did the Brazilian publishers choose to advertise the magazine? A t-shirt? A squishy ball? Okay, let’s stop the “list of novelty items” charade—we all know the answer is the semi-official MSDN Magazine race car! At press time, we have no news of how the car actually finished, so we won’t make any painful quips about how we crossed the finish line first with great content or how we displayed the yellow caution flag when covering beta software. All in all, that’s probably for the better.Thanks to the following Microsoft technical experts for their help with this issue: Uwa Agbonile, Mike Ammerlaan, Keith Ballinger, Jay Bazuzi, Rebecca Dias, Mike Fitzmaurice, Kirill Gavrylyuk, Matthew Gibbs, Chris Hays, Polita Huff, Bryan Keller, John Koropchak, Ronald Laeremans, Rangan Majumder, Constantin Mihai, Erik Olson, Matt Powell, Kurt Schmucker, Maura Van Der Linden, and Reid Wilkes.

Active Directory, ActiveX, BizTalk, DirectX, FrontPage, JScript, Microsoft, Microsoft Press, MSDN, SharePoint, Visual Basic, Visual C++, Visual SourceSafe, Visual Studio, Windows, Windows NT, and Win32 are registered trademarks of Microsoft Corporation. Windows Server is a trademark of Microsoft Corporation. Other trademarks or tradenames mentioned herein are the property of their respective owners.

MSDN Magazine does not make any representation or warranty, express or implied with respect to any code or other information herein. MSDN Magazine disclaims any liability whatsoever for any use of such code or other information.

Receive the MSDN Flash e-mail newsletter every other week, with news and information personalized to your interests and areas of focus.

View the original article here

.NET Interop: Getting Started With IronRuby And RSpec, Part 1

james — Fri, 14 Oct 2011 08:59:39 +0000

Getting Started With IronRuby And RSpec, Part 1
This article is based on a prerelease version of IronRuby. All information is subject to change.
This article discusses: Ruby and Duck Typing Ruby and the Microsoft .NET Framework Using IronRuby and RSpec This article uses the following technologies:
IronRuby
“That’s not what we asked for!” I’m sure most developers have heard this cry from a customer shortly after delivering the latest build. The customer could be yelling at the poor developer for various reasons—maybe the requirements were not correctly understood or part of the system simply did not work. Because customers are often unclear as to their own requirements, they will opt for the safety of the 100-page technical document that tries to define everything the system might have to do. Meanwhile, the developer struggles with undocumented legacy code, trying to understand how the app is intended to work while attempting to implement new requirements without breaking anything. Times are changing. New approaches to software development are aimed at solving these problems, helping you to meet the customer requirements at the first attempt without causing failures in the process. These approaches are taking advantage of languages such as Ruby, allowing you to create easily readable and maintainable code with much shorter development iterations. In this article, I will introduce you to Ruby and IronRuby and demonstrate some basics of Ruby interoperating with Microsoft .NET Framework-based code. I also explain how frameworks such as RSpec can be used to generate examples of how objects are intended to behave, providing both documentation and verification that the system is built correctly. This will set the stage for a future article in which I will explain acceptance testing in detail and demonstrate creating acceptance tests with IronRuby. Defining Requirements and Examples in CodeTo help write examples and define requirements you need a framework. There are many different approaches to writing automated acceptance tests or executable specifications. Some people use standard xUnit test frameworks successfully, while others use Fit and Fitness frameworks. I have found the best approach is to use Behavior-Driven Development (BDD). Dan North devised a BDD framework called JBehave as a way to define scenarios that describe the application’s behavior in a way that can be communicated to the whole team, regardless of technical ability. JBehave was the result of problems North faced with Test-Driven Development (TDD) and was implemented for the Java language. Later, North created RBehave, which has since been integrated into RSpec, a popular framework for Ruby. RSpec accommodates two different approaches to BDD: Dan North’s approach, based on stories and scenarios to describe application’s behavior, and Dave Astels’ approach, which is more focused on creating examples at an object level.C# does have some BDD frameworks, such as NSpec and NBehave. The main problem with writing tests in C# is that the true intent of the test is often hidden due to fact that you have extra structural elements and metadata such as braces, public and private. Overall, I don’t think what is on offer via C# and NSpec/NBehave can match what is available via IronRuby and RSpec. Previously, this would have been a major problem for C# developers, as you could not have used the Ruby-based RSpec to test C# apps. With IronRuby, this is no longer a problem. While still early in development, IronRuby is taking advantage of the Dynamic Language Runtime (DLR) to create an implementation of the Ruby language on top of the CLR. With IronRuby, you can employ existing Ruby apps and frameworks together with the .NET Framework and .NET-compliant languages. The result is that you can use the Ruby language and RSpec to test C# applications!As a language, Ruby is concise, allowing you to write less code and express it in a much more natural manner, making your code easier to maintain. For example, to read all the lines of text from a file and write them out to the console, you would write this: File.readlines(‘AboutMicrosoft.txt’).map {|line| puts line}The code opens the file AboutMicrosoft.txt and reads all the lines, passing each line into the block, with the line as a parameter that is then written to the console. In Ruby, a block is the collection of statements between the braces and is similar to invoking a method in C#, which uses the yield statement to return control to the calling method. Its natural language approach is one of the reasons why Ruby is excellent to use when testing. The tests, or scenarios in this case, are much more readable. Ruby and Duck TypingOne of the reasons why Ruby and dynamic languages are easier to read is due to how they handle typing. Instead of the developer defining a type, when the Ruby code is interrupted, the type of the variable is inferred. The language is still strongly typed, but it determines the variable’s type dynamically. With C#, interfaces allow for decoupling of objects while still defining a contract for the implementation. With Ruby, there is no need for contracts and interfaces. Instead, if the object has an implementation of the method you are calling, then it’s called. If it doesn’t, then an error is returned. As such, the contract for the object is defined by its implementation, not its relationship to other parts of the code. To demonstrate this, I created a method that accepts a value, the type of which will be inferred at run time. It will then output Hello plus the value: def SayHello(val) puts “Hello #{val}” endBecause of the way types are handled, you can reuse the same method for both strings and integers without changing any of the code: SayHello(“Test”) => “Hello Test” SayHello(5) => “Hello 5″I could have also have called the method without brackets, as these are optional in Ruby and can make the syntax much more readable: SayHello “Test” => “Hello Test”This can be achieved in C# using object, but let’s look at how this works with more complex objects. I defined a new method to output the result of a call to GetName. As long as the value of the parameter n implements a method called GetName, the code will work:def outputName(n) puts n.GetName()end Here are two unrelated classes: class Person attr_accessor :name def GetName() @name endendclass Product def GetName() “The product with no name” endendPerson has an accessor allowing the name variable to be set, while Product returns a hardcoded value. In Ruby, there is no need to write the return statement, the last result of the last line of code of a method is returned by default. If you call the method outputName, the GetName method is called, demonstrating how Ruby’s typing can enhance code reusability since we are not fixed to a single type:outputName(Product.new) => “The product with no name”$x = Person.new$x.name = “Ben Hall”outputName($x) => “Ben Hall”If you call the method using an argument that doesn’t implement GetName, then a NoMethodError is raised:outputName(“MyName”) => :1:in ‘outputName’: \ undefined method ‘GetName’ \ for MyName:String (NoMethodError)While this concept concerns some people, it can be useful; it allows for improved testability and application design, especially when testing C# applications, which I will discuss in the next section. Ruby and MetaProgrammingSimilar to other dynamic languages, Ruby embraces the concept of meta-programming. This allows you to extend any class or object at run time, enabling you to define methods and behavior at run time, giving you the ability to develop a self-modifying application. This is extremely useful in dynamic languages, especially in unit testing, as it allows you to customize the behavior of objects and methods to your own requirements. In a similar fashion, you can extend the built-in Ruby classes to provide your own functionally, much like the extension methods in C# 3.0. With C#, there are various restrictions about the methods you can create. For example, they have to be static and in a separate class. This all harms readability. With Ruby, you just create a normal method on an existing class. For example, Integer is a class to handle whole numbers. Integer has a series of methods, but it does not have a method to say whether the number is even. To create such a method, you simply create a new class with the same name (Integer) and define the new method. The question mark at the end of the method denotes that a Boolean is returned with the aim of making it easier to read: class Integer def even?() self.abs % 2 == 0 endendputs 2.even? => trueputs 1.even? => falseRuby and the .NET FrameworkIn place of the old debate about whether to use a static or dynamic language, IronRuby lets you use the right language for the job. If it makes more sense to use C#, then you can use C#. If you need a more dynamic approach, you can easily use IronRuby, taking advantage of its interoperability with .NET and the dynamic nature of the Ruby language. I anticipate people using a mixture of different languages and technologies, such as C# for the main application and Ruby for testing. The DLR provides the foundation to enable .NET interop. With this in place, you can take advantage of UI technology such as Windows Forms, Windows Presentation Foundation (WPF), and Silverlight while writing application code in Ruby. Taking advantage of WPF from IronRuby is easy. When the following code is executed, you can have a fully functioning WPF window created from IronRuby. You still must reference mscorlib and the two WPF assemblies shipped with the .NET Framework 3.0, as you did with C#, using the require statements: require ‘mscorlib’require ‘PresentationFramework, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35′require ‘PresentationCore, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35′You can now create and interact with the WPF objects. First, create aliases for all the objects you want to use. This makes the code cleaner, as you won’t need to access the object via its namespace: Window = System::Windows::WindowApplication = System::Windows::ApplicationButton = System::Windows::Controls::ButtonNext, create a WPF window and give it a title: win = Window.newwin.title = ‘IronRuby and WPF Interop’Create a button in the same fashion: mainButton = Button.newmainButton.content = ‘I’m a WPF button — press me’In both instances I’m using the alias to access the full name of the object. When the button is clicked, I want to display a MessageBox to the user. As you would in C#, you can subscribe to the event and provide a block that is called when the click event is invoked: mainButton.click do |sender, args| System::Windows::MessageBox.Show(“Created using IronRuby!”)endFinally, set the content of the window to be the button, create a new Application object, and launch the window:win.content = mainButtonmy_app = Application.newmy_app.run win Figure 1 Using IronRuby to Create a WPF App At this point, a fully interactive WPF window will be displayed with the click event correctly wired up as shown in Figure 1. I interacted with the .NET Framework in the same way I would interact with Ruby libraries. One of the core principals set out by John Lam and team was to stay true to the Ruby language. This is an important principal for adoption by current Ruby developers, who won’t want to change the way they create Ruby applications just because they’re moving to IronRuby. Instead they can access all the rich .NET code in the same way. Ruby and the CLRHere is another example. If you attempt to access AddRange on IEnumerable, in C# the code would look like this:ArrayList list = new ArrayList();list.AddRange(new [] { 1,2,3,4 });However, with Ruby, the accepted convention is for words within method names to be separated by underscores to improve readability. Creating a separate library to follow this convention would be too time-consuming and error-prone, and it would not support additional third-party development. Instead, for CLR objects, IronRuby translates Ruby method calls into the equivalent method name for the CLR: $list = ArrayList.new $list.add_range([1,2,3,4])Here I am accessing the AddRange method following Ruby’s approach, which is lower case with words separated by an underscore. If you prefer, you can keep with the CLR naming convention as the method name still exists from Ruby: Both work the same; it’s just a personal preference which to use.As I mentioned, Ruby infers the type of objects at run time, and this is the same when handling C# objects. Consider a series of C# methods that return different object types, while the same object is being returned, the method signature returns either the interface or the concrete type. Within my example, I have an interface that defines a single HelloWorld method:public interface IHello { string HelloWorld(); }I created a Hello4Times class that inherits from Hello3Times, which inherits from the interface but has implemented three extra HelloWorld methods. Within the class I define a new method called HelloMethodOn4Times that calls the base implementation: public class Hello4Times : Hello3Times { public string HelloMethodOn4Times () { return base.HelloWorld(); }}I then defined a static class and methods that will return a new instance of the Hello4Times class but return it to the calling code as an interface. This means the calling code should only know about HelloWorld, not any of the additional methods defined:public static class HelloWorld { public static IHello ReturnHello4TimesAsInterface() { return new Hello4Times(); }In my Ruby code, I have two method calls. The first call is on the interface-defined method, which you would expect to work without a problem. However, the second call is to the method on the concrete class. IronRuby has determined the type of the object being returned and can dispatch the method calls: puts InteropSample::HelloWorld.ReturnHello4TimesAsInterface.HelloWorldputs interopSample::HelloWorld.ReturnHello4TimesAsInterface.HelloMethodOn4TimesAll of this happens without you ever needing to worry about casting the object to the correct type to call the method.Following this theme, you can extend .NET objects in exactly the same fashion as you can with Ruby objects. When displaying a MessageBox, I don’t want to keep defining which icons and buttons to use. Instead, I just want to provide a message.Maybe you don’t like the built-in functionality and want to extend the actual MessageBox class for seamless interaction. With Ruby, this is simple. You define a new class that is the same as the built in MessageBox within WPF. You then create a new method on the class that simply calls the show method with various defaults: class System::Windows::MessageBox def self.ShowMessage(msg) System::Windows::MessageBox.Show(msg, msg, \ System::Windows::MessageBoxButton.OK, \ System::Windows::MessageBoxImage.Stop) endendAfter executing this block of codeSystem::Windows::MessageBox.ShowMessage( \ “I’m going to show you a message”) you can call the ShowMessage method, the result being the message box shown in Figure 2. Figure 2 Displaying a Custom MessageBox from Ruby Inside IronRubyHow is interop possible? The answer is the DLR. At a high level, when you execute Ruby code using IronRuby, a lot of work takes place behind the scenes. First, the code you write is tokenized and parsed by the IronRuby engine. The parsed code is then converted into the DLR’s Abstract Syntax Tree (AST). This is a standardized AST that all language implementations, such as IronPython, need to provide to the DLR in order to execute the code. Once the DLR has the AST, it converts the tree into Intermediate Language (IL). All .NET code is compiled down into IL to provide a generic language for the CLR. This is how IronRuby can interoperate with the .NET Framework—behind the scenes, all the code is executed as IL instructions. After the DLR has completed the conversion, it passes the IL to the CLR for execution and the result is returned to IronRuby. During this process there are additional steps to improve performance and reliability, such as caching. For a deeper explanation, I recommend reading Bill Chiles’s CLR Inside Out column “IronPython and the Dynamic Language Runtime” in the October 2007 issue of MSDN Magazine.Within a separate assembly there is a hosting API that allows you to embed IronRuby within your own applications, allowing you to execute Ruby code from C# or users to execute their own code. For more about the IronRuby implementation, download the entire source code from RubyForge. IronRuby is an open-source project implemented in C# and is an excellent example of a dynamic language implementation. IronPython is available as an open source project hosted at CodePlex, and it includes a sample language called ToyScript if you want an introduction into how the DLR works.To ensure continued compatibility with the core Ruby platform, the IronRuby team is using RubySpecs. This is a shared set of examples based around how the Ruby language should be implemented. The aim of RubySpecs is to ensure different implementations including Matz’s Ruby Interpreter (MRI), JRuby, MacRuby, and IronRuby have the same behavior. RubySpecs uses a syntax-compatible version of RSpec called MSpec. These are viewed as the acceptance tests for the IronRuby implementation.Testing a C# App with IronRubyFor many years now, the awareness and practice of TDD has increased as a way to develop software, resulting in higher quality code, in terms of design and maintainability, along with fewer defects along the way. With IronRuby, I can use the RSpec specification framework and runner to provide examples of how my C# objects work, following a BDD approach instead of TDD. While the scenario framework, which I’ll cover in a future article, is more aligned with acceptance testing at the app level, the specification framework is more aligned to developers writing their own specifications about the code they are just about to implement and its expected behavior. The specification framework is based around providing examples describing the behavior at the object level. These examples can be run to verify that the implementation of the system is still working as the developer expects while providing documentation about how the objects are expected to behave.This is an important distinction compared to unit-testing frameworks such as NUnit and MbUnit. Both of those use test attributes to indicate a method is a test for the system. RSpec takes a different approach. RSpec says that each method is an example of how the code is intended to work. While the distinction is subtle, it changes the way you write the examples, including the terminology you use, the way you organize the methods, and how easily the concept can be understood compared to TDD. With BDD and RSpec, together with the close integration of IronRuby and the .NET Framework, you can start using IronRuby to test C# applications. In terms of an example, the classic RSpec example is the Bowling game. First, you need to access the bowling game’s implementation in a C# assembly: require File.dirname(__FILE__) + \ ‘/InteropSamples/Bowling/Bowling/bin/Debug/bowling.dll’You then need to access RSpec:require ‘rubygems’require ‘spec’Now you can begin writing examples. This is an example showing how bowling implementation should work. RSpec has a Domain Specific Language (DSL) that you must follow for the examples to be executable. The first part of the DSL is the describe block. Here you simply state the object you are “describing” together with an optional description. In this case, I define the Bowling object, which will be implemented in C#:describe Bowling, ” defines the bowling game” doTo improve readability, any setup will be performed within a before block: before(:each) do @bowling = Bowling.new EndI am now in a position to interact with the object, create the example and verify that it works correctly. I use the “it” block, providing a string stating the context of the example and what it is demonstrating. I then write the section of code that interacts with the system and verifies the correct action has happened: it “should score 0 for a gutter game” do 20.times { @bowling.hit(0) } @bowling.score.should == 0 endendThe C# implementation simply looks like this:public class Bowling { public int Score { get; set; } public void Hit(int pins) { Score += pins; }}Finally, execute this to verify that the bowling example works as expected with RSpec testing C# objects: >ir bowling_spec.rb.Finished in 1.0458315 seconds1 example, 0 failuresIf I wanted a more detailed report, I could select the format to be specdoc. This outputs the object description together with all the examples, stating if they have passed or not:>ir bowling_spec.rb –format specdocBowling defines the bowling game- should score 0 for a gutter gameFinished in 1.43728 seconds1 example, 0 failuresAt the time of writing, the IronRuby team is not regularly shipping binaries. The team has said it is waiting until it is happy with the implementation, compatibility, and performance before releasing official binaries. However, because the source code is freely available, you can download the code and build it yourself. To download the source code, you will need to have a Git source control client, such as msysgit, installed. The IronRuby source control is available online. If you would like more information then I recommend you visit the IronRuby project site or my blog post “Downloading IronRuby from GitHub.” Once you have downloaded the source code, you can compile the assembly either with Visual Studio using the IronRuby.sln solution file or, if you have MRI installed, then you can use the command: Once you have IronRuby compiled, you must download various libraries, such as RSpec, to gain the extra functionality. Ruby has a concept of RubyGems where the gems are packages you can download to gain the functionality and the additional dependencies. To download RSpec, type the following at a command prompt:You would now be able to access the RSpec library from within your Ruby code. However, at the time of writing, RSpec does not work with IronRuby due to one or two bugs. Hopefully by the time this article is published RSpec should be working with IronRuby. To find out the status of the RSpec support, see the RSpec Web site or the IronRuby mailing list. If the bug hasn’t been fixed, I have made a binary with the bug fixed, together with the RSpec library in place. (Note: don’t use this version once the team has fixed the problem.)Once you have the correct binaries in place, ir.exe is the IronRuby interpreter that can execute your code. If you simply launch the app from the command line, you’ll enter the interactive console. In Ruby, code is executed on a line-by-line basis, excellent for learning and debugging, but also for running short commands to solve daily problems. If you provide a file name as a parameter, then the file is executed with the results output to the console.Moving ForwardIn my next article, I will introduce the concept of acceptance testing and how it can improve communication between customer and developer. I will demonstrate how acceptance testing can be automated using IronRuby and RSpec to verify .NET applications and create an executable specification for the system.
Ben Hall is a C# developer/tester with a strong passion for software development and loves writing code. Ben works at Red Gate Software in the U.K. as a Test Engineer and enjoys exploring different ways of testing software, including both manual and automated testing, focusing on the best ways to test different types of applications. Ben is a C# MVP and maintains a blog at Blog.BenHall.me.uk.

View the original article here

Smart Client: Building Distributed Apps with NHibernate and Rhino Service Bus

james — Fri, 14 Oct 2011 02:35:39 +0000

For a long time, I dealt almost exclusively in Web applications. When I moved over to build a smart client application, at first I was at quite a loss as to how to approach building such an application. How do I handle data access? How do I communicate between the smart client application and the server?

Furthermore, I already had a deep investment in an existing toolset that drastically reduced the time and cost for development, and I really wanted to be able to continue using those tools. It took me a while to figure out the details to my satisfaction, and during that time, I kept thinking how much simpler a Web app would be—if only because I knew how to handle such apps already.

There are advantages and disadvantages to smart client applications. On the plus side, smart clients are responsive and promote interactivity with the user. You also reduce server load by moving processing to a client machine, and enable users to work even while disconnected from back-end systems.

On the other hand, there are the challenges inherent in such smart clients, including contending with the speed, security, and bandwidth limitations of data access over the intranet or Internet. You’re also responsible for synchronizing data between front-end and back-end systems, distributed change-tracking, and handling the issues of working in an occasionally connected environment.

A smart client application, as discussed in this article, can be built with either Windows Presentation Foundation (WPF) or Silverlight. Because Silverlight exposes a subset of WPF features, the techniques and approaches I outline here are applicable to both.

In this article, I start the processes of planning and building a smart client application using NHibernate for data access and Rhino Service Bus for reliable communication with the server. The application will function as the front end for an online lending library, which I called Alexandra. The application itself is split into two major pieces. First, there’s an application server running a set of services (where most of the business logic will reside), accessing the database using NHibernate. Second, the smart client UI will make exposing those services to the user easy.

NHibernate is an object-relational mapping (O/RM) framework designed to make it as easy to work with relational databases as it is to work with in-memory data. Rhino Service Bus is an open source service bus implementation built on the Microsoft .NET Framework, focusing primarily on ease of development, deployment and use.

The first task in building the lending library is to decide on the proper distribution of responsibility between the front-end and back-end systems. One path is to focus the application primarily on the UI so that most of the processing is done on the client machine. In this case the back end serves mostly as a data repository.

In essence, this is just a repetition of the traditional client/server application, with the back end serving as a mere proxy for the data store. This is a valid design choice if the back-end system is just a data repository. A personal book catalog, for example, might benefit from such architecture, because the behavior of the application is limited to managing data for the users, with no manipulation of the data on the server side.

For such applications, I recommend making use of WCF RIA Services or WCF Data Services.If you want the back-end server to expose a CRUD interface for the outside world, then leveraging WCF RIA Services or WCF Data Services allows you to drastically cut down the time required to build the application. But while both technologies let you add your own business logic to the CRUD interface, any attempt to implement significant application behavior using this approach would likely result in an unmaintainable, brittle mess.

I won’t cover building such an application in this article, but Brad Adams has shown a step-by-step approach for building just such an application using NHibernate and WCF RIA Services on his blog at blogs.msdn.com/brada/archive/2009/08/06/business-apps-example-for-silverlight-3-rtm-and-net-ria-services-july-update-part-nhibernate.aspx.

Going all the way to the other extreme, you can choose to implement most of the application behavior on the back end, leaving the front end with purely presentation concerns. While this seems reasonable at first, because this is how you typically write Web-based applications, it means that you can’t take advantage of running a real application on the client side. State management would be harder. Essentially you’re back writing a Web application, with all the complexities this entails. You won’t be able to shift processing to the client machine and you won’t be able to handle interruptions in connectivity.

Worse, from the user perspective, this approach means that you present a more sluggish UI since all actions require a roundtrip to the server.

I’m sure it won’t surprise you that the approach I’m taking in this example is somewhere in the middle. I’m going to take advantage of the possibilities offered by running on the client machine, but at the same time significant parts of the application run as services on the back end, as shown in Figure 1.

Figure 1 The Application’s Architecture

The sample solution is composed of three projects, which you can download from github.com/ayende/alexandria. Alexandria.Backend is a console application that hosts the back-end code. Alexandria.Client contains the front-end code, and Alexandria.Messages contains the message definitions shared between them. To run the sample, both Alexandria.Backend and Alexandria.Client need to be running.

One advantage of hosting the back end in a console application is that it allows you to easily simulate disconnected scenarios by simply shutting down the back-end console application and starting it up at a later time.

With the architectural basics in hand, let’s take a look at the implications of writing a smart client application. Communication with the back end is going to be through an intranet or the Internet. Considering the fact that the main source for remote calls in most Web applications is a database or another application server located in the same datacenter (and often in the same rack), this is a drastic change with several implications.

Intranet and Internet connections suffer from issues of speed, bandwidth limitations and security. The vast difference in the costs of communication dictate a different communication structure than the one you’d adopt if all the major pieces in the application were residing in the same datacenter.

Among the biggest hurdles you have to deal with in distributed applications are the fallacies of distributed computing. These are a set of assumptions that developers tend to make when building distributed applications, which ultimately prove false. Relying on these false assumptions usually results in reduced capabilities or a very high cost to redesign and rebuild the system. There are eight fallacies:

The network is reliable.Latency is zero.Bandwidth is infinite.The network is secure.Topology doesn’t change.There is one administrator.Transport cost is zero.The network is homogeneous.

Any distributed application that doesn’t take these fallacies into account is going to run into sever problems. A smart client application needs to deal with those issues head on. The use of caching is a topic of great importance in such circumstances. Even if you aren’t interested in working in a disconnected fashion, a cache is almost always useful for increasing application responsiveness.

Another aspect you need to consider is the communication model for the application. It may seem that the simplest model is a standard service proxy that allows you to perform remote procedure calls (RPCs), but this tends to cause problems down the road. It leads to more-complex code to handle a disconnected state and requires you to explicitly handle asynchronous calls if you want to avoid blocking in the UI thread.

Next, there’s the problem of how to structure the back end of the application in a way that provides both good performance and a degree of separation from the way the UI is structured.

The ideal scenario from a performance and responsiveness perspective is to make a single call to the back end to get all the data you need for the presented screen. The problem with going this route is that you end up with a service interface that mimics the smart client UI exactly. This is bad for a whole host of reasons. ?Mainly, the UI is the most changeable part in an application. Tying the service interface to the UI in this fashion results in frequent changes to the service, driven by purely UI changes.

That, in turn, means deployment of the application just got a lot harder. You have to deploy both the front end and the back end at the same time, and trying to support multiple versions at the same time is likely to result in greater complexity. In addition, the service interface can’t be used to build additional UIs or as an integration point for third-party or additional services.

If you try going the other route—building a standard, fine-grained interface—you’ll run head on into the fallacies (a fine-grained interface leads to a high number of remote calls, resulting in issues with latency, reliability and bandwidth).

The answer to this challenge is to break away from the common RPC model. Instead of exposing methods to be called remotely, let’s use a local cache and a message-oriented communication model.

Figure 2 shows how you pack several requests from the front end to the back end. This allows you to make a single remote call, but keep a programming model on the server side that isn’t tightly coupled to the needs of the UI.

Figure 2 A Single Request to the Server Contains Several Messages

To increase responsiveness, you can include a local cache that can answer some queries immediately, leading to a more-responsive application.

One of the things you have to consider in these scenarios is what types of data you have and the freshness requirements for any data you display. In the Alexandria application, I lean heavily on the local cache because it is acceptable to show the user cached data while the application requests fresh data from the back-end system. Other applications—stock trading, for example—should probably show nothing at all rather than stale data.

The next problem you have to face is handling disconnected scenarios. In many applications, you can specify that a connection is mandatory, which means you can simply show the user an error if the back-end servers are unavailable. But one benefit of a smart client application is that it canwork in a disconnected manner, and the Alexandria application takes full advantage of that.

However, this means the cache becomes even more important because it’s used both to speed communication and to serve data from the cache if the back-end system is unreachable.

By now, I believe you have a good understanding of the challenges involved in building such an application, so let’s move on to see how to solve those challenges.

In Alexandria, there’s no RPC communication between the front end and the back end. Instead, as shown in Figure 3, all communication is handled via one-way messages going through queues.

Figure 3 The Alexandria Communication Model

Queues provide a rather elegant way of solving the communication issues identified earlier. Instead of communicating directly between the front end and the back end (which means supporting disconnected scenarios is hard), you can let the queuing subsystem handle all of that.

Using queues is quite simple. You ask your local queuing subsystem to send a message to some queue. The queuing subsystem takes ownership of the message and ensures that it reaches its destination at some point. Your application, however, doesn’t wait for the message to reach its destination and can carry on doing its work.

If the destination queue is not currently available, the queuing subsystem will wait until the destination queue becomes available again, then deliver the message. The queuing subsystem usually persists the message to disk until it’s delivered, so pending messages will still arrive at their destination even if the source machine has been restarted.

When using queues, it’s easy to think in terms of messages and destinations. A message arriving at a back-end system will trigger some action, which may then result in a reply sent to the original sender. Note that there’s no blocking on either end, because each system is completely independent.

Queuing subsystems include MSMQ, ActiveMQ, RabbitMQ, and others. The Alexandria application uses Rhino Queues (github.com/rhino-queues/rhino-queues), an open source, xcopy-deployed queuing subsystem. I chose Rhino Queues for the simple reason that it requires no installation or administration, making it ideal for use in samples and in applications that you need to deploy to many machines. It’s also worth noting that I wrote Rhino Queues. I hope you like it.

Let’s see how you can handle getting the data for the main screen using queues. Here’s the ApplicationModel initialization routine:

protected override void OnInitialize() { bus.Send( new MyBooksQuery { UserId = userId }, new MyQueueQuery { UserId = userId }, new MyRecommendationsQuery { UserId = userId }, new SubscriptionDetailsQuery { UserId = userId });}

I’m sending a batch of messages to the server, requesting several pieces of information. There are a number of things to notice here. The granularity of the messages sent is high. Rather than sending a single, general message such as MainWindowQuery, I send many messages, (MyBooksQuery, MyQueueQuery, and so on), each for a very specific piece of information. As discussed previously, this allows you to benefit both from sending several messages in a single batch (reducing network roundtrips) and reducing the coupling between the front end and the back end.

One of the most common mistakes in building a distributed application is to ignore the distribution aspect of the application. WCF, for example, makes it easy to ignore the fact that you’re making a method call over the network. While that’s a very simple programming model, it means you need to be extremely careful not to violate one of the fallacies of distributed computing.

Indeed, it’s the very fact that the programming model offered by frameworks such as WCF is so similar to the one you use for calling methods on the local machine that leads you to make those false assumptions.

A standard RPC API means blocking when making a call over the network, higher cost for each remote method call and the potential for failure if the back-end server is not available. It’s certainly possible to build a good distributed application on top of this foundation, but it takes greater care.

Taking a different approach leads you to a programming model based on explicit message exchanges (as opposed to the implicit message exchanges common in most SOAP-based RPC stacks). That model may look strange at first, and it does require you to shift your thinking a bit, but it turns out that by making this shift, you significantly reduce the amount of complexity to worry about overall.

My example Alexandria application is built on top of a one-way messaging platform, and it makes full use of this platform so the application is aware of the fact it’s distributed and actually takes advantage of that.

Note that all of the messages end with the term Query. This is a simple convention I use to denote pure query messages that change no state and expect some sort of response.

Finally, notice that I don’t seem to be getting any sort of reply from the server. Because I’m using queues, the mode of communication is fire and forget. I fire off a message (or a batch of messages) now, and I deal with the replies at a later stage.

Before looking at how the front end deals with the responses, let’s see how the back end processes the messages I just sent. Figure 4 shows how the back-end server consumes a query for books. And here, for the first time, you can see how I use both NHibernate and Rhino Service Bus.

Figure 4 Consuming a Query on the Back-End System

public class MyBooksQueryConsumer : ConsumerOf { private readonly ISession session; private readonly IServiceBus bus; public MyBooksQueryConsumer( ISession session, IServiceBus bus) { this.session = session; this.bus = bus; } public void Consume(MyBooksQuery message) { var user = session.Get(message.UserId); Console.WriteLine(“{0}’s has {1} books at home”, user.Name, user.CurrentlyReading.Count); bus.Reply(new MyBooksResponse { UserId = message.UserId, Timestamp = DateTime.Now, Books = user.CurrentlyReading.ToBookDtoArray() }); }}

But before diving into the actual code that handles this message, let’s discuss the structure in which this code is running.

Rhino Service Bus (hibernatingrhinos.com/open-source/rhino-service-bus) is, unsurprisingly, a service bus implementation. It’s a communication framework based on a one-way queued message exchange, heavily inspired by NServiceBus (nservicebus.com).

A message sent on the bus will arrive at its destination queue, where a message consumer will be invoked. That message consumer in Figure 4 is MyBooksQueryConsumer. A message consumer is a class that implements ConsumerOf, and the Consume method is invoked with the appropriate message instance to handle the message.

You can probably surmise from the MyBooksQueryConsumer constructor that I’m using an Inversion of Control (IoC) container to supply dependencies for the message consumer. In the case of MyBooksQueryConsumer, those dependencies are the bus itself and the NHibernate session.

The actual code for consuming the message is straightforward. You get the appropriate user from the NHibernate session and send a reply back to the message originator with the requested data.

The front end also has a message consumer. This consumer is for MyBooksResponse:

public class MyBooksResponseConsumer : ConsumerOf { private readonly ApplicationModel applicationModel; public MyBooksResponseConsumer( ApplicationModel applicationModel) { this.applicationModel = applicationModel; } public void Consume(MyBooksResponse message) { applicationModel.MyBooks.UpdateFrom(message.Books); }}

This simply updates the application model with the data from the message. One thing, however, should be noted: the consume method is notcalled on the UI thread. Instead, it’s called on a background thread. The application model is bound to the UI, however, so updating it must happen on the UI thread. The UpdateFrom method is aware of that and will switch to the UI thread to update the application model in the correct thread.

The code for handling the other messages on both the back end and the front end is similar. This communication is purely asynchronous. At no point are you waiting for a reply from the back end, and you aren’t using the .NET Framework’s asynchronous API. Instead, you have an explicit message exchange, which usually happens almost instantaneously, but can also stretch over a longer time period if you’re working in a disconnected mode.

Earlier, when I sent the queries to the back end, I just told the bus to send the messages, but I didn’t say where to send them. In Figure 4, I just called Reply, again not specifying where the message should be sent. How does the bus know where to send those messages?

In the case of sending messages to the back end, the answer is: configuration. In the App.config, you’ll find the following configuration:

This tells the bus that all messages whose namespace starts with Alexandria.Messages should be sent to the alexandria_backend endpoint.

In the handling of the messages in the back-end system, calling Reply simply means sending the message back to its originator.

This configuration specifies the ownership of a message, that is, to whom to send this message when it’s placed on the bus and where to send a subscription request so you’ll be included in the distribution list when messages of this type are published. I’m not using message publication in the Alexandria application, so I won’t cover that.

You’ve seen how the communication mechanism works now, but there are infrastructure concerns to address before moving forward. As in any NHibernate application, you need some way of managing the session lifecycle and handling transactions properly.

The standard approach for Web applications is to create a session per request, so each request has its own session. For a messaging application, the behavior is almost identical. Instead of having a session per request, you have a session per message batch.

It turns out that this is handled almost completely by the infrastructure. Figure 5 shows the initialization code for the back-end system.

Figure 5 Initializing Messaging Sessions

public class AlexandriaBootStrapper : AbstractBootStrapper { public AlexandriaBootStrapper() { NHibernateProfiler.Initialize(); } protected override void ConfigureContainer() { var cfg = new Configuration() .Configure(“nhibernate.config”); var sessionFactory = cfg.BuildSessionFactory(); container.Kernel.AddFacility( “factory”, new FactorySupportFacility()); container.Register( Component.For() .Instance(sessionFactory), Component.For() .ImplementedBy(), Component.For() .UsingFactoryMethod(() => NHibernateMessageModule.CurrentSession) .LifeStyle.Is(LifestyleType.Transient)); base.ConfigureContainer(); }}

Bootstrapping is an explicit concept in Rhino Service Bus, implemented by classes deriving from AbstractBootStrapper. The bootstrapper has the same job as the Global.asax in a typical Web application. In Figure 5 I first build the NHibernate session factory, then set up the container (Castle Windsor) to provide the NHibernate session from the NHibenrateMessageModule.

A message module has the same purpose as an HTTP module in a Web application: to handle cross-cutting concerns across all requests. I use the NHibernateMessageModule to manage the session lifetime, as shown in Figure 6.

Figure 6 Managing Session Lifetime

public class NHibernateMessageModule : IMessageModule { private readonly ISessionFactory sessionFactory; [ThreadStatic] private static ISession currentSession; public static ISession CurrentSession { get { return currentSession; } } public NHibernateMessageModule( ISessionFactory sessionFactory) { this.sessionFactory = sessionFactory; } public void Init(ITransport transport, IServiceBus serviceBus) { transport.MessageArrived += TransportOnMessageArrived; transport.MessageProcessingCompleted += TransportOnMessageProcessingCompleted; } private static void TransportOnMessageProcessingCompleted( CurrentMessageInformation currentMessageInformation, Exception exception) { if (currentSession != null) currentSession.Dispose(); currentSession = null; } private bool TransportOnMessageArrived( CurrentMessageInformation currentMessageInformation) { if (currentSession == null) currentSession = sessionFactory.OpenSession(); return false; }}

The code is pretty simple: register for the appropriate events, create and dispose of the session in the appropriate places and you’re done.

One interesting implication of this approach is that all messages in a batch will share the same session, which means that in many cases you can take advantage of NHibernate’s first-level cache.

That’s it for session management, but what about transactions?

A best practice for NHibernate is that all interactions with the database should be handled via transactions. But I’m not using NHibernate’s transactions here. Why?

The answer is that transactions are handled by Rhino Service Bus. Instead of making each consumer manage its own transactions, Rhino Service Bus takes a different approach. It makes use of System.Transactions.TransactionScope to create a single transaction that encompasses all the consumers for messages in the batch.

That means all the actions taken in a response to a message batch (as opposed to a single message) are part of the same transaction. NHibernate will automatically enlist a session in the ambient transaction, so when you’re using Rhino Service Bus you have no need to explicitly deal with transactions.

The combination of a single session and a single transaction makes it easy to combine multiple operations into a single transactional unit. It also means you can directly benefit from NHibernate’s first-level cache. For example, here’s the relevant code to handle MyQueueQuery:

public void Consume(MyQueueQuery message) { var user = session.Get(message.UserId); Console.WriteLine(“{0}’s has {1} books queued for reading”, user.Name, user.Queue.Count); bus.Reply(new MyQueueResponse { UserId = message.UserId, Timestamp = DateTime.Now, Queue = user.Queue.ToBookDtoArray() });}

The actual code for handling a MyQueueQuery and MyBooksQuery is nearly identical. So, what’s the performance implication of a single transaction per session for the following code?

bus.Send( new MyBooksQuery { UserId = userId }, new MyQueueQuery { UserId = userId });

At first glance, it looks like it would take four queries to gather all the required information. In MyBookQuery, one query to get the appropriate user and another to load the user’s books. The same appears to be the case in MyQueueQuery: one query to get the user and another to load the user’s queue.

The use of a single session for the entire batch, however, shows that you’re actually using the first-level cache to avoid unnecessary queries, as you can see in the NHibernate Profiler (nhprof.com) output in Figure 7.

Figure 7 The NHibnerate Profiler View of Processing Requests

As it stands, the application won’t throw an error if the back-end server can’t be reached, but it wouldn’t be very useful, either.

The next step in the evolution of this application is to turn it into a real occasionally connected client by introducing a cache that allows the application to continue operating even if the back-end server is not responding. However, I won’t use the traditional caching architecture in which the application code makes explicit calls to the cache. Instead, I’ll apply the cache at the infrastructure level.

Figure 8 shows the sequence of operations when the cache is implemented as part of the messaging infrastructure when you send a single message requesting data about a user’s books.

Figure 8 Using the Cache in Concurrent Messaging Operations

The client sends a MyBooksQuery message. The message is sent on the bus while, at the same time, the cache is queried to see if it has the response for this request. If the cache contains the response for the previous request, the cache immediately causes the cached message to be consumed as if it just arrived on the bus.

The response from the back-end system arrives. The message is consumed normally and is also placed in the cache. On the surface, this approach seems to be complicated, but it results in effective caching behavior and allows you to almost completely ignore caching concerns in the application code. With a persistent cache (one that survives application restarts), you can operate the application completely independently without requiring any data from the back-end server.

Now let’s implement this functionality. I assume a persistent cache (the sample code provides a simple implementation that uses binary serialization to save the values to disk) and define the following conventions:

A message can be cached if it’s part of a request/response message exchange.Both the request and response messages carry the cache key for the message exchange.

The message exchange is defined by an ICacheableQuery interface with a single Key property and an ICacheableResponse interface with Key and Timestamp properties.

To implement this convention, I write a CachingMessageModule that will run on the front end, intercepting incoming and outgoing messages. Figure 9 shows how incoming messages are handled.

Figure 9 Caching Incoming Connections

private bool TransportOnMessageArrived( CurrentMessageInformation currentMessageInformation) { var cachableResponse = currentMessageInformation.Message as ICacheableResponse; if (cachableResponse == null) return false; var alreadyInCache = cache.Get(cachableResponse.Key); if (alreadyInCache == null || alreadyInCache.Timestamp < cachableResponse.Timestamp) { cache.Put(cachableResponse.Key, cachableResponse.Timestamp, cachableResponse); } return false;}

There isn’t much going on here—if the message is a cacheable response, I put it in the cache. But there is one thing to note: I handle the case of out-of-order messages—messages that have an earlier timestamp arriving after messages with later timestamps. This ensures that only the latest information is stored in the cache.

Handling outgoing messages and dispatching the messages from the cache is more interesting, as you can see in Figure 10.

Figure 10 Dispatching Messages

private void TransportOnMessageSent( CurrentMessageInformation currentMessageInformation) { var cacheableQuerys = currentMessageInformation.AllMessages.OfType< ICacheableQuery>(); var responses = from msg in cacheableQuerys let response = cache.Get(msg.Key) where response != null select response.Value; var array = responses.ToArray(); if (array.Length == 0) return; bus.ConsumeMessages(array);}

I gather the cached responses from the cache and call ConsumeMessages on them. That causes the bus to invoke the usual message invocation logic, so it looks like the message has arrived again.

Note, however, that even though there’s a cached response, you still send the message. The reasoning is that you can provide a quick (cached) response for the user, and update the information shown to the user when the back end replies to new messages.

I have covered the basic building blocks of a smart client application: how to structure the back end and the communication mode between the smart client application and the back end. The latter is important because choosing the wrong communication mode can lead to the fallacies of distributed computing. I also touched on batching and caching, two very important approaches to improving the performance of a smart client application.

On the back end, you’ve seen how to manage transactions and the NHibernate session, how to consume and reply to messages from the client and how everything comes together in the bootstrapper.

In this article, I focused primarily on infrastructure concerns; in the next installment I’ll cover best practices for sending data between the back end and the smart client application, and patterns for distributed change management.

Oren Eini (who works under the pseudonym Ayende Rahien) is an active member of several open source projects (NHibernate and Castle among them) and is the founder of many others (Rhino Mocks, NHibernate Query Analyzer and Rhino Commons among them). Eini is also responsible for the NHibernate Profiler (nhprof.com), a visual debugger for NHibernate. You can follow Eini’s work at ayende.com/blog.

View the original article here

Editor’s Note: Best Practices

james — Thu, 13 Oct 2011 19:48:39 +0000

View the original article here

New Stuff: Resources for Your Developer Toolbox

james — Thu, 13 Oct 2011 14:39:39 +0000

View the original article here

Data Points: LINQ Projection Queries and Alternatives in WCF Services

james — Thu, 13 Oct 2011 07:21:39 +0000

The presenter at my local .NET user group was writing a LINQ query during his session last month when I asked him, “How did we ever live without LINQ?” “I have no idea,” he replied.

It’s true. Since it was introduced in Visual Studio 2008, LINQ has made such a difference in how we code in the Microsoft .NET Framework. In combination with the many new language features that were introduced in Visual Basic and C#, it’s a consistent problem solver for querying in-memory objects and data sources.

One of LINQ’s abilities that is both a blessing and an occasional source of frustration is that it can project randomly shaped data into anonymous types. When you simply need to grab a special view of your data, without having to declare a new class for this throwaway type, anonymous types are a great solution. LINQ projections and anonymous types have certainly spoiled us. So why do I say they can also be a source of frustration?

If you have ever used a LINQ projection in a method that needs to return data to another method—or worse, used a LINQ projection in a Windows Communication Foundation (WCF) service operation—you may understand.

Because anonymous types are throwaway types, they have no declaration and are understood only within the method where they’re created. If you write a query that returns a list of anonymous types, there’s no way to define a method ar-gument to say “I’m going to return a list of … ” because there’s no way to express “… of anonymous types.”

Here’s a LINQ to Entities query with a simple projection:

var custQuery = from c in context.Customers select new {c.CustomerID, Name=c.LastName.Trim() + “, ” + c.FirstName};

At run time, the custQuery variable will actually be an ObjectQuery<<>f__AnonymousType0>.

The var (and the alternate use of Visual Basic Dim) allows us to get away with not having (or needing) a way to express this non-type.

If you want to return the results of that query from a method, the only reasonable solution is to create a class to represent the type being returned. Doing this, however, renders the beauty of the anonymous type moot. Now you have to write more code, define classes and (possibly) new projects to house the new classes, ensure the various assemblies using these classes have access to them and so on.

Until recently, data services provided an additional conundrum. In order to project data, you had to create a custom operation in a service, execute your own query and then return some type of pre-defined class that could be understood by the client.

When you’re working with services, there are many scenarios where you want to work with a particular view of data without paying the price of moving larger types across the wire.

It turns out, there are more options besides creating an extra type in your domain to satisfy this temporary need.

The Data Services Update for the .NET Framework 3.5 SP1 introduces a handful of powerful features for WCF Data Services, which are also part of the .NET Framework 4. Among these features is the ability to use projections in queries against the data services. I highly recommend checking out the WCF Data Services team blog post on all that’s new in this update at blogs.msdn.com/astoriateam/archive/2010/01/27/data-services-update-for-net-3-5-sp1-available-for-download.aspx.

The $select operator has been added to the data services URI syntax. It allows for property and even navigation property projection.

Here’s a simple example of a projection that gets a few scalar properties for a customer along with the SalesOrderHeaders navigation property:

http://localhost /DataService.svc/Customers(609) $select=CustomerID,LastName,FirstName,SalesOrderHeaders&$expand= SalesOrderHeaders

The expand operator forces the results to include not just a link to those orders, but the data for each order as well.

Figure 1 shows the results of this query. The expanded SalesOrderHeaders (which contains only a single order) is highlighted in yellow while the customer information is highlighted in green.

Figure 1 Results of a Data Services Query Projection Requesting Three Customer Properties and the Customer’s SalesOrderHeaders

The LINQ to REST feature in the .NET Framework and Silverlight client APIs for WCF Data Services has been updated to allow projections as well:

var projectedCust = (from c in context.Customers where c.CustomerID==609 select new {c.CustomerID, c.LastName}) .FirstOrDefault();

ProjectedCust is now an anonymous type I can use in my client application.

It’s also possible to project into known entity types, and in some cases, the DataContext can keep track of changes made by the client and these changes can be persisted back through the service’s SaveChanges method. Be aware that any missing properties will get populated with their defaults (or null if they’re nullable) and be persisted to the database.

If you’re using an Entity Framework Entity Data Model (EDM), there’s a convenient way to avoid being stuck with anonymous types when you need to pass them out of the method in which they were created.

The EDM has a mapping called QueryView. I’ve pointed many clients to this in the past, prior to data services projection support. Not only does it solve the problem nicely for data services, but for custom WCF Services and RIA Services as well.

What is a QueryView? It’s a special type of mapping in the Entity Framework metadata. Typically, you map properties of an entity to database tables or view columns as they’re described in the store model—Storage Schema Definition Language (SSDL)—of metadata, as shown in Figure 2.

Figure 2 Mapping Table Columns Directly to Entity Properties

A QueryView, however, lets you create a view over those SSDL table columns rather than map directly to them. There are many reasons to use a QueryView. Some examples include: to expose your entities as read-only, to filter entities in a way that conditional mapping does not allow or to provide different views of the data tables from the database.

It’s the last of these purposes that I will focus on as an alternative to the anonymous types you frequently find yourself projecting in your application. One example would be a pick list. Why return an entire customer type for a drop-down that needs only an ID and the customer’s name?

Before creating a QueryView, you need to create an entity in the model that represents the shape of the view you’re aiming for—for example, the CustomerNameAndID entity.

But you can’t map this entity directly to the Customer table in SSDL. Mapping both the Customer entity and the CustomerNameAndID entity to the table’s CustomerID column would create a conflict.

Instead, just as you can create a view of a table in your database, you can create a view of the SSDL Customer directly in the metadata. A QueryView is literally an Entity SQL expression over the SSDL. It’s part of the mapping specification language (MSL) metadata of the model. There is no designer support to create the QueryView, so you’ll need to type it directly in the XML.

Because you’ll be mapping to the store schema of the table, it’s a good idea to see what that looks like. Figure 3 lists the SSDL description of the Customer database table, which looks similar to the Customer entity in the conceptual model’s metadata, except for the use of provider data types.

Figure 3 The SSDL Description of the Database Customer Table

Another important element for the QueryView will be the store schema’s namespace, ModelStoreContainer. Now you have the pieces necessary to construct the QueryView expression. Here’s a QueryView that projects the three required fields from the SSDL into the CustomerNameAndID entity that I created in the model:

SELECT VALUE AWModel.CustomerNameAndID(c.CustomerID, c.FirstName, c.LastName) FROM ModelStoreContainer.Customer as c

Translating the Entity SQL to English: “Query the Customer in the store schema, pull out these three columns and give them back to me as a CustomerNameAndID entity.” AWModel is the namespace of the conceptual model’s entity container. You’re required to use the strongly typed names of both the Conceptual Schema Definition Language (CSDL) and SSDL types that are referenced in the expression.

As long as the results of the projection (an integer, a string and a string) match the schema of the target entity, the mapping will succeed. I’ve tried to use functions and concatenation within the projection—for example, (c.CustomerID, c.FirstName + c.LastName)—but this fails with an error stating that FUNCTIONs are not allowed. So I’m forced to use the FirstName and LastName properties and let the client deal with concatenation.

You must place the QueryView expression within the EntitySetMapping element for the entity that goes inside the EntityContainerMapping in the metadata. Figure 4 shows this QueryView (highlighted in yellow) in the raw XML of my EDMX file.

Figure 4 A QueryView in the Mappings Section

Now my CustomerNameAndID is part of my model and will be available to any consumer. And there is another advantage to the QueryView. Even though the goal of this QueryView is to create a read-only reference list, you can also update entities that are mapped using QueryViews. The context will track changes to CustomerNameAndID objects. Although Entity Framework is not able to auto-generate insert, update and delete commands for this entity, you can map stored procedures to it.

Now that you have the QueryView in the model, you don’t need to depend on projections or anonymous types to retrieve these views of your data. In WCF Data Services, CustomerNameAndIDs becomes a valid entity set to query against, as shown here:

List custPickList = context.CustomerNameAndIDs.ToList();

No messy projections. Better yet, you can create service operations in your custom WCF Services that are now able to return this strongly typed object without having to define new types in your application and project into them.

public List GetCustomerPickList() { using (var context = new AWEntities()) { return context.CustomerNameAndIDs.OrderBy( c => c.LastName).ToList(); } }

Because of the limitation that prevents us from concatenating the first and last names in the QueryView, it’s up to the developers who consume the service to do this concatenation on their end.

WCF RIA Services can also benefit from the QueryView. You may want to expose a method for retrieving a restaurant pick list from your domain service. Rather than having to create an extra class in the domain service to represent the projected properties, this RestaurantPickList entity is backed by a QueryView in the model, which makes it easy to provide this data:

public IQueryable GetRestaurantPickList() { return context.RestaurantPickLists; }

Having the ability to project views over your data types is a huge benefit in querying, and it’s a great addition to WCF Data Services. Even so, there are times when having access to these views, without having to project and without having to worry about sharing the result, will simplify some of your coding tasks.

One last note: With the introduction of foreign keys in the .NET Framework 4 version of Entity Framework, QueryView pick lists make even more sense because you can return read-only entities and simply use their properties to update foreign key properties in the entities you’re editing.

Julie Lerman is a Microsoft MVP, .NET mentor and consultant who lives in the hills of Vermont. You can find her presenting on data access and other Microsoft .NET topics at user groups and conferences around the world. Lerman blogs at thedatafarm.com/blog and is the author of the highly acclaimed book, “Programming Entity Framework” (O’Reilly Media, 2009). Follow her on Twitter: julielerman.

Thanks to the following technical expert for reviewing this article: Alex James

View the original article here

Concurrent Affairs: Solving The Dining Philosophers Problem With Asynchronous Agents

james — Thu, 13 Oct 2011 00:46:39 +0000

Solving The Dining Philosophers Problem With Asynchronous AgentsThis article is based on a prerelease version of Visual C++ 2010. All information is subject to change.Enabling C++ developers to write highly concurrent applications is a major focus of Visual Studio 2010. The beta release includes the Concurrency Runtime, parallel libraries, and development tools aimed at addressing several common problems preventing developers from unlocking the performance potential inherent to multicore hardware. Notably, this includes ensuring that developers can identify and take advantage of opportunities for concurrency in their code, productively manage shared state and its side effects, and not having to worry about building low-overhead concurrency infrastructure that is scalable at run time on a variety of hardware.In this article, I’m going to demonstrate how to use the new Asynchronous Agents Library included as part of Visual C++ 2010 to manage the difficulties that can arise with shared state. To show you how this works, I will walk through an implementation of a classic concurrency problem: Djikstra’s Dining Philosophers. You’ll see how the actor-based programming construct of an agent in combination with asynchronous message-passing APIs can be used to provide a correct and easy to understand solution to this problem that doesn’t rely directly on threading or synchronization primitives.The Dining PhilosophersFor those who aren’t familiar with it, the Dining Philosophers problem is intended to illustrate the complexities of managing shared state in a multithreaded environment. Here’s the problem:At a round table sit five philosophers who alternate between thinking and eating from a large bowl of rice at random intervals. Unfortunately for the philosophers, there are only five chopsticks at the table (see Figure 1), and before the philosophers can begin eating, they must acquire a pair of chopsticks. Since the chopsticks are shared between the philosophers, access to the chopsticks must be protected, but if care isn’t taken, concurrency problems arise. Most notably, starvation (pun intended) via deadlock can occur: if each philosopher picks up a chopstick as soon as it is available and waits indefinitely for the other, eventually they will all end up holding only one chopstick with no chance of acquiring another. Figure 1 The Philosophers and ChopsticksThe Dining Philosophers problem is typically represented in code by a thread for each philosopher and some form of shared state used to represent each of the chopsticks. Straightforward solutions to this problem often involve introducing a waiter entity to coordinate access to the chopsticks, introducing lock ordering heuristics, and manually working with threading APIs, critical sections, semaphores, or monitors to manage the state. Most developers will agree that building correct, nontrivial locking is considered challenging and fragile at best. As a result, most implementations of the Dining Philosophers problem tend to have implementation details leaked into them. For example, the Philosophers may be aware of the synchronization primitives or of their neighbors’ chopsticks. This becomes particularly problematic if you want to generalize the problem to an arbitrary number of philosophers or refocus an implementation on the problem domain—such as what each philosopher does—rather than focusing on the synchronization and threading primitives.An Actor-Based ApproachLet’s walk through a design that is built on top of the Asynchronous Agents Library. Dining Philosophers really only has two moderately difficult sections: creating a thread for each philosopher to run in and coordinating the philosophers’ access to the chopsticks. The Asynchronous Agents Library provides an actor-based programming model and asynchronous message passing APIs, and you’ll need both of these in the implementation. In the Asynchronous Agents Library, the agent class models an actor pattern and provides an asynchronous run method. I use an agent for each philosopher and take advantage of the run method to fulfill the requirement of each philosopher running in its own thread. The second portion of a correct solution involves managing concurrent access to the chopsticks and, in contrast to traditional solutions using semaphores or locks, I’ll use the message passing APIs in the Asynchronous Agents Library to move the chopsticks between the philosophers’ hands and the table. As each philosopher transitions between thinking and eating, he picks up the chopsticks and returns them to the table by sending messages. You’ll see that in addition to providing some abstraction on top of threads and shared state, this will also allow the implementation to stay more focused on the domain of the problem than other solutions enable. A Solution In Five ClassesThe basic design is going to use five types:A Chopstick class that is relatively self explanatory.A ChopstickProvider class that will be used to hold the chopsticks on the table and provide them to the philosophers.A Philosopher class that is responsible for thinking and eating and is aware only of its two ChopstickProviders.A PhilospherState enum that can be thinking or eating. A Table class that contains a set of Philosophers and a set of Chopsticks. Table is responsible for setting the table by placing a single Chopstick in a ChopstickProvider between each Philosopher. Figure 2 shows the relationships between the classes. Figure 2 Classes Used In The Dining Philosophers SolutionLet’s start with the simplest class: Chopstick. The purpose of the Chopstick is to simply exist as a chopstick and be able to identify itself. As a result, the implementation is a simple class that has an identifier member variable, a constructor that initializes the identifier, and a method to return the identifier. class Chopstick{ const std::string m_Id;public: Chopstick(std::string && Id):m_Id(Id){}; const std::string GetID() { return m_Id; };};It’s important to note that the identifier field m_Id is a const member variable. I don’t want the chopstick to have any state on its own that can get changed accidentally by another thread. Also notice that the constructor is using an r-value reference in its parameter list (the &&). This is a new language construct in Visual Studio 2010 that is part of the draft C++0x standard. An r-value reference here allows the compiler to avoid a copy constructor call and move Id into m_Id within the Chopstick instance if Id is a temporary variable or r-value. Message Blocks And MessagesChopstickProvider is also straightforward. It can be thought of as a storage buffer whose purpose is to accept Chopsticks when they are sent to it and to provide Chopsticks to a Philosopher upon request. Here’s the first place where I use the agent APIs. I use what’s known as a message block for the ChopstickProvider. A message block is defined as a class that is able to send and receive messages. More specifically, if you’re able to send a message to message block, the block is referred to as a target and, if you’re able to receive a message from a message block, it’s referred to as a source. In this example, I’ll need the ChopstickProvider to be both a source and a target because the Philosophers will be both receiving chopsticks from the ChopstickProviders when the Philosophers are hungry and sending them back to the ChopstickProviders when the Philosophers are ready to think again. Here’s an example of putting them to work://create a chopstickChopstick chopstick(“chopstick one”);//create a ChopstickProvider to store the chopstickChopstickProvider chopstickBuffer;//put the chopstick in the chopstick holderConcurrency::send(chopstickBuffer, &chopstick);//request a chopstick from the chopstick bufferChopstick* result = Concurrency::receive(chopstickBuffer);You can see that, after declaring a chopstick and a chopstickBuffer (an instance of the still undefined ChopstickProvider), Concurrency::send is used to place the address of chopstick into the ChopstickBuffer and Concurrency::receive returns a pointer to a Chopstick from the chopstickBuffer. Concurrency::send is a template method that synchronously places a message in a message block. Concurrency::receive is a template method that returns a message from a message block. Concurrency::receive blocks until a message is available in the message block. The Asynchronous Agents Library also provides Concurrency::asend and Concurrency::try_receive. Concurrency::asend asynchronously spawns a task to send the message without waiting for acknowledgement of receipt. Concurrency::try_receive is a non-blocking call that will acquire a message if one is available.Now let’s actually define the ChopstickProvider. Remember I said it was straightforward? It’s a typedef:typedef Concurrency::unbounded_buffer ChopstickProvider;The agents library provides a message block called the unbounded_buffer that has the functionality you need. An unbounded_buffer is a template class that is both a source and a target. It stores a memory-limited number of messages in an internal queue. Messages are removed from the queue when they are requested. You won’t need the unbounded quality in this implementation of Dining Philosophers as there is a limited number of chopsticks that will be moving between each chopstick holder. The key functionality you’re interested in from the unbounded_buffer is its move semantics.Agents And The Join Message BlockNow let’s take a look at implementing the Philosopher class. This is the interesting portion of the Dining Philosophers problem, where you ensure that each philosopher is running on its own thread and that access to the shared resource is coordinated appropriately. The Philosopher class will use two additional constructs from the Asynchronous Agents Library, the abstract agent class and the join construct. First I’m going to describe the agent and how it’s used.The agent class is an abstract class that models what is sometimes referred to as an actor pattern or actor-based programming. Agents are intended to be used to facilitate removing dependencies and shared state between active components of an application by having the agents encapsulate state and communicate with each other solely via message passing through a small set of public interfaces. This sounds more complicated than it is, but hopefully you can see the similarities to the Philosopher class. As I mentioned earlier, the Philosopher class needs to be running on its own thread. Translating this to the terminology of agents this means it needs to be an active task, so Philosopher is going to derive publicly from Concurrency::agent. The Philosopher class also needs to be aware of two ChopstickProviders (one for each hand) and then use them to transition successfully between thinking and eating. The Philosopher class isn’t complete yet, but you can see how this is starting to translate into a class synopsis:enum PhilosopherState {Thinking,Eating};typedef Concurrency::unbounded_buffer ChopstickProvider;class Philosopher : public Concurrency::agent { ChopstickProvider* m_LeftChopstickProvider; ChopstickProvider* m_RightChopstickProvider;public: const std::string m_Name; const int m_Bites; Philosopher(const std::string&& name, int bites=500):m_Name(name), m_Bites(bites){};…}Philosopher has a pointer to two ChopstickProviders, a name, the number of bites or turns the Philosopher will take, and a constructor for initialization. What is missing, though, is a way to initialize the ChopstickProviders in the agent. You don’t want to initialize them at construction time because the Philosophers are independent from the Chopsticks and their ChopstickProviders. You could create an additional public method like AssignChopsticks (ChopstickProvider* left, ChopstickProvider* right), but then you would have to take some care—or a lock—to ensure that this method is thread safe. Instead, I’m going to create a public interface so that the ChopstickProviders can be assigned asynchronously. I do this by adding two more public unbounded_buffer member variables and use these to send the Philosophers two ChopstickProviders:Concurrency::unbounded_buffer LeftChopstickProviderBuffer;Concurrency::unbounded_buffer RightChopstickProviderBuffer;Now I’m ready to start implementing the Philosopher method that will run in its own thread, wait for the ChopstickProviders to be initialized, and then start alternating between thinking and eating. I implement the virtual run method from the agent base class. agent::run will be started in its own task when a call to agent::start is made. The first thing I need to do inside run is initialize the ChopstickProviders by waiting for messages to be sent on the public methods with receive://initialize the ChopstickProvidersm_LeftChopstickProvider = Concurrency::receive(LeftChopstickProviderBuffer);m_RightChopstickProvider = Concurrency::receive(RightChopstickProviderBuffer);Now I need to transition between thinking and eating. To do this I’m going to use two additional methods, PickupChopsticks and PutDownChopsticks:for(int i = 0; i < m_Bites;++i) { Think(); std::vector chopsticks(PickupChopsticks()); Eat(); PutDownChopsticks(chopsticks);}The only thing remaining to implement in the run method is to clean up by returning the ChopstickProviders so that they can be reused if needed, and to set the agent status to done.Concurrency::send(LeftChopstickProviderBuffer, m_LeftChopstickProvider);Concurrency::send(RightChopstickProviderBuffer, m_RightChopstickProvider);this->done(Concurrency::agent_done);}Moving on, the additional methods Eat and Think are straightforward. In the original problem they are described as random spin loops:private: void Eat() { RandomSpin(); }; void Think() { RandomSpin(); };};Implementing PickupChopsticks is significantly more interesting because it manages acquiring and releasing the chopsticks without introducing deadlocks or race conditions. To implement this I’m going to use another messaging block from the Asynchronous Agents Library called join. Concurrency::join is a message block that waits for messages from multiple sources, potentially of varying types. Once all messages have been received, it produces a conglomerate message. A join can support sources of the same or multiple types and will acquire messages in a greedy or non-greedy manner. This means that there are four variants of join in the agents library. The Philosopher class will use a single-type non-greedy join. Figure 3 shows a simple example of a join in code.Now I’m ready to implement PickupChopsticks. The first thing to notice is that you return a set of chopsticks by returning a vector. To get a pair of chopsticks from the chopstick provider means using a non-greedy join. The non-greedy variant of join waits for an offer of messages from all linked sources, and then, once it has an offer from each source, confirms that it can actually take ownership of the message. This is what prevents deadlock in this example. If I had used greedy as the template parameter for the join, the message would be taken as soon as an offer is made, and sooner or later each Philosopher would have a single chopstick and they would all be deadlocked. Here’s the code for PickupChopsticks. I create a join, link it to the chopstick providers, and wait for the chopsticks to arrive:std::vector PickupChopsticks() { //create the join Concurrency::join j(2); m_LeftChopstickProvider->link_target(&j); m_RightChopstickProvider->link_target(&j); //pickup the chopsticks return Concurrency::receive (j);}Implementing PutDownChopsticks is straightforward as well. All I need to do is return the chopsticks that I’ve acquired from the vector using the asynchronous method asend:void PutDownChopsticks(std::vector& v) { Concurrency::asend(m_LeftChopstickProvider,v[0]); Concurrency::asend(m_RightChopstickProvider,v[1]);};Testing The Philosopher And Displaying StateThe Philosopher class can be used as is, but you need to understand how to start the Philosopher class and have a way of reporting on its status—whether it is eating or thinking. To start the philosopher, call the agent::start method, which spawns a task that invokes the virtual run method. To report on the status, I introduce two more message blocks from the agents library: Concurrency::overwrite_buffer and Concurrency::call. Concurrency::overwite_buffer is a message block that stores a single value that can be overwritten, and it produces a copy of the value when requested. As with the other message blocks, overwrite_buffer can be linked to additional targets, and message propagation happens in order. I’ll add a public member variable to the Philosopher class that is an overwrite_buffer and update it by sending messages to it as the Philosopher transitions from Thinking to Eating. Specifically, this will involve adding a member variable to the Philosopher class:Concurrency::overwrite_buffer CurrentState;It also means modifying Eat and Think to update status:void Eat() { send(&CurrentState,PhilosopherState::Eating); RandomSpin();};void Think() { send(&CurrentState,PhilosopherState::Thinking); RandomSpin();};Concurrency::call is a message block that is constructed with a functor and spawns a task to execute the functor when it receives the message. You can use the call in combination with the newly defined overwrite_buffer on the Philosopher to report on its state, as shown in Figure 4.And that’s it for the Philosopher class.Implementing The Table ClassYou’ve already seen everything needed to create the Table class. As mentioned previously, it will contain a set of Philosophers, a set of Chopsticks, and a set of ChopstickProviders. The Table will be responsible for seating the Philosophers at the table, placing a ChopstickProvider between each of them, and placing a Chopstick in each of the ChopstickProviders. For flexibility I’ve chosen to make it a template class, and it contains a reference to a list of Philosophers, a vector of Chopsticks, and a vector of Chopstick Holders. templateclass Table { PhilosopherList & m_Philosophers; std::vector m_ChopstickProviders; std::vector m_Chopsticks; …The only public method in the Table class is the constructor, which initializes the vectors and assigns ChopstickProviders to each Philosopher (see Figure 5).Time For Lunch Finally, to implement main, I need to declare the list of Philosophers, create a Table, start the Philosophers, hook up a reporting mechanism, and wait for them to eat (see Figure 6). Hopefully, it’s clear how the agent class and asynchronous message passing that the Asynchronous Agents Library provides can help avoid the difficulties of coordinating access to shared state. I’ve managed to implement the Dining Philosophers problem without using a single explicit lock and without having to call any threading APIs directly. I’ve also managed to keep the implementation within the terms of the domain—specifically, I’ve spent time working with Chopsticks, Philosophers, and Tables as opposed to arrays of integers and semaphores.What I haven’t done yet is added any inherent scalability to the Philosophers. This application as written won’t run significantly faster on any more than four cores. But this can be fixed by replacing RandomSpin with something more meaningful, like an algorithm implemented with task-based parallelism using the Parallel Pattern Library (PPL). I encourage you to read about this on the native concurrency blog , where I’ve got the source code for this article, but where I’ve also added a few fine-grained parallel thinking algorithms implemented to demonstrate composability between the PPL and the agents library. I’ve also used the Concurrency Runtime’s resource management features to manage the quality of service issues that can arise with an abundance of parallel work. When processing resources get scarce, I can ensure that the Philosophers still are able to get enough to eat.

Rick Molloy is a Program Manager on the Parallel Computing Platform team at Microsoft.

View the original article here

DCOM Interop: Generate Custom Managed C++ Wrappers for Easier COM Interoperation Using DCOMSuds

james — Wed, 12 Oct 2011 18:41:39 +0000

View the original article here

Editor’s Note: I Am The Business

james — Wed, 12 Oct 2011 10:50:39 +0000

Here is the same content in en-us.

I’m writing this month’s editor’s note while at an open spaces conference in Austin, Texas. You may remember the same conference last year that kicked off a firestorm in the blogosphere. Many of the participants from that conference, which was called ALT .NET, have returned to Austin this year to talk about continuous improvement in software development.Unlike last year, where the focus was on sharing techniques related to design and code, this year’s meeting is focused on the methods and philosophies behind measuring and managing development projects. There is quite a bit of interest in incorporating ideas and methods from other disciplines, such as manufacturing and engineering, and there is a growing interest in trying to understand development efforts from a business perspective. I’ve heard terms like ROI used here so often, I had to walk outside and double-check the sign to make sure I was at the right conference.While all of this discussion is beneficial for us as a community, what has been most meaningful for me is the realization of just how dramatically my own perspectives have changed. A year ago, I remember lamenting the fact that my day-to-day job seemed to be moving farther away from the technology—away from everything with which I had experience and confidence, and away from the people with whom I shared so many common stories and frustrations.However, as I sat in the back of the room last night listening to the discussion about techniques, successes, and frustrations related to the friction between developers and the business, I realized something for the first time: I am the business. And I’m OK with that.You may be surprised that it took me a year to come to such a simple conclusion, but it’s hard to let go of such a large component of one’s identity. In my job with MSDN Magazine, I have two main priorities: strategy and management (both budget and people). Part of our strategy includes software development projects, to be used internally and externally. And those projects are usually funded out of my budget—so from my perspective, characteristics like maintainability are not end goals worthy of funding. This is not to say that maintainability is never worth funding—just that it has to be driven by larger, more measurable business goals. Speaking of maintainability, when thinking of such good practices, my sense is that we tend to elevate them to the level of end goals because we understand the problems they solve, and we are not nearly as comfortable with the overall business context. Therefore, when we don’t understand the nature of the business that our software will support, our natural tendency is to generalize the business and focus on the non-functional “goodness” of the system. This creates an immediate break between the development group and the business that the software is supposed to be helping, and it ultimately leads to the failure of so many projects.The simple fact that developers are meeting to talk about things such as ROI is a great step to start addressing this pattern. I also think there are some creative methods that can be adapted from other industries and applied to software projects. For example, I’m considering having my developers actually do the job of my users in lieu of a formal “requirements interview.” This may help to avoid some assumption-making in the requirements-gathering process.In the end, I think it takes both successful projects and failures to mature as a profession. That said, I’m grateful that there are developers thinking proactively about moving us along with more wins than losses.

Thanks to the following Microsoft technical experts for their help with this issue: Mukundan Agaram, Spotty Bowles, Paul Despe, Elisa Flasko, Buck Hodges, Bryce Jonasson, Irena Kennedy, Ravi Krishnamurthy, Sesha Mani, Beth Massi, Dwayne Need, Michael Puleio, Karl Reinsch, Gerhard Schneider, Cliff Simpkins, and Don Smith.

Receive the MSDN Flash e-mail newsletter every other week, with news and information personalized to your interests and areas of focus.

View the original article here