Trying Out Persistence Ignorance with LINQ (Part II)

Wednesday, August 29, 2007 by scott
2 comments

In Part I, we defined an IUnitOfWork interface to avoid coding directly to the System.Data.Linq.DataContext class. IUnitOfWork gives us the opportunity to manipulate IDataSource<T> objects.

IDataSource<T> is another abstraction. When the software is running in earnest, we need to back an IDataSource<T> with a table in a SQL Server database. However, during unit testing we might prefer to have an IDataSource<T> backed by an in-memory data structure. Essentially, we want the ability to switch between a System.Data.Linq.Table<T> implementation (SQL Server) and a System.Collections.Generic.List<T> implementation (in-memory) - without changing the code and LINQ expressions in the upper layers of software.

Fortunately, a handful of interfaces define every Table<T> object. If we implement the same interfaces, we can walk and talk just like a real Table<T>.

public interface IDataSource<T> : IQueryable<T>, IEnumerable<T>,
                                 
ITable, IQueryProvider
{

}

Implementing a persistent data source, one that talks to SQL Server, is as simple as forwarding the calls to an underlying Table<T> implementation.

class PersistentDataSource<T> : IDataSource<T> where T: class
{
    
Table<T> _table = null;
    
DataContext _context = null;
  
    
public PersistentDataSource(DataContext context)
    {
        
Check.ArgIsNotNull(context, "context");

        _context = context;
        _table = context.GetTable<T>();

        
Check.IsNotNull(_table, "Could not retrieve table for "
                                +
typeof(T).ToString());
    }

    
void ITable.Add(object o)
    {
        ((
ITable)_table).Add(o);
    }

    
public IQueryable CreateQuery(Expression expression)
    {
        
return ((IQueryProvider)_table).CreateQuery(expression);
    }
    

    
// ...

Implementing an in-memory data source is a little bit trickier, but appears possible thanks to extension methods like AsQueryable on the System.Linq.Queryable class.

class InMemoryDataStore<T> : IDataSource<T>
{
    
List<T> _list = new List<T>();

    
public void Add(object entity)
    {
        _list.Add((T)entity);
    }

    
public IQueryable CreateQuery(Expression expression)
    {
        
IQueryable queryable = _list.AsQueryable();
        
return queryable.Provider.CreateQuery(expression);
    }
    
    
// ...

Of course, this naïve implementation could never support the full application, but should be suitable for the majority of isolated unit tests.

In the next post, we'll tie everything together to see how all these abstractions work together.

In the meantime, read Rick Strahl's "Dynamic Expression in LINQ to SQL". It is scenarios like the ones that Rick is presenting that make me worry that this approach will fall apart when it hits real requirements.

What’s Wrong With This Code (#17)

Tuesday, August 28, 2007 by scott
16 comments

We interrupt this LINQ series with an emergency!

Well, there is no real emergency, but there hasn’t been a WWWTC for some time, so …

The following program is suppose to compress its own source code into a Program.cs.zip file, then reverse the compression and produce a Program.txt file.

The problem is: Program.txt always shows up as an empty file!

What’s wrong?

using System;
using System.IO;
using System.IO.Compression;

namespace YippyZippy
{
    
class Program
    {
        
static void Main(string[] args)
        {
            Compress(
@"..\..\Program.cs", @"..\..\Program.cs.zip");
            Decompress(
@"..\..\Program.cs.zip", @"..\..\Program.txt");
        }

        
private static void Compress(string inFileName, string outFileName)
        {
            
FileStream inStream = File.Open(inFileName, FileMode.Open);
            
FileStream outStream = File.Open(outFileName, FileMode.Create);
            
GZipStream zipStream = new GZipStream(outStream, CompressionMode.Compress);

            
try
            {
                
byte[] buffer = new byte[inStream.Length];
                inStream.Read(buffer, 0, buffer.Length);
                zipStream.Write(buffer, 0, buffer.Length);
            }

            
finally
            {
                outStream.Close();
                inStream.Close();              
            }
        }

        
private static void Decompress(string inFileName, string outFileName)
        {
            
FileStream input = File.Open(inFileName, FileMode.Open);
            
GZipStream zipStream = new GZipStream(input, CompressionMode.Decompress);
            
FileStream output = File.Open(outFileName, FileMode.Create);

            
try
            {                
                
int data = zipStream.ReadByte();
                
while (data > 0)
                {
                    output.WriteByte((
byte)data);
                    data = zipStream.ReadByte();
                }
            }
            
finally
            {
                output.Close();
                input.Close();
            }
        }
    }
}

Hint: You can fix the program by adding a single line of code.

Trying Out Persistence Ignorance with LINQ (Part I)

Monday, August 27, 2007 by scott
2 comments

In June, Ian Cooper wrote an article (Being Ignorant With LINQ to SQL) that provided marvelous details about using POCOs with persistence ignorant repositories. Ian's conclusion: "LINQ to SQL is usable with a TDD/DDD approach to development. Indeed the ability to swap between LINQ to Objects and LINQ to SQL promises to make much more of the code easily testable via unit tests than before."

I've been tinkering along the same lines as Ian, and although I think I started working at the other end of the problem first, I'm reaching the same conclusions (and ended up with much of the same code).

I started working with a trivial case, thinking if the trivial case becomes difficult, the topic isn't worth pursuing. Let's suppose we have the following class.

public class FoodFact
{
    
public int ID
    {
        
get { return _id; }
        
set { _id = value; }
    }
    
    
public string Name
    {
        
get { return _name; }
        
set { _name = value; }

    }

    
public int Calories
    {
        
get { return _calories; }
        
set { _calories = value; }
    }

    
int _id;
    
int _calories;
    
string _name;
}

The idea is that each FoodFact object will ultimately be persisted as a row in a SQL Server database, but I don't want to clutter up the FoodFact class with unrelated attribute decorations or other database nonsense. To get into the database, I do need some sort of gateway, or class that can track POCO objects and determine if they need persisted. In LINQ, this class is the System.Data.Linq.DataContext class. The DataContext class itself doesn't implement any interesting interfaces, so my plan was to abstract the DataContext in a a stubable, fakeable, mockable fashion.

public interface IUnitOfWork : IDisposable
{
    
IDataSource<T> GetDataSource<T>() where T : class;
    
void SubmitChanges();
}

We have not looked at IDataSource yet, but there will be at least two implementations of IUnitOfWork. One implementation will work against a real database using a persistent data source (where FoodDB is a class derived from DataContext):

public class UnitOfWork : IUnitOfWork
{
    
public UnitOfWork()
    {
        _context =
FoodDB.Create();
    }

    
public IDataSource<T> GetDataSource<T>() where T : class
    {
        
return new PersistentDataSource<T>(_context);
    }
  
    
public void SubmitChanges()
    {
        _context.SubmitChanges();
    }

    
public void Dispose()
    {
        _context.Dispose();
    }

   
FoodDB _context = null;
}

Another implementation will work with an "in memory data source". This class would be defined in a unit test project.

class InMemoryUnitOfWork : IUnitOfWork
{
    
public IDataSource<T> GetDataSource<T>() where T : class
    {
        
return new InMemoryDataStore<T>();
    }
    
    
public void SubmitChanges()
    {
        
    }

    
public void Dispose()
    {
        
    }
}

The goal is to consume IUnitOfWork along these lines:

public class FoodFactRepository
    :
IRepository<FoodFact>
{
    
IDataSource<FoodFact> _dataSource;
    
IUnitOfWork _context;
    
    
public FoodFactRepository(IUnitOfWork context)
    {
        
Check.ArgIsNotNull(context, "context");
        
        _context = context;
        _dataSource = _context.GetDataSource<
FoodFact>();

        
Check.IsNotNull(_dataSource, "Could not retrieve a data source");
    }

    
public FoodFact FindByID(int ID)
    {
        
return
            (
                
from fact in FoodFacts
                
where fact.ID == ID
                
select fact
            ).FirstOrDefault();
    }

    
public FoodFact Add(FoodFact fact)
    {
        
Check.ArgIsNotNull(fact, "fact");

        _dataSource.Add(fact);
        
return fact;
    }

    
public IDataSource<FoodFact> FoodFacts
    {
        
get { return _dataSource; }
    }        

    
// ...

What are these data sources? Stay tuned for part II ...

Handling Faults in Windows Workflow Custom Activities

Friday, August 24, 2007 by scott
0 comments

Here is one I had to track down recently.

The code in the following activity is trivial, but FaultyActivity represents any activity that might throw an exception during Execute and overrides HandleFault (which the WF runtime will schedule to run when it catches the exception tossed up by Execute).

public partial class FaultyActivity: Activity
{
   
public FaultyActivity()
   {
      InitializeComponent();
   }

    
protected override ActivityExecutionStatus Execute(
                        
ActivityExecutionContext Context)
    {
        
// something went wrong
        throw new InvalidOperationException();
    }

    
protected override ActivityExecutionStatus HandleFault(
                        
ActivityExecutionContext context,
                        
Exception exception)
    {
        
// something still went wrong
        throw new InvalidOperationException();
    }
}

The problem is that once an activity enters the Faulting state, it can only transition to Closed or back to Faulting. If another exception escapes during fault handling, the activity transitions back into the Faulting state, and WF schedules HandleFault to execute again - it’s an infinite loop if the exception continues to occur.

Adding some tracing to the app config file..

<system.diagnostics>
    <
switches>
        <
add name="System.Workflow.Runtime" value="All" />
    </
switches>
</
system.diagnostics>

.. confirms the problem (abridged version):

Activity Status Change - Activity: faultyActivity1 Old:Initialized; New:Executing
Scheduling entry: ActivityOperation((1)faultyActivity1, Execute)
Execute of Activity faultyActivity1 threw System.InvalidOperationException
Activity Status Change - Activity: faultyActivity1 Old:Executing; New:Faulting
Scheduling entry: ActivityOperation((1)faultyActivity1, HandleFault)
Compensate of Activity faultyActivity1 threw System.InvalidOperationException
Activity Status Change - Activity: faultyActivity1 Old:Faulting; New:Faulting
Scheduling entry: ActivityOperation((1)faultyActivity1, HandleFault)
Compensate of Activity faultyActivity1 threw System.InvalidOperationException
Activity Status Change - Activity: faultyActivity1 Old:Faulting; New:Faulting
Scheduling entry: ActivityOperation((1)faultyActivity1, HandleFault)
Compensate of Activity faultyActivity1 threw System.InvalidOperationException ...

The sinister aspect to this behavior is that if enough workflows get into the infinite faulting loop, the threadpool becomes starved and the entire application just sits and spins with no work getting done.

Moral of the story – never let an exception escape from HandleFault.

Images That Strike Fear In A Developer's Heart

Thursday, August 23, 2007 by scott
9 comments

Anytime you open up a design tool, and see something like this ...

... you know you'll need stiff drink at the end of the day. It doesn't matter if the boxes represent tables or classes – it's a complicated mess. In this case, the screenshot is from a Visual Studio 2005 Analysis Services project.

Anyone have some aspirin to spare?

What's Wrong With This Well?

Wednesday, August 22, 2007 by scott
2 comments

Water . When you don't have it - you can find out who your true friends are. Your true friends are the ones who open the door when you are standing outside with a towel, toothbrush, razor, and clean pair of underwear in hand.

Last week I turned the tap at the kitchen sink and .... nothing happened. Plumbing is far beyond my area of expertise, so I called for professional help. The plumber poked around and decided everything from the well pump to the kitchen tap was in perfect working order. Like any good debugging session, there was only one possibility left – not enough water left in the well.

There are about 20 million people in the United States drawing water from private wells. These wells can go dry for a number of reasons. A lack of rain is one reason, but there is also overdraft (removing too much water from the ground) and the mysteries of geological change.

I had three options:

  1. Fill the well.
  2. Drill a new well.
  3. Frack the existing well.

In option 1, I call someone who can pour a truckload of water into the well. This solution is viable when there is a severe drought and the water table has dropped below the well. The standard 15 cm diameter well can hold about 1.5 gallons of water per foot of depth. I'd be hoping that heavy rains can once again raise the water table and the well will begin a natural recovery before exhausting the pumped in water. We've been dry, but it didn't sound like this was a long term solution to my problem.

In option 2, I faced a few unknowns. Nobody can tell me how deep someone will need to drill before reaching an aquifer. Wells in my area can range from 100 feet to 900 feet (30m to 270m) in depth. Since a well driller might charge anywhere from $20 to $40 a foot, there is a lot of variability in the cost.

Option 3 is something I hadn't previously heard of. In fact, when someone suggested "fracking" my well, I took it as a euphemism for "screw it - fuggetaboutit - it's dead". As it turns out, "fracking" means "hydro-fracturing" - a technology adopted from the oil and gas industries. The frackers can give you a set cost ($2k - $4k), and fracking works 97% of the time. Fracking is the option I took. Fracking uses high pressure to clean the rock fissures of sediment and create new fractures in the rock, thus allowing more water into the well.

The fracking took about 4 hours. First, the frackers have to remove the pump, hose, and cables from the well. Next, the frackers lower a packer into the well, and inflate the packer (think of a packer as a heavy-duty balloon). The frackers then fill the area under the packer with potable, chlorinated water and add pressure. Tremendous amounts of pressure – typically 1000 to 3000 psi. The frackers continue to add water and raise the pressure until the pressure drops off and stabilizes. For my well, the drop-off happened at 1500 psi. After that point, there is the reinstallation of the pump and the cleanup.

Water flows once again!

Here are two of the links I found interesting during this process.

Wellowner.org – informing consumers about ground water and water wells.

USGS – Ground Water and the Rural Homeowner

by K. Scott Allen K.Scott Allen
My Pluralsight Courses
The Podcast!