.net ORM or OPF authors/users questions

For those using a persistence layer in .NET--

Did you write your own, or adopt one of the dozen or so free or commercial 
frameworks? which? why? What did you look at?

Specific technical question: what approach are you using for object caching? 
If you use a running objects table (graph, list, whatever) in .net, are you 
doing anything to make sure that once the programmer releases an object, it 
isn't resurrected without a database re-fetch?

IOW, in win32 when the refcount of a business object drops to zero, I free 
the object from the active objects list. So recalling the object will force 
a reload. But in .net, even if the list is based on weak references, there 
is always a change that no garbage collection takes place before the 
instance is invoked again, so unless you require a BO to implement 
IDisposable, I don't see how to prevent unintended 'resurrections' from the 
object cache.

bobD 

Bob Dawson a écrit :

> For those using a persistence layer in .NET--
> 
> Did you write your own, or adopt one of the dozen or so free or commercial

> frameworks? which? why? What did you look at?

We are almost done writing our own. Why didn't we use a commercial one? 
Because virtually every one that we looked at was more concerned with 
creating classes to "wrap" tables than with providing a data storage 
mechanism for a proper OO class design.

We now have a mechanism, with which, the only indication that there is 
anything different from a "normal" business class is that we have to 
derive from a BaseObject class. One important difference, for us, is 
that the business classes have no idea about their database identity, in 
fact, nothing outside of the OPF knows about database identity at all, 
there isn't even an "ID" field or property. Hint for you Bob, we 
couldn't have done it without your idea of the data packet :-)

> Specific technical question: what approach are you using for object
caching? 
> If you use a running objects table (graph, list, whatever) in .net, are you

> doing anything to make sure that once the programmer releases an object, it

> isn't resurrected without a database re-fetch?

We are just about to tackle caching, so this discussion couldn't have 
come at a more apposite time.

We are looking at detecting when an object has been edited and then 
marking it as in need of a refetch.

We have concerns about the resources involved in caching large numbers 
of objects, and are researching the best way of detecting when a list of 
objects is no longer being used by any users. We are looking at seeing 
if we can detect when the GC sweeps happen and using that to check for 
unused objects, possibly using weak references; otherwise it will mean 
using a separate thread on a timer to perform our own sweeps. do you 
know anything about this aspect?

> IOW, in win32 when the refcount of a business object drops to zero, I free

> the object from the active objects list. So recalling the object will force

> a reload. But in .net, even if the list is based on weak references, there

> is always a change that no garbage collection takes place before the 
> instance is invoked again, so unless you require a BO to implement 
> IDisposable, I don't see how to prevent unintended 'resurrections' from the

> object cache.

Would the idea of monitoring when an object has been edited and stored 
help with that?

Joanna

-- 
Joanna Carter [TeamB]

"Joanna Carter [TeamB]" wrote
> virtually every one that we looked at was more concerned with
> creating classes to "wrap" tables than with providing a data
storage
> mechanism for a proper OO class design.

Well, there are systems like DeKlarit that will create the db schema for you
from a set of BO definitions (as well as vice-verse--infer a set of BOs from
a schema.) I haven't found anything that acts the way I'd want it to yet.

> there isn't even an "ID" field or property. Hint for you Bob, we
> couldn't have done it without your idea of the data packet :-)

The packet is still at the core of my approach: to be persistent, an object
has to contain (or create) one. That means that the persistence layer only
knows how to persist a single class (but in an infinite variety of ways
<g>).

> We are looking at detecting when an object has been edited and then
> marking it as in need of a refetch.

My packet class maintains a dynamic list of original values, so it knows the
clean/dirty status of each field by whether a reference value for it exists.
Whether that functionality actually belongs there is debateable I suppose:
it could be moved down to the individual packet element, or up to the BO (as
an independent object). But in any case there's no reliance on an edit event
tripping a flag: edit an item multiple times and it may of may not be
dirty--just depends on whether the last edit value equaled the original
value.

Not sure what you mean by editing "marking it as in need of a
refetch."
Perhaps a use case would help

programmer creates Order instance
works with order, as a result of which Order.Items, Order.User, etc. have
all been lazy loaded.
programmer would like to revert to original clean state, so
order = nil;
order = PersistenceFrame.ProvideNew(Order, [indentifier])

except:
 caching allows that if order is in memory already, the framework can
provide a pointer to that instance instead of refetching.
 Even if we assert that no other reference to order exists, we cannot be
sure that a weak reference to order in the cache won't resurrect the
existing instance--the only way to do that would be to force a collection
each time the cache is searched, which would be likely to alter app
performance quite a bit.

> We have concerns about the resources involved in caching large
> numbers of objects,

What I mean by caching is a system to quarantee instance uniqueness: never
more than one copy of object[x] in memory at a time. So resources are not
that big a deal: you just have a single central list of BO pointers. And if
an object is in several collections, it's the same object instance in each.

> if we can detect when the GC sweeps happen

Richter demos a collection detection class in his book. Could be adapted.

> using a separate thread on a timer to perform our own sweeps. do you
> know anything about this aspect?

I don't thing timed sweeps gain anything. Again, I'd refer to Richter's.

> Would the idea of monitoring when an object has been edited and
> stored help with that?

Don't see how. The issue is basically how to walk away from a dirty object
and guarantee a new, re-fetched instance without (1) abandoning caching or
(2) requiring treating BOs as IDisposable (that solves the problem because
the cache can refuse to return a disposed instance).

Only thing I've thought of so far that might work is simply exposing the
problem--something like:

order = PersistenceFrame.ProvideNew(Order, [indentifier], ForceRefetch=True)

What that really does, though, is make caching optional--which wasn't the
intent either

bobD

Bob Dawson a écrit :

> Well, there are systems like DeKlarit that will create the db schema
> for you from a set of BO definitions (as well as vice-verse--infer a
> set of BOs from a schema.) I haven't found anything that acts the way
> I'd want it to yet.

Hmmm, I took a look at DeKlarit and found pretty much what I have
previously found with other systems; although it does a reasonable job
of mapping "proper" class definitions (it didn't require the Id of
the
Invoice in the Invoice Line), I didn't like the naming convention and
also that you had to have an Id property on every class.

> The packet is still at the core of my approach: to be persistent, an
> object has to contain (or create) one. That means that the
> persistence layer only knows how to persist a single class (but in an
> infinite variety of ways <g>).

Yes, we use the packet as the "field list" and, as you do, we also
base
all our transformations to/from SQL by examining that packet.

>> We are looking at detecting when an object has been edited and then
>>  marking it as in need of a refetch.
> 
> My packet class maintains a dynamic list of original values, so it
> knows the clean/dirty status of each field by whether a reference
> value for it exists.

Subtly different, we hold a list of modified property names which is
updated when a given property is changed from its default/retrieved
value. Although I am considering adding a Modified property to each
"field" in the packet.

> Whether that functionality actually belongs there is debateable I
> suppose: it could be moved down to the individual packet element, or
> up to the BO (as an independent object).

We use the INotifyPropertyChanged interface on our BaseObject, so we
really need to know when a value has changed, otherwise the UI controls
don't get updated.

> But in any case there's no reliance on an edit event tripping a flag:
> edit an item multiple times and it may of may not be dirty--just
> depends on whether the last edit value equaled the original value.

We only fire change notifications if the new value is different from the
previous.

> Not sure what you mean by editing "marking it as in need of a
> refetch." Perhaps a use case would help

Hmmm, upon a reread, that wording looks like a brain fart. Now, I am not
sure what I meant ;-)

> programmer creates Order instance works with order, as a result of
> which Order.Items, Order.User, etc. have all been lazy loaded. 
> programmer would like to revert to original clean state, so order =
> nil; order = PersistenceFrame.ProvideNew(Order, [indentifier])

Maybe. But we also implement IEditableObject and so that allows us the
facility of a complete rollback to the state before the BeginEdit method
was called.

> except: caching allows that if order is in memory already, the
> framework can provide a pointer to that instance instead of
> refetching. Even if we assert that no other reference to order
> exists, we cannot be sure that a weak reference to order in the cache
> won't resurrect the existing instance--the only way to do that would
> be to force a collection each time the cache is searched, which would
> be likely to alter app performance quite a bit.

Yes, the pros and cons of various "scanning" strategies is exercising
our minds considerably, but only in a background thread (of our thinking).

> What I mean by caching is a system to quarantee instance uniqueness:
> never more than one copy of object[x] in memory at a time. So
> resources are not that big a deal: you just have a single central
> list of BO pointers. And if an object is in several collections, it's
> the same object instance in each.

Understood and agreed.

> Richter demos a collection detection class in his book. Could be
> adapted.

> I don't thing timed sweeps gain anything. Again, I'd refer to
> Richter's.

Do you have any kind of web reference to this? I really don't want to
buy a book just for the one reference :-)

> Don't see how. The issue is basically how to walk away from a dirty
> object and guarantee a new, re-fetched instance without (1)
> abandoning caching or (2) requiring treating BOs as IDisposable (that
> solves the problem because the cache can refuse to return a disposed
> instance).

What do you see as being the possible benefit of using IDisposable?

> Only thing I've thought of so far that might work is simply exposing
> the problem--something like:
> 
> order = PersistenceFrame.ProvideNew(Order, [indentifier],
> ForceRefetch=True)

I've noticed that you seem to ask for objects based on their identifier; 
that is something that we could not agree with, using instead the idea 
that the only way to retrieve object from the "store" is to submit a 
query with optional conditions. This ties in with our philosophy of not 
exposing an identifier outside of the data storage layer. We would 
always retrieve, even a single item, by means of a query, possibly with 
the condition of a known property value like Code for a Customer or 
Customer.Name and Date for a list of Invoices, as suits the BO in question.

We are also taking a gentle look at using Linq to query our OPF but that 
is a very low priority.

Joanna

-- 
Joanna Carter [TeamB]
Consultant Software Engineer

"Joanna Carter [TeamB]" wrote

> Hmmm, I took a look at DeKlarit

No recommendation, I just pulled it at random from the systems I've looked 
at. I'm sure our criteria are different <g>.

> Subtly different, we hold a list of modified property names which is
> updated when a given property is changed from its default/retrieved
> value. Although I am considering adding a Modified property to each
> "field" in the packet.

I didn't do it at the property level for the same reason I didn't do 
nullability at the property level. In my characteristic domains at least, 
the vast majority of fields in most object instantiations do not get edited 
and are not nullable. SO I think it's both faster and more memory efficient 
to track these as special cases (dynamic lists) rather than as parts of the 
value type. my the thinking was documented in the thread "Design issue: 
Value types and nullability" this forum begun 28 July 2007

> We use the INotifyPropertyChanged interface on our BaseObject, so we
> really need to know when a value has changed, otherwise the UI controls
> don't get updated.

Yes, for UI purposes the eventing is necessary. But that still doesn't tell 
you whether the new value is clean/dirty wrt the database, and that's what I 
want to know on the persistence layer side.

> Maybe. But we also implement IEditableObject and so that allows us the
> facility of a complete rollback to the state before the BeginEdit method
> was called.

I'll keep that in mind--I haven't really started on the forward-facing BO 
design yet.

> Do you have any kind of web reference to this? I really don't want to
> buy a book just for the one reference :-)

On further thought, probably a red herring anyway: Basicly Richter wrote a 
little class that beeps in its finalizer, and creates a new instance of 
itself (which will immediately become unreachable). So every time the GC 
runs, you get a beep. My first thought was simply to replace the beep with a 
callout event that the collection occurred, but on second thought since 
finalization occurs in its own thread, calling back to the framework would 
probably require a thread synchronization--definitely not something you want 
to do from the finalization thread.

Anyway I couldn't find the class online anywhere, but did come up with his 
detailed explanation of how GC and finalization work:
http://msdn.microsoft.com/msdnmag/issues/1100/gci/
https://msdn.microsoft.com/msdnmag/issues/1200/GCI2/

and this interesting piece of code toward the end of the first

public class BaseObj {

    protected override void Finalize() {
        Application.ObjHolder = this;
        GC.ReRegisterForFinalize(this);
    }
}

(this would seem to create an immortal object that would be refinalized on 
every garbage collection.) But the end point is the same: you're still out 
of thread context from your app (all of which's threads have been 
suspended), so there doesn't seem to be a good way to feed the information 
back to the app. Bottom line: I now doubt that immediate GC run notification 
is possible, or ever will be unless MS decides to add it to the GC class.

> What do you see as being the possible benefit of using IDisposable?

Thinking of the cannonical implementation as

    protected virtual void Dispose(bool disposing)

    {
        if (!disposed)
        {
            if (disposing)
            {
                if (handle != null)
                    handle.Dispose();
            }
            disposed = true;
        }
    }

And having the cache retrieval check the Disposed property. If the BO is 
disposed, then remove it from the cache and return nil.

> I've noticed that you seem to ask for objects based on their identifier;

Yes, and I suspect that there is more there than simply an implementation 
detail--it indicates that the goals of our respective persistence layers 
differ. I also give my normal BO's Save methods.

> the condition of a known property value like Code for a Customer or

and if the BO has no natural distinquishing property?

> Customer.Name and Date for a list of Invoices, as suits the BO in 
> question.

Given an invoice, how does it load its owning customer?

bobD