Just about every Java ORM has a notion of "shallow" [vs. "full"] entities.
And just about all of them suck.
Fundamentally, the problem is this: there is no first-class (Java) notion of
what a "shallow" and "full" entity are. Sure, your developers know this, but
very rarely is the notion actually captured within your application's domain
model. If you're lucky, the getter documentation will sometimes mention that a
reference to a related entity may be null under some circumstances, but even
that is a luxury in many codebases.
The ORMs themselves don't make this much easier. Yes, there are ORM-specific
ways of handling the shallow vs. full distinction (and of determining which of
those some object is), but the API for this is usually cumbersome at best. If
you're really unlucky you don't even get a proper API, so you're left dropping
instanceof checks all over the place.
What can we do to make this suck less?
The first pre-condition to this approach working is that you have an immutable model, or at the very least that you not be afraid of creating a little bit of short-lived heap garbage. (If you are concerned, go read up on how modern JVM implementations deal with this. The executive summary is that it's nowhere near the sin it was in the bad old days.)
The second pre-condition is that you have a storage layer that you control. You can still use an ORM, but you have to have a service or DAO layer that all of your persistence calls go through. You'd think this is a given for any application of size, but you'd be surprised...
The third pre-condition is that you have not drunk deep from the kool-aid waters of HATEOAS. HATEOAS actually does actually lend itself towards solving this problem, but it comes with other tradeoffs. Those tradeoffs make it a pretty bad value proposition when you're not shipping a public API with long-running support obligations (i.e. most of the time.)
With the preconditions out of the way, first let's think about the use case.
Our use case goes something like this:
public final interface User
{
public Group group();
public String username();
}
In this simple example, a user is a member of a group. That's all well and
good, but let's imagine for a moment that Group looks like this:
public final interface Group
{
public Group parent();
public String name();
}
Now imagine that groups can be nested to arbitrary depths.
Unfortunately, this isn't such a contrived case: user-defined, nested
hierarchies are both common and annoying to deal with. A common solution is
to make the "parent" object references nullable, but then you can't use null
for any other meaning (such as "this exists at the top level of the hierarchy,
and thus has no parent"). [NB: While this usage of null may be anathema to
some data modellers, it is not at all uncommon in the real world.]
So we can either make the reference nullable at the cost of having to
double-check whether the parent is null due to not existing or simply
because the child was loaded shallowly, OR we can simply load everything all
the time. (The third option that some ORMs like is that all of your model
classes are actually proxies that magically load any outbound references as
needed, but that sort of thing is a special type of hell in which reasoning
about your application's performance becomes borderline impossible. So we'll
ignore it for the rest of this article.)
It gets worse though. Imagine that you have a nice RESTful API for dealing with users and groups. You're serializing to JSON. You want to wind up with:
{
"username" : "example",
"group" : {
"name" : "child-group",
"parent" : {
"name" : "parent-group",
"parent" : undefined
}
}
}
That's fine if you're loading the entire object reference chain, but what if you have a shallowly-loaded user?
{
"username" : "example",
"group" : undefined
}
Oh.
That's not great. Let's say your front-end loads that user, but then needs to figure out which groups they're in. Well... it's got to pass back the user ID or the user object, which means that your API needs to have something like this:
public User loadFull(User user);
This is API bloat, and this is not good.
So what can we do?
To solve this, we need to adopt two interfaces. The first defines how we will identify entities, as well as provides a common interface ancestor for every model entity.
public interface Entity
{
/* UUIDs are nice because you can generate them without asking your
* storage layer, but really any sane ID type will work here.
*/
@NotNull
public UUID id();
}
Now we need to define what makes up a reference.
To be effective for all our loading and serialziation scenarios, our references need to be able to do four things:
That looks like this:
public interface Ref<T extends Entity>
{
@NotNull
public UUID id();
public T entity();
public Ref<T> resolve(Function<UUID, T> resolver);
public Ref<T> unresolve();
}
Now for bonus points, we'll want a few other things that will make our lives easier:
public interface Ref<T extends Entity>
{
@NotNull
public UUID id();
public T entity();
public Ref<T> resolve(Function<UUID, T> resolver);
public Ref<T> unresolve();
/**
* Whether the reference is resolved.
* @return {@code true} if the reference is resolved, {@code false}
* otherwise
*/
public boolean resolved();
/**
* The complement of {@link #resolved()}.
*
* @return {@code true} if the reference is unresolved, {@code false}
* otherwise
*/
public boolean unresolved();
/**
* Determine whether the reference references the given entity.
*
* @return {@code true} if the reference references the given entity,
* {@code false} otherwise
*/
public boolean references(Entity entity);
/**
* Get the class of the referenced entity.
*
* @return referenced entity class, not {@code null}
*/
public Class<? extends Entity> entityClass();
}
Now for the implementation. To implement this, we'll create two concrete reference classes.
public class EntityRef<T extends Entity>
implements Ref<T>
{
private final UUID id;
private final T ref;
private final Class<? extends Entity> cls;
public EntityRef(UUID id, Class<T> cls)
{
this.id = id;
this.ref = null;
this.cls = cls;
}
public EntityRef(T entity)
{
this.id = entity.id();
this.ref = entity;
this.cls = entity.getClass();
}
public UUID
id()
{
return this.id;
}
public T
entity()
{
return this.ref;
}
public boolean
unresolved()
{
return this.ref == null;
}
public boolean
resolved()
{
return ! this.unresolved();
}
public Ref<T>
resolve(final Function<UUID, T> fn)
{
if (! this.unresolved())
{
return this;
}
T t = fn.apply(this.id());
if (t == null)
{
return this;
}
return new EntityRef<>(t);
}
public Ref<T>
unresolve()
{
return new EntityRef<>(this.id(),
(Class<T>) this.entityClass());
}
public Class<? extends Entity>
entityClass()
{
return this.cls;
}
@Override
public int hashCode()
{
/* We only need to consider the ID here, assuming that our ID provides
* a sane hashing implementation. As far as all of our users will be
* concerned, references are interchangeable regardless of whether
* they are resolved (since the caller can always resolve it on their
* own later.)
*/
final int prime = 31;
int result = 1;
result = prime * result + ((this.id == null) ? 0 : this.id.hashCode());
return result;
}
@Override
public boolean equals(Object obj)
{
if (this == obj)
{
return true;
}
if (obj == null)
{
return false;
}
/* A reference may only be compared to other references. */
if (! (obj instanceof Ref))
{
return false;
}
Ref<?> other = (Ref<?>) obj;
if (this.id == null)
{
/* If we don't have an ID, we can only be compared to other
* references that are resolved (via delegation to the referenced
* entity's #equals(Object) implementation)
*/
if (other.id() != null || other.unresolved())
{
return false;
}
return this.entity().equals(other.entity());
}
/* Now, this part is tricky. If you're using universally-unique
* identifiers of some sort like in this example, you can just
* compare that and call it a day. Otherwise, you'll have to take
* into account ID collisions between this and the other reference
* where one is a reference to a class that is a parent of the other.
* This will necessitate imposing your own rules about inheritence and
* equality (common ancestor may or may not be a suitable check), and
* is thus left up to you, the reader.
*/
if (! this.id.equals(other.id()))
{
return false;
}
return true;
}
@Override
public boolean
references(Entity entity)
{
if (entity == null)
{
return false;
}
return this.id().equals(entity.id())
&& this.entityClass().isAssignableFrom(entity.getClass());
}
}
Now so as to make it easier for our domain entities to handle references gracefully, we'll create an abstract reference implementation for them to extend. If object hierarchy prevents it, they're under no obligation to do so. This is just for convenience.
/* Note: you can technically do this as an interface as well, with all of the
* work being done by default implementations, but that is veering dangerously
* close to multiple-inheritance, and thus bad idea territory.
*/
public abstract class DirectRef<T extends Entity>
implements Entity, Ref<T>
{
public T entity()
{
return (T) this;
}
public Ref<T> resolve(Function<UUID, T> resolver)
{
return this;
}
public Ref<T> unresolve()
{
return new EntityRef<>(this);
}
public boolean unresolved()
{
return false;
}
public boolean resolved()
{
return true;
}
public boolean references(Entity entity)
{
/* Wait, what?
*
* If the entity itself can be used as a reference, then by necessity
* #equals(entity) will be true if the entity references itself (or
* if it's the same instance.) This relies on the entity's
* implementation of {@link #equals(Object)} taking the ID into
* account, as well as being logically correct... but if those two
* conditions are met, then this will work just great.
*/
return this.equals(entity);
}
}
Now you'll note that the above implementations allow for references to
entities that do not yet have IDs assigned. This is a judgement call. In our
example, it's reasonable. Depending on your application you may be able to
simplify things by ensuring that every entity has an ID -- but since that is
often dependent on how you're doing your persistence, we won't assume that
here. Instead, in our case, we'll treat a null ID as an indication that an
entity hasn't been persisted yet... but since we'll need to reference
as-of-yet-unpersisted entities in our domain model, our reference API needs to
handle that gracefully.
You'll also note that the above code can take some shortcuts if you're using UUIDs or other universally-unique identifiers. This is an argument in their favor, since otherwise the notion of equality between references may require you to handle references constructed arguably incorrectly (specifically two references to the same object, but one of them typed as a reference to an entity of a type that is an ancestor of the object's type).
Finally, the model above plays nicely with both immutable and mutable domain models, although the latter will require some changes to handle IDs changing behind the reference's back. That's far from the only problem if you have a mutable domain model though, and this reference approach can still be useful in such cases -- just remember that it won't protect you from the usual pitfalls of using a mutable domain model in a concurrent environment.
So that's the model. What does it look like in practice? Well let's go back to our user and group example and change it a bit:
public final interface User
{
/**
* Get the group to which this user belongs.
*
* @return reference to group, not {@code null}
*/
@NotNull
public Ref<Group> group();
/**
* Set what group the user belongs to.
*
* @param group reference to the new group that the user should belong to
* @return copy of the user with the given group set
*/
public User group(Ref<Group> group);
public String username();
}
public final interface Group
{
/**
* Get a reference to the parent group of this group.
*
* @return reference to parent group, or {@code null} if this group has
* no parent
*/
public Ref<Group> parent();
public String name();
}
Now what would a DAO for this look like? Let's sketch out a couple (incomplete) interfaces for our DAO layer:
/**
* Base interface for DAOs. Handles simple persistence for entities of a given
* type.
*
* @type T entity type
*/
public interface DAO<T extends Entity>
{
/**
* Load the entity with the given ID from underlying storage.
*
* @return entity, or {@code null} if no such entity exists
*/
public T load(UUID id);
/**
* Save the given entity, assigning an ID if required.
*
* @return entity reflecting underlying storage
*/
public T save(T entity);
}
Our entity-specific DAO APIs will probably have some other useful methods, but
for now let's just mostly use the ones from DAO:
public interface UserDAO
extends DAO<User>
{
public User loadByUsername(String username);
}
public interface GroupDAO
extends DAO<Group>
{}
So that's all the API changes we have to make. Now, how does this all work together?
First, let's load a user:
/* from wherever you usually get DAO instances... */
UserDAO dao = this.getUserDAO();
User user = dao.loadByUsername("zaphod");
At this point we have a user, and we can use it however we want, without worrying about dragging some group hierarchy along with it. But let's say we want that. Easy!
GroupDAO groupDAO = this.getGroupDAO();
user = user.group(user.group().resolve(groupDAO :: load));
Now at this point, one of two things will happen. If the referenced group was
a DirectRef (because the Group implementation extends DirectRef and
the UserDAO loaded it eagerly), nothing will happen -- you'll just get an
identical copy of the object back. If it was an EntityRef, but was already
resolved (again, if the UserDAO eagerly loaded it), nothing will happen.
Only if the group reference is "shallow" (i.e. an EntityRef with no
associated entity) will the reference be resolved.
Of course the caller doesn't need to know any of this, nor do they have to keep track of what is "shallow" or "full". They just need to say "hey, resolve this if you haven't already" and that's that. (For the performance minded: yes, you will eat the cost of a method invocation, but if that sort of overhead is your concern, you're probably doing something like HFT and are very much not the target audience of this article.)
But what about when you're getting ready to serialize something to send it over the wire? Let's continue our example:
User readyToSerialize = user.group(user.group().unresolve());
That's that. Now we don't have to worry about sending the whole object hierarchy over the wire.
Now that's all well and good, but let's take a look at another more complex usage case. Let's say we need to load some massive set of users.
We could try to write some kitchen-sink DAO method with some massive self-join from hell, and then have our DB admin start drinking heavily.
Or we could rely on our ORM to handle it gracefully... which realistically
will result in going over to the office of the resident Hibernate guru and
begging him to sprinkle on the magic annotations and HSQL necessary to get
results in some time other than O(terrible).
Or we can be lazy. Let's load the users first, then the first level of groups that they belong to.
Set<User> users = this.getUserDAO().loadAll(userIDs);
Set<UUID> groupIDs = users.stream().map(u -> u.group().id())
.collect(Collectors.toList());
Map<UUID, Group> groups = this.getGroupDAO().loadAll(groupIDs);
users = users.stream()
.map(u -> u.group(u.group().resolve(groups :: get)))
.collect(Collectors.toSet());
Now obviously for a single level of the hierarchy this isn't a big win over
the more traditional way of doing things. But you should be able to see how
the reference concept lets you bring some of the hierarchy-traversal work up
into the application layer without too much pain. (Doubly so if your storage
is some NoSQL-type deal where you don't have actual relations...)
Whether or not doing this is a good idea will obviously depend on your
application, storage layer, etc. But the flexibility is certainly there if you
need it -- and it doesn't cost you anything if you don't.
One more example where this is useful: you need to persist a relation, but the thing that you're relating to is expensive to load. This is by far the application of the reference concept that I find most useful on a day-to-day basis.
Typically this happens in response to servicing a request from the UI layer.
Let's say your UI lets an end-user move a User from one Group to another.
The UI knows the ID of the user and of the destination group. It can issue a
request with those two parameters... but what about the server that gets the
request? Well traditionally you'd have to load the user and also the
destination group, do the re-assignment and persist. Depending on what's
above the destination group in the hierachy, this could range from trivial to
super expensive.
Now it doesn't have to:
/* Controller method, with whatever request binding annotations, etc. you
* need.
*/
public User moveUser(UUID userID, UUID destGroupID)
{
User user = this.getUserDAO().load(userID);
/* if paranoid, check that the group exists first... */
user = user.group(new EntityRef<>(destGroupID, Group.class));
return this.getUserDAO().save(user);
}
Move is now O(1) without the need to get clever with an ORM.
So should you use this?
Maybe.
This is a pretty opinionated way to handle object hierarchies re: persistence. As-written it won't work with Hibernate at all, and the concept will undoubtedly seem anathema to the sort of developers who believe that fully-normalized databases and "by the book" ORM-generated query approaches trump all.
On the other hand, I've used this approach in production not infrequently, and there is definitely a niche where it is a useful tool. You don't have to use it for all of your relations, as it's a complete waste for anything that you always need and/or that's always cheap to load. Yet I'd still consider it a useful pattern to keep in the back of your mind when dealing with complex entities with diverse usage patterns.
All code in this example is hereby placed into the public domain. If your country doesn't explicitly acknowledge the concept of public domain, you may choose to use it under the terms of the Creative Commons Zero license.