Effective use of Sitecore LinkDatabase

Keep track of your Reference Field references

I’ve often mentioned the Sitecore LinkDatabase, which I’ve found to be a hugely useful resource over the years but it’s getting very little exposure. And given I need to test out the Open Source version of my favourite blogging tool, I figured I might as well hit 2 birds with 1 stone and write a post about it.

What is Sitecore LinkDatabase?

Sitecore LinkDatabase (to save my sanity, hereafter referred to as SLDB) is a simple table that keeps track of Reference Field references. And what are Reference Fields?   Basically anything that stores one or more IDs as the raw field value. So Droplink, Treelist, Image, Internal Link and so on.

And raw values looking like this:

And this is where the SLDB comes into play. What it does, is keep track of those references. It basically stores something to the effect of; “Item {id} on Field {id} keeps a reference to Target Items {id}{id}{id}”.  The reasons this functionality exists, is to support the Sitecore Content Editor so that when you are about to delete an item that might be used elsewhere, it pops a warning.

In this case I am not given other choice (as I’m trying to delete a Template in use), but in other cases Sitecore will offer you to remove the references to the Item you’re about to delete, or relink the references to someplace else. As a side note; this, too, is immensely useful functionality if you’re refactoring your content – but that’s a story for a different day.

The nuts and bolts

Currently, the SLDB looks like this:

It sits in your “core” database. Be mindful of this if you’re going to be putting it to use – “core” may or may not be available in your Content Delivery environment depending on your setup. Easily fixed, however. Configuration for it sits here:

  <!-- LINK DATABASE -->
  <LinkDatabase type="Sitecore.Data.$(database).$(database)LinkDatabase, Sitecore.Kernel">
    <param connectionStringName="core" />
  </LinkDatabase>

Migrate the “Links” table from “core” to “web” or wherever you please, and you’re good to go.

The day to day is handled by this event handler, you’ll find it on most of the item events.

<handler type="Sitecore.Links.ItemEventHandler, Sitecore.Kernel" method="OnItemCopied" />

And there’s also a setting. Leave it at it’s default if you’re going to be using the SLDB at runtime (if you toggle this “off”, SLDB won’t be kept up to date for “web”).

<setting name="LinkDatabase.UpdateDuringPublish" value="true" />

Caveat: If you’re using the excellent Unicorn; be sure to allow it to update SLDB during sync operations. It is currently off by default. You need version 3.1.5 or later.

The nitty-gritty

Right, so enough. WHY should you care?  Well let me show you.

So I’ve set up a vanilla solution, and defined me some templates. It’s not particularly advanced, but it represents a pretty common IA challenge in our solutions today.

So we have a “News” template. It can exist in several categories, and has 1 Author.

IA challenge #1: We can’t organise “News” in category “folders”, as it is not a 1:1 relationship
IA challenge #2: We need news organised in a deep “folder” structure for SEO and performance

Using a bit of c# code so hacky I would never share it, I end up with 1000 “News” articles, with a random number of categories and so on.

I’ve kept the organisation simple, as this will be enough for what I’m trying to demonstrate.

So basically a “month” folder. In a real scenario, likely you would go 1 level deeper and include “year”. It won’t matter for this example however.

Now; our common problems with this are:

Problem #1: How do I list all “News” in a specific Category?
Problem #2: How do I list all “News” by a specific Author?

I’m going to pretend we live in a world where indexing this data isn’t an option, and we’re left with just the basic tools available to us.

Solution #1: Use Sitecore Query
Solution #2: Use Sitecore Fast Query
Solution #2: Use SLDB

Let’s see what that looks like. I cook up some rudimentary (and ugly) code:

var db = Factory.GetDatabase("web");
List<Item> authors = db.GetItem("/sitecore/content/Solution Data/Authors").GetChildren().ToArray().ToList();
List<Item> categories = db.GetItem("/sitecore/content/Solution Data/Categories").GetChildren().ToArray().ToList();
ID newsTemplateId = new ID("{5CB0FC6D-DCAA-4A51-8745-D9933F77679A}");
var newsRoot = db.GetItem("/sitecore/content/Home/news");

// Make sure Sitecore is warmed up
newsRoot.Axes.GetDescendants();

Response.Write("<h2>Authors</h2>");
var sw = new Stopwatch();
foreach (var author in authors)
{
    Response.Write($"<strong>{author.Name}</strong><br />");
    sw.Start();
    var referrers = GetAuthorReferrers(newsRoot, author.ID);
    sw.Stop();
    Response.Write(referrers.Length + " articles; Time in Ms: " + sw.ElapsedMilliseconds + "<hr />");
    sw.Reset();
}

And the most important bit, how I query it.

Regular Sitecore Query

Item[] GetAuthorReferrers(Item root, ID authorId)
{
    string query = root.Paths.FullPath + "//*[contains(@Author, '" + authorId + "')]";
    return root.Database.SelectItems(query);
}

I run this a couple of times (I’m not exactly in a stable environment to conduct any real scientific test of this – keep this in mind).

I then rewrite it for Sitecore Fast Query.

Sitecore Fast Query

BE AWARE!   Sitecore Fast Query sacrifices functionality to gain performance. Be especially aware of its limitations if your solution is multilingual.

Code now looks like this:

Item[] GetAuthorReferrers(Item root, ID authorId)
{
    string query = "fast:" + root.Paths.FullPath + "//*[@Author = '%" + authorId + "%']";
    return root.Database.SelectItems(query);
}

And an output that comes out like this. (Believe me, I ran this dozens of times).

Are we having fun yet?  I can offer up a bit of speculation on what’s going on here, but I’m honestly not completely sure. Mostly because I would never use neither solution to query my data, so my experience is somewhat limited. My best guess would be; to benefit from fast: query, you need more data. I will try this out with a larger dataset a bit further down.

So anyway. Roll on SLDB.

Sitecore LinkDatabase

Drawback of this approach is, that it’s a bit more code heavy. Not by much though.

Item[] GetAuthorReferrers(Item root, Item author)
{
    ID authorFieldId = new ID("{438E45E5-9F85-4705-976E-FC76E563F5EF}");

    var items = new List<Item>();
    ItemLink[] itemLinks = Sitecore.Globals.LinkDatabase.GetItemReferrers(author, false);
    foreach (var il in itemLinks)
    {
        if (il.SourceDatabaseName.Equals(root.Database.Name) && il.SourceFieldID == authorFieldId)
        {
            items.Add(il.GetSourceItem());
        }
    }
    return items.ToArray();
}

And here’s how we end up.

Am I managing to convince you yet?   so far we’re doing ok. We’re outperforming Sitecore Query by about 10 to 1. And Sitecore Fast Query by 50 to 1. A pretty decent start I’d say.

The sleight of hand

So I’m left with two problems. I’ve not brought “Category” into play. And I’ve not got a simple output, that fully compares these 3 side by side. I’ve created a Gist with the full source code – it is to long to include here, even by my standards.

Enter, a code rewrite. This time I’m looping through all Categories, then all Authors – finding all News articles that match the Category AND the Author. It completes the setup, and it actually demonstrates one of the slightly more complex situations when using SLDB. Using querying this becomes just another “and” expression.

SLDB code looks like this:

private Item[] GetNewsArticlesSitecoreLinkDatabase(Item newsRoot, Item authorItem, Item categoryItem)
{
    // Note. SLDB version of this actually needs to make 2 lookups. First to find all news articles referencing our author, then all that reference the category.
    // Note #2: Still, my hand is not shaking ;-)

    var authorFieldId = new ID("{438E45E5-9F85-4705-976E-FC76E563F5EF}");
    var categoriesFieldId = new ID("{31D2A6CC-6984-4CA0-BB73-9D39C3B8D0AA}");

    var authorLinks = Globals.LinkDatabase.GetItemReferrers(authorItem, false).Where(al => al.SourceFieldID == authorFieldId).ToList();
    var categoryLinks = Globals.LinkDatabase.GetItemReferrers(categoryItem, false).Where(cl => cl.SourceFieldID == categoriesFieldId).ToList();

    var newsArticles = new List<Item>();
    foreach (var authorIl in authorLinks)
    {
        var categoryIl = categoryLinks.ToList().Find(cl => cl.SourceItemID == authorIl.SourceItemID);
        if (categoryIl != null && categoryIl.SourceDatabaseName == newsRoot.Database.Name)
        {
            newsArticles.Add(categoryIl.GetSourceItem());
        }
    }

    return newsArticles.ToArray();
}

And what do we get?   Well this.

It matches up pretty well the results already established. Adding the extra query seems to have added a bit to all of the query execution times – this is completely expected. In relative terms however, I think it is still very clear who the winner is.

Sitecore Decennial Series #2 - Don't Sitecore Query your content

Sitecore does not query, how you think it queries

As a consultant, I am often brought in to projects and solutions that have - to some extent - gone wrong, or does not quite meet up to expectations. And when it comes to those, absolutely without question, the most common problem I come across, is under-performing solutions.

And let's just clear something up. While you will find a lot of posts out there discussing the speed and performance of Sitecore (Sitecore 8, in particular); these are all referring to the performance of the Sitecore Experience Editor. The client environment. And while there is rarely this much smoke without a fire somewhere, I will say this:

The Sitecore runtime is a very well oiled machine. If it doesn't perform, it means YOU broke it.

You are welcome to quote me on this.

Let me exemplify.

Pretty gruesome, right?  You'd not blame this on "poor performance of ASP.NET", if you found this in some web application I assume.

Now believe me when I tell you; this horrifying example is actually very mild compared to what I often find hidden deep in some of the Sitecore solutions I visit. Usually tucked away in some "*Helper" or "*Utils" (pet peevees of mine) class, used arbitrarily throughout the entire project, never to have been visited by a developer since it was first put in place.

So anyway. Let's take a look at, how situations like this come to be. I will begin with some fundamentals.

The Sitecore (Sql) Data Store

To try and understand what's going on, we're going to take a bit of a dive into the Sitecore Sql data store. Now before you start; I am very well aware that Sitecore abstracts this layer from view (and rightly so), and under no circumstances should you ever be working on the database layer when doing Sitecore. The structures I am about to show you do change from time to time, and you will end up with a bloody nose if you go down this road.

But we cannot escape the fact, that Sitecore data storage is based on Sql Server (in most cases, actually in every single one I have come across in the past 10 years) - and any data storage mechanism comes with a set of built-in limitations and restrictions. So how Sitecore uses its data storage matters, and it will affect how your solution performs.

[Items]

Without question, the table most central to the Sitecore data store. I imagine the fields are self explanatory. 1 row in this table, for every Item in your Sitecore Database ("master", "web" and so on).

This table holds no other values other than Item.Name. For this, Sitecore looks to 3 other tables. They look almost identical.

[VersionedFields]

[UnversionedFields]

[SharedFields]

If you haven't guessed already; Sitecore decides where to place field values depending on how you check your "Shared" and "Unversioned" checkboxes when defining your Sitecore Templates. I will not go further into this, in this post.

But what does this mean?   Well there's a couple of things to pay close attention to here. The first one being; Sitecore sees its data store as a Tree Structure (d'uh I hear you say, you probably already knew that). But this is important. Notice how [Items] stores Id and ParentId (and a few other Ids we don't care about right now). To go anywhere in the "Tree", Sitecore must therefore traverse up and down this chained list of Ids - starting from the top.

So if I were to do something simple as say; Sitecore.Context.Database.GetItem("/sitecore/content") - what this ends up as, is something like this:

exec sp_executesql N' SELECT [ID] FROM [Items] WHERE [ParentID] = @parentId AND [Name] = @childName',N'@parentId uniqueidentifier,@childName nvarchar(7)',@parentId='11111111-1111-1111-1111-111111111111',@childName=N'content'

(Sitecore caching is in effect, so it was already aware of the parentId)

Which is not so bad; but do remember. Had I queried "/sitecore/content/home/data/articles/2015/february" - this would (uncached) have resulted in 6 queries like the one above, to Sql Server. Yep. Again; this is without considering caching, obviously, or we'd all be in big trouble.

And this is just getting an Item. As we saw above, the field values sit in separate tables. Once you start accessing actual content on the Item, this is what it looks like behind the scenes.

exec sp_executesql N'SELECT [ItemId], [Order], [Version], [Language], [Name], [Value], [FieldId], [MasterID], [ParentID]

                     FROM (

                        SELECT [Id] as [ItemId], 0 as [Order], 0 as [Version], '''' as [Language], [Name], '''' as [Value], [TemplateID] as [FieldId], [MasterID], [ParentID]

                        FROM [Items]

                        UNION ALL                          

                        SELECT [ParentId] as [ItemId], 1 as [Order], 0 as [Version], '''' as [Language], NULL as [Name], '''', NULL, NULL, [Id]

                        FROM [Items] 

                        UNION ALL 

                        SELECT [ItemId], 2 as [Order], 0 AS [Version], '''' as [Language], NULL as [Name], [Value], [FieldId], NULL, NULL

                        FROM [SharedFields] 

                        UNION ALL 

                        SELECT [ItemId], 2 as [Order], 0 AS [Version],       [Language], NULL as [Name], [Value], [FieldId], NULL, NULL

                        FROM [UnversionedFields] 

                        UNION ALL 

                        SELECT [ItemId], 2 as [Order],      [Version],       [Language], NULL as [Name], [Value], [FieldId], NULL, NULL 

                        FROM [VersionedFields]

                     ) as temp  WHERE [ItemId] IN (SELECT [ID] FROM [Items] WITH (nolock)  WHERE [ID] = @itemId) 

                     ORDER BY [ItemId], [Order] ASC, [Language] DESC, [Version] DESC',N'@itemId uniqueidentifier',@itemId='0DE95AE4-41AB-4D01-9EB0-67441B7C2450'

Incidentally; ever had a [Shared Field] that you switched to [Unshared], and now you're not getting the field values you expect?  this query is the reason. When you toggle that checkbox, Sitecore starts a background task that will migrate the field values from [SharedFields] to [UnversionedFields]. It has to, as the above query will otherwise return the first field value it finds in the UNION - which will be the one near the top; [SharedFields]. If this task somehow doesn't complete - your data storage gets left in a bit of a pickle. 

So yea. Sitecore essentially goes and grabs all your field values once you start accessing content on an item. And yes, this IS the efficient approach - a roundtrip to Sql for every field value you access hereafter would be absolutely crippling.

But keeping this in mind; picture this:

Sitecore.Context.Database.SelectItems("/sitecore/content//*[@PageTitle='news']")

Caches aside; you've just instructed Sitecore to recursively go find anything that has /sitecore/content as an ancestor (1 SQL statement for each item, to find them all). And then your request for the PageTitle field will then, for each of the found items, execute the big UNION statement from above so that it can determine the field value for PageTitle.

Yep, you read that right. You may be imagining things like; "but I can construct a SQL statement that finds this more effectively". And you're right, you can. But you'd also be ignoring Sitecore through and through - you might as well not bother using a CMS at all. What you're overlooking (to name a few) are things like Versions (which of the 7 versions of /Home is the active one?), Language, Security, Workflow Status. 

I'll say this as plainly and calmly as I possibly can. Do NOT use Sitecore Query for any code that executes at runtime on your solution. "If it doesn't perform, it means YOU broke it."

When it comes to constructing a more efficient query, you're not the first one to have this thought though; enter Sitecore Fast Query.

Sitecore Fast Query

SFQ is a query mechanism that aims to address some of the limitations of regular Sitecore Query. It does so by compromising exactly some of the things I mention above. So sacrifice a little functionality to gain a little speed. I'm almost tempted to paraphrase Frankling and claim "you shall have neither".

But let's look at it. From the "Using Sitecore Fast Query Cookbook" (how I miss those)

Now; given everything that I have just shown you - this makes perfect sense. The only supported special attributes - all of those are what you find on the [Items] table. Standard Values are not supported (as these sit on a separate item). I'll leave it to you to speculate why field values need to be surrounded by % signs ;-)

Ok, so let's take a look then. I cook up something new:

Sitecore.Context.Database.SelectItems("fast:/sitecore/content//*[@#Page Title#=\"%article%\"]");

And the resulting Sql looks like this:

exec sp_executesql N'SELECT DISTINCT [i].[ID] [ID], [i].[ParentID] [ParentID] FROM [Items] [i] WITH (NOLOCK)  LEFT OUTER JOIN (SELECT [Fields].* from [Fields] INNER JOIN [Items] ON [Fields].[FieldID] = [Items].[ID] AND lower([Items].[Name]) = ''page title'') [Fields1] ON [i].[ID] = [Fields1].[ItemId] INNER JOIN [Descendants] ON [i].[ID] = [Descendants].[Descendant] INNER JOIN (SELECT DISTINCT [i].[ID] [ID], [i].[ParentID] [ParentID] FROM [Items] [i] WITH (NOLOCK)  INNER JOIN (SELECT DISTINCT [i].[ID] [ID], [i].[ParentID] [ParentID] FROM [Items] [i] WITH (NOLOCK)  WHERE LOWER([i].[Name]) = ''sitecore'' AND [i].[ParentID] = @value1) [a] ON [i].[ParentID] = [a].[ID] WHERE LOWER([i].[Name]) = ''content'') [a] ON [Descendants].[Ancestor] = [a].[ID] WHERE (coalesce([Fields1].[Value], '''') LIKE @value2)',N'@value1 uniqueidentifier,@value2 nvarchar(9)',@value1='00000000-0000-0000-0000-000000000000',@value2=N'%article%'

And a footprint like this.

Ok. So there's no doubt, this is BETTER than our previous situation. But only by a measure, it's not the end-all to our performance woes. I have about 20 (!) items in the vanilla solution I am using to write this blog post by the way. Twenty.

Interesting (unofficial) factoid: Do you know why FAST: query was introduced to begin with?  To solve performance problems in Sitecore Content Editor. Yes. Think about it; opening up any item in CE presents you with a series of fields. Each of these fields may be Treelists and whatnot, and loading up all that related meta-data and presets, populating all the dropdown boxes and so on - well it pretty much killed performance in early Sitecore (5 and 6) releases. And for Content Editor, the limitations of Fast Queries are fine. No doubt they're (often) better than the regular Sitecore Query equivalent. But that still doesn't make them good.

Don't look to caching to save you either. It helps, surely. But even cached results need to be generated at least once; and now you find yourself in a whole heap of other problems instead. When to clear cache?  on publish?  you sure?   So with editors publishing new content every 5 minutes (to test it out - trust me, they do), how much "cache debt" does your solution need to deal with?

Look; we're in my Decennial Series. I'm giving you my hard earned advice. 

Just stay away from any form of querying. There are other and better ways. I'll give you a few pointers; most of what I'm highlighting here is already well covered by blog posts everywhere (far from all of them, mine). Seek them out, please. Please. Next time you feel, you need to query for your content.

What to do instead

Information Architecture

The first and most important thing to keep in mind, is how you organise your content. Try and keep related items close together and organise them in a meaningful deep tree hierarchy. So for, as an example, News Articles, do this:

/sitecore/content/news/2016/04/[your articles here]

Which gives you the option to do:

var root = Sitecore.Context.Database.GetItem("/sitecore/content/news/2016/04");

var newsArticles = root.GetChildren();

And unless you have an extraordinarily "news'y" site, I assure you this will be better than the Query equivalent which might look something like:

var newsArticles = Sitecore.Context.Database.SelectItems("/sitecore/content/news/*[@newsDate > '2016-04-01' & @newsDate < '2016-05-01'])

Don't be overly worried about reading in 10-20 items in a .GetChildren() call; be worried about reading in 2000 items and discarding 1980 of them.

Code Smart(er)

There are often multiple ways of achieving something. This certainly holds true for Sitecore as well. One of the little known gems is the Sitecore LinkDatabase. Ever tried deleting an item, and have Sitecore pop up and tell you this:

How does Sitecore know this?  And you probably guessed it already. The LinkDatabase. The SLDB is easily worthy of a post of its own, but in essence it's this:

A table maintained by Sitecore, where all reference field references (!) are stored. This allows Sitecore to keep track of inter-Item relations so it can pop this warning to you.

Since we're doing the low-level thing today, here's what it looks like:

Turns out, this is an excellent tool in many other situations as well. I think my Listing "Related Articles" using LinkDatabase was the first time I blogged about it; and I regularly dust off SLDB and pick it up from my tool shelf even to this day.

A common scenario that often comes up, is "I need to find all News articles, but they're spread out all over my content tree. Silly editors". Well SLDB has a fix for that; although here it is used indirectly.

In case you're wondering; .GetUsageIDs() uses SLDB internally to achieve it's result.

I highly recommend you read up on the Sitecore LinkDatabase. It can really make your life easier in a lot of situations, where you would otherwise feel tempted to start Sitecore Querying.

Index your content

And finally, but also most importantly, use an index for every single Sitecore solution you ever do. I mean it. Seriously.

Want to find all News Articles in a specified date range?  use an index.

Want to find all News Articles in the "Business News" category, sorted by Author? use an index.

Even with all the tricks I have show you here, sooner or later you will come to a situation where the content cannot be organised any differently, where LinkDatabase cannot help you as we're dealing with content and not references - or flat out, we're dealing with so much content that we just have to narrow down the dataset to be able to work with it.

Fortunately, I've already given the answer to these problems. Use an index.

There are quite literally dozens and dozens of blog posts out there, on how to set up and use Sitecore ContentSearch (which is where I recommend you start out). It takes so little effort to get going, it's active out of the box (using Lucene as it's provider), there quite simply is not a single valid excuse not to use it.

I know it's "another thing to learn". This one is not optional however; you really cannot go around it. You may feel you're doing just fine and your Sitecore Queries run fine. Great. Give yourself a nice "It works on my machine" sticker, and then get those indexes going ;-)

Seriously.

Until next time :-)

Sitecore Decennial Series #1 - Know your item, remember your context

Understand the basics

The most common problems I find when working with (other people's) Sitecore solutions, has a root cause in either a lack of understanding of the basic concepts of Sitecore and/or a misunderstanding of same. I don't actually know if this is surprising or to be expected - it is what it is.

I'll try and isolate some of the most common basic mistakes, in no significant order. This is also to say, I consider them ALL equally significant ;-)

1. Understand your Item

Yes I'm talking about you, Sitecore.Data.Item.

Chances are, Item is one of the very first concepts you come across when you start developing your first solution. It might look something like this:

    Item home = Sitecore.Context.Database.GetItem("/sitecore/content/home");

    string headline = home["Headline"];

Simple, yea?  Not so. Reading the syntax like this has a high risk of tricking your mind into thinking a lot of things, all of which are false and will lead you astray sooner or later.

1a. A context is implied

As with the very large majority of all interaction with the Sitecore API, a context is required for any interaction. I will dig into this in detail a bit further on.

That method call, is just a method overload for a call that looks like this:

    Item home = Sitecore.Context.Database.GetItem("/sitecore/content/home", 

                Language.Current, 

                Version.Latest);

Why is this significant?  It's significant because here, as in most API calls, Sitecore breaks out and pulls additional information required to execute the call; in this case it needs to determine what Language Version of the Item to get. And once it knows the language, it needs to know which Version of the Language Version to get.

And while this may not seem obvious to you while still learning the robes of Sitecore development (it didn't, to me), this is actually very significant.

Very many things in the Sitecore API requires some sort of context. And knowing them will be important to you, when you advance into more advanced development.

1b. People say "Item" when they really mean "Item Version".

There really is no such thing as an "Item". Item is a construct in Sitecore that holds a lot of information related to the content of the Item Versions - but really holds none of the actual content. (For the advanced readers; I realise we could argue the merits of this - but as a general principle, this holds true).

So what is on Item?    Important things. Like:

• Name (by default, defines the URL string for the item)
• Security
• Template (could also be called the "Schema" for the Item Versions)
• Statistics (Last Updated, Updated By, etc.)
• Publishing Information
• Workflow Information
• Validation Rules

But for most intents and purposes, not things you need in your day to day life of creating Accordion components or whatever your task.

Sitecore.Data.ID uniquely identifies any Item in a Sitecore solution. And from this, we now also see, that Sitecore.Data.ID does not adequately represent an Item Version.

1c. Understand the different identifiers. ID is not always what you need.

Unbeknownst to many, judging from rarely I find these in use in Sitecore solutions I look at, Sitecore actually has many better options than Sitecore.Data.ID available. All in the Sitecore.Data namespace.

Given this piece of hackety webforms code (had to destroy the BR tags to keep Blogger happy):

  var home = Sitecore.Context.Database.GetItem("/sitecore/content/home");

  litOutput.Text += $"ID (home.ID): {home.ID}$br />";

  litOutput.Text += $"Uri (home.Uri): {home.Uri}$br />";

  litOutput.Text += $"DataUri: {new DataUri(home.ID, home.Language, home.Version)}$br />";

  litOutput.Text += $"ItemUri: {new ItemUri(home.ID, home.Language, home.Version, home.Database)}$br />";

  litOutput.Text += $"VersionUri: {new VersionUri(home.Language, home.Version)}$br />";

The output is:

  ID (home.ID): {DAC24EDD-44FB-42EF-9ECD-1E8DAF706386}
  Uri (home.Uri): sitecore://master/{DAC24EDD-44FB-42EF-9ECD-1E8DAF706386}?lang=en&ver=1
  DataUri: sitecore://{DAC24EDD-44FB-42EF-9ECD-1E8DAF706386}?lang=en&ver=1
  ItemUri: sitecore://master/{DAC24EDD-44FB-42EF-9ECD-1E8DAF706386}?lang=en&ver=1
  VersionUri: en, 1

Any Item you have instantiated (like from a GetItem() API call) will be uniquely identified by an ItemUri, as found on the .Uri property. .ID tells you only the ID of the underlying Item.

From this we also learn, that the Item we get from the API does not exist outside of a Sitecore Context. With both Database, Language and Version information. This is undoubtedly become a pet peevee for you at one point or another, if you start looking to do Unit Testing or any kind of abstractions to the Sitecore API. My honest advice; leave this be for now. For at least a couple of years into your Sitecore learning curve.

Keep your Item identifiers in mind. Don't use .ID as a cache key when publishing Item Versions. Do use DataUri, ItemUri and VersionUri as appropriate, don't re-invent the wheel with your own bespoke implementations or - worse - just ignore the fact that ID does not tell you all you need to know.

Speaking of pet peevees, here's one of mine.

2. Understand your context

2a. Don't break context or get a context you don't require

Given the following code:

    public Item[] GetNewsInCategory(Item categoryItem)

    {

        List articles = new List();

        if (!string.IsNullOrEmpty(categoryItem["Articles"]))

        {

            foreach (var articleId in categoryItem["Articles"].Split("|".ToCharArray()))

            {

                articles.Add(Sitecore.Context.Database.GetItem(articleId));

            }

        }

        return articles.ToArray();

    }

Actually there are 2 pet peevees of mine in here. One is not using the Sitecore API to properly deal with the MultilistField. The other is the breakout to Sitecore.Context.Database to get the items defined in the field. Why? We already take an Item as a parameter, so we already HAVE a Language and a Database context. At the very LEAST, do this:

    articles.Add(categoryItem.Database.GetItem(articleId));

In the inner loop. If you make methods and these happen to take an Item as an argument, by all means USE the context of that argument to carry on. You'll be happy you did, as you will one day find yourself wanting to call your code from say an Index Handler, an Item Saving event or whatnot - and you cannot assume you have your normal page context available in these cases. I've seen what happens on this particular road to hell, and it usually ends up with a line of code like this getting injected.

    database myDb = Factory.GetDatabase("master");

To try and solve the problem, with Sitecore.Context.Database being NULL in some cases. No, no, no, nope, please, just don't. Use the .Database property of the Item you're dealing with. Whoever instantiated it, already made a context for it (see above; no Item with no context).

That said; what the above code SHOULD look like this this. (Leaving out the argument about argument assertion for now).

    public Item[] GetNewsInCategory(Item categoryItem)

    {

        MultilistField articlesField = categoryItem.Fields["Articles"];

        if (articlesField != null)

            return articlesField.GetItems();

        return new Item[] {};

    }

2b. Context, context, context everywhere

I think you realise by now, I find the subject of Context in Sitecore very important ;-)

Here's the thing. Try and avoid using it. While it is indeed very convenient to just jump out and grab a Sitecore.Context.Site whenever you need it, or Sitecore.Context.Language or whatever it may be. But it is also very bad form for your code. It is in fact an anti-pattern.

"But all of Sitecore is written like this?"

CAREFUL!  PERSONAL OPINION WITH SOME SPECULATION FOLLOWS!

Yes. I don't know what to tell you. I'm pretty sure if the original development team was to start today, much of this codebase would have been done following a different mindset. But most of the codebase you're looking at is between 10 and 15 years old, and Sitecore has always been adamant that backwards compatibility be preserved unless there were very good reasons to break it. They follow principles followed by Microsoft very closely when it comes to this.

A good example of this is my own CorePoint.DomainObjects, one of the very first public ORM mappers for Sitecore. Written almost 8 years ago, and it can still be built on recent Sitecore versions without too much headache.

Sitecore uses static constructs all over the place. It will drive you nuts if you try and code to modern standards, e.g. using Dependency Injection (you should), but that's just how it is. I tell you another thing though; calling through a layered API of static methods and classes is faster than dynamically resolving types at runtime. While we accept this cost today, performance was a much different beast 10 to 15 years ago.

So anyway. Back to my original point. Forgive me for pointing out the obvious here. YOU'RE NOT WRITING A CMS SYSTEM. What you're doing, is writing a codebase that will eventually turn out to be an excellent Sitecore solution, running the website of your (or your client's/employer's) dreams. Nowhere is it stated, your code standards need to follow those of Sitecore. Yes, you need to adhere to Sitecore Best Practices in your interactions with the Sitecore API and all that, obviously, but nothing else in Sitecore dictates how you should organise your project and solution.

And yes, this is a two edged sword, and why initiatives like the Habitat solution surfaces. Complete freedom, unfortunately, also means you have complete freedom to mess up things. Badly.

To bring some concrete suggestions out; if you need a Database in your method or class, ask for it in the constructor or as a parameter. If you need a SiteContext, ask for it. Don't - please don't - try and configure a full Dependency Injection setup and abstract all of Sitecore into interfaces if this is your first Sitecore solution. It will cost you a LOT of time, much much more than you realise, and chances are no one will ever make back that investment of time in your first solutions lifetime. 

Yes, I really did just write that ;-)  Take my word for it. 

2c. Are you sure you need that event handler? And if you do, are you sure you're hooked into the right one?

Look, I'm pretty sure you don't need that item:Saved handler. Why?  Because the real need for them is so very rare. I can probably count on one hand, how many times I've needed to implement one over the course of 10 years of Sitecore development.

Chances are, you're trying to make Sitecore do something it shouldn't really do. This is actually a reference I wrote a blog post about years ago; Just because you can, doesn't mean you should. I'm going to rewrite this as part of this Decennial series, but for now the original post will have to do.

Take a step back; consider if what you're doing is really trying to solve a user training problem with a programming solution. Still need that handler?  Ok then.

Consider the context of your handler then. I often see examples, like an item:Saved handler that manipulates other items or possibly creates and re-creates parts of the Sitecore content tree, all based on a particular field value or something similar. Are you aware that item:Saved is fired as part of the PublishItem process?  (like when the published item is Saved to the "web" Database). 

Check your context, filter your context.

If you do implement handlers and processors, at least make sure they only execute when you expect them to. Check if item.Database.Name really matches the ContentDatabase, abort if it doesn't. Check that your bespoke ItemResolver code is currently serving content for the website you expect. "publishing", "shell", "scheduler" etc. are all websites on your solution, are you aware of that?

Consider these things, whenever you hook into anything. Be it item events, request processors, link managers or otherwise. 

As an example, look at the item:Saved handler that deal with keeping your LinkDatabase updated. (a hugely underestimated resource when it comes to Sitecore development, but this will be a subject for one of my next posts in this series).

    protected void OnItemSaved(object sender, EventArgs args)

    {

      if (args == null || LinkDisabler.IsActive || !Settings.LinkDatabase.UpdateDuringPublish && PublishHelper.IsPublishing())

        return;

      Item obj = Event.ExtractParameter(args, 0) as Item;

      Assert.IsNotNull((object) obj, "No item in parameters");

      LinkDatabase linkDatabase = ItemEventHandler.LinkDatabase;

      if (linkDatabase == null)

        return;

      linkDatabase.UpdateItemVersionReferences(obj);

    }

Notice how, the first statements in the event handler actually deals with asserting, if it should run at all. Sitecore makes no such determination for you, it is YOUR responsibility to ensure this.

This also goes for your PageMode.

2d. What is your current PageMode. Is it relevant?

Considering the current PageMode becomes important, when you're making run-time decisions that could affect the user experience.

Let's say you're putting in some code, to prevent your component from failing if it has been configured with a faulty Datasource. Or alternatively if you want to explicitly throw an Exception in that case, to help your fellow developers track down a bug.

Be careful. At site run-time (when PageMode.IsNormal) it could indeed be considered an error condition, if a component is configured with a Datasource that does not exist. This is quite likely NOT true for many of the other PageModes. Consider this:

An Editor is Page Editing (or Experience Editing, the new bling expression) inserts your component onto a page. What happens (simplified) is, that Sitecore adds your component to a placeholder key and renders it. You may or may not have a Datasource defined at this stage. Don't blow up. Don't YSOD. Your component is in a staging state, and your code needs to consider this. This is what PageMode is for. 

In general, I find it really bad form to YSOD on these specific conditions; like a missing Datasource or a reference field pointing to items that do not exist. Why?  Because it's very likely just User Error. An Editor forgetting to publish a related item (something that is VERY easy to do in Sitecore, even if the current publishing tools make this slightly easier). You really DON'T want to teach your users, if they make a mistake you're going to YSOD their site. You really don't.

Alternatively, if you can, discuss with your users what should happen. Either the component outputs some harmless content to alert them of this condition, or perhaps it hides itself completely. Again, only do this if PageMode.IsNormal or PageMode.IsPreview. Or maybe PageMode.IsDebugging, if you're using the Sitecore Debugger (you should). But consider it, don't just ignore it.

So I think that's it. For this post, anyway. Until next time :-)

10 years of Sitecore blogging – whereto now?

10 years of this blog and me (but probably mostly me)

So it all started in early 2006 with a post about IDTables (of all things…). I was a newly coined Sitecore Certified Developer, having recently left my “vanilla” .NET development consultant job to go work for one of the premiere Sitecore Partners and was now doing Sitecore full time.

As the blog posts of that time will testament, I was very much learning at the time. I was thrown straight into a rather deep dive, doing work that involved custom Data Providers, item proxying and in general trying to figure out ways to best integrate a lot of external data into Sitecore. Early versions of Sitecore 5 were… let’s use the word “challenging”, and Sitecore documentation and release notes were scarce, at best.

So yea; the first couple of years of blogging was almost entirely focused on sharing information that perhaps wasn’t otherwise apparent and openly discussing the direction Sitecore was taking on what I then believed to be a vitally important data integration features. These features are still important of course, but my view on this has become somewhat more nuanced over the years. More on this later.

Right around the launch of Sitecore 6, we’re talking 2008/2009 here, I had left the safety of employment behind and had started CorePoint IT Ltd – my 1 man Sitecore freelance/contracting company and was trying to work closely with the Sitecore UK office on helping out new Sitecore partners as they were brought into the fold. Sitecore UK was a very different entity those days, only a handful of people, working very hard to spread the Sitecore gospel to the British Isles.

“Yes, all fine and well, but how does that relate to this blog?”.  Well it does, see. Unlike today, there WAS no real market for Sitecore contractors back then. We were maybe a handful doing it, and LinkedIn did not have a constant flow of “Contract Requests” coming through on a daily basis.

So initially, I had a lot of time on my hands. I wrote what I believe to be one of the first ORM mappers for Sitecore – CorePoint Domain Objects; built a lesser known Ecommerce framework, and as my post count for 08/09 will show – wrote a lot of blog posts. It was all about visibility.

And it paid off, too. Work started coming in, to pay the almost extortionist London rent rates. I was awarded Sitecore MVP as one of the 10 first in the world and things were getting busy. So much so that I pretty soon found myself almost drowning in work and Sitecore services was coming in demand all over London (it was still predominantly London, at this time).

It also shows in my blogging activity ;-)   More posts were written in 2009 alone, than all remaining 9 years on this blog. As a contractor, one doesn’t get paid to blog (obviously) and while many bloggers today find time to blog on their employer’s dime, I have always been doing this on my own time. Time I take away from either work or (more often) family activities.

2010 through 2012 was “more of the same”. I went back to Denmark due to a family related issue, ended up staying almost 2 years working almost exclusively with Sitecore ecommerce related sites and solutions – architected yet another version of an ecommerce framework (based on ideas developed in 2008/09 and further refined here) – again being kept so busy, there was little time (or rather; energy) to focus on other things such as this blog.

2013 sees me back in the UK, this time outside of London. The market in London was now brimming with Sitecore contractors (though still not enough, by far) and my idea was to head out west – to Bristol – and service an area that was not well covered by other consultants. Not long after (but entirely unrelated, although I like to claim otherwise ;-)), Sitecore themselves opened up a branch office in Bristol. Excellent news, this also meant local Sitecore User Group sessions – and I remember the very first one I attended, was when Sitecore 7 was being presented.

Initially I get involved in yet more ecommerce on Sitecore with a local Sitecore client (as opposed to an agency) where I spent the better part of a year, but pretty soon thereafter it becomes apparent that the market “out west” isn’t all I had initially hoped. To keep work coming, meant I had to consider contracts pretty much all over the UK. That meant a lot of travelling, a lot of hotel rooms and you know… to be honest, it all became a bit too much.

I spend 2015 re-considering what I want to do with it all and by late 2015, having arrived at some conclusions, I make my biggest change yet and move to Switzerland. The German speaking market is massive; and there is currently quite a huge lack of qualified Sitecore developers and consultants – I figure I can keep quite busy here for years to come :-)

But enough about me.

Visitors and readers – Who are you?

Apparently, Blogger has decided to discard some of its data so unfortunately I cannot share with you the complete traffic numbers. Only data from the period "May 2010 – January 2016" is kept.

So who are you?    This blog doesn’t run on WordPress so I cannot do one of those fancy “You filled the Sydney Opera House 27 times over” type of posts. I do have some statistics to share with you however.

And it would appear, the vast majority of you are from the US, with United Kingdom and France as runners up. If anything, France being in the top 3 surprises me; I am unaware of France having a particularly high Sitecore adoptation rate. Looks like I could be very wrong on that :-)

All of this adding up to: “Pageviews all time history - 237,063” (again, since May 2010).

In terms of what posts you liked, 2009 seems to have been the absolute golden year with 6 of 10 posts in the top 10 coming from that year.

On a personal level it pleases me to see topics such a Sitecore migration at the top since this is one of the areas I would consider myself a specialist in. For some reason or other, I’ve just ended up doing SO many data migration projects with Sitecore over the years. Looking at this, I should probably schedule a few follow-up posts on this.

Also pleased to see some of my “recent” posts (and one from my friend and fellow blogger; Finn Nielsen) make it into this list. Gives me hope that this blog has more than just historical significance ;-)

There is also a statistic for traffic sources that I won’t even bother screenshotting and posting here. 97+% of traffic comes from Google referral, the rest comes from StackOverflow and a few forum posts on Sitecore Community Forums.

And whereto from here?

The next 10 years of this blog

10 years is a very long time. When speaking of Internet technologies it’s almost more than a lifetime. So no, I don’t actually know that I will be actively blogging and working with Sitecore, 10 years from now. What I do know, however, is that I expect to be.

What I also know, is where I want this blog to be heading in the time to come. 

In my opinion, for this blog to have any justified existence (or any blog for that matter), it must be focused on original content. To me this translates to less “How-to” content, and more “Why or why not?” content. If you take a quick look around the Sitecore blogosphere landscape today, the reasons for this should be obvious. 

When I started blogging in 2006 there was little to no documentation, and probably there were less than 20 of us blogging. If even 10. Blogging original content was a no-brainer. We were all working with pretty much a blank slate and there was so much to talk about.

Today, 10 years later, there are more Sitecore blogs than I know the number of. I used to follow them all on a blogroll (up until Google shut down Reader :/) – today this is (almost) impossible. Impractical, at least. Ironically this leads to the almost exact opposite situation as we had in 2006; it is becoming increasingly difficult to now find quality information on any particular Sitecore subject. Not because it isn’t there; there’s almost too much of it. 

So anyway. To stay relevant, I am going to take my 10 years of experience, over 20.000 billed hours of Sitecore consulting, and put myself out there and post (even more) opinions. The way I see it, opinions and practices I have formed on working with Sitecore over all these years are highly sought after by employers and agencies; surely these must have value to the readers of this blog as well.

So more “What does IoC and DI do to your development teams velocity?”, “What developer profiles make for good Sitecore developers?”, “The Content Release project milestone”, “How does Agile fit with your Sitecore development project?” – and less “Using Lucene and SOLR side by side with Sitecore Content Search”, “Load balancing your MongoDB setup”, “Multi-select dialogues with SPEAK 1.1”.

To be clear, there is absolutely nothing wrong with the “less of” examples I mention here. I just won’t be focusing on them; I believe we as a community are pretty well covered on those as it is.

To start things off, I will launch a project I have been working on for quite some time now. The “Sitecore Decennial Series”. What this is, is basically 10 posts each focusing on one aspect of Sitecore that in my opinion is key to success with Sitecore implementations. Not necessarily all from a developers perspective. Each post will be rather long – beware, those of you not into that kind of thing – and for this reason I am also going to set a realistic schedule for myself on these. The series will run over the course of this year, with a post every month – leaving 2 blank slots (TBD) which I hope to fill up with a bit of R&R and holiday time ;-)

Happy New Year everyone.