TreelistEx not registering links in LinkDatabase

Ahh nothing like being kept up till 11pm on the last day before xmas holidays... :P So here's the thing. TreelistEx is broken. You're supposed to be using this field type to "tag" content. Probably most often, you will use this to tag up categories for news articles, blog posts, meta keywords and similar stuff. And on that same note, quite obviously, you're supposed to be able to "reverse lookup" all these tags you can set. Not much point in having to loop through ALL of your 100.000 articles to find the ones that are marked "Holiday Capers" - Sitecore has a Link Database for that. If you've not worked with the Link Database before, Lars Nielsen has an excellent post about it. But here's the catch. As I said, this is broken. Nothing you store in a TreelistEx field will be registered in the LinkDatabase. "tree list", reference, droplink - sure. They all work. But not this one. A little work with Dr. Reflector eventually led me in the right direction. Now why one would need to, is an entirely different story for a different day - I believe I already had a few comments on that earlier. It comes down to TreelistEx being unknown to Sitecore itself - odd as this may sound. The field type simply doesn't exist in FieldTypes.config - and this will eventually force Sitecore to skip the field when doing it's internal RebuildItem functionality. This is, as Reflector will tell you, the method that Sitecore will recursively call when rebuilding the Link Database. Add the missing field type to /App_Config/FieldTypes.config - force a full rebuild of the Link Database - and everything now clicks into place as it should. fieldType name="TreelistEx" type="Sitecore.Data.Fields.MultilistField,Sitecore.Kernel" Thanks Sitecore. Where do I bill my lack of sleep? ;-) EDIT: 14-February-2009 This fix was released in build 090212 (also known as Service Release 1).

Showing Sitecore how to improve...

A while back, Alex De Groot invited to an open debate on areas where Sitecore could improve. This post is my response.

Readers of this blog will have noticed, I often take a somewhat critical stance to Sitecore. What may not be so obvious at a first glance however is, my beef is not so much with the product itself, but rather Sitecore as the company behind the product. The issues that come from working with the product ultimately comes down to the practices that are in place in the company that creates it. Sure, a particular bug might be annoying but it's just the smoke - not the fire itself.

So I won't be requesting features, in short. In fact, if anything, I will request "no new features" at all; but instead point out that good solid project management practices, quality assurance, usability studies and whatnot make up a very important (if not the most important) part of any ongoing software development endeveaour. Even if I couldn't spell that right... ;-) But then - it is also obviously very easy for me to just sit here and say "hey guys, manage your stuff - how hard can it be?!". I've been doing software development for many years now, and if anything I've come to realise that perfection is more a direction than a goal in itself.

I've seen improvements in the way Sitecore has been working, especially over the last year or so. Documentation for instance, has always been a weakness when working with Sitecore. So I was very pleased to find, that documentation is taking on a whole new level with The Sitecore 6 Documentation Package - while far from covering everything there is to say about working with Sitecore, it's certainly a step in the right direction. Through Sitecore Support I've had the priviledge of looking at some of the other documents still being drafted, and I'm glad to see that this new trend is taking on. Documentation would have been top of my list still, as the first thing to improve. It doesn't matter that you have a good product, if noone knows about it. With that said however, for crying out loud guys will you fix and consolidate your online information already? Per Bering from Codehouse pointed this out as well as a comment to Alex' post. Your site search is absolute bollox and there is not a single other thing to say about it :P "Did you mean.. xsl rendering xsls rendering xslext rendering exslt rendering xslt rederings xslt renderingid xslt renderingitem xslt renderendtag " ... :P

Right now, the only way I am staying "on top" (if that's what I am, I don't know) of current Sitecore technical information is, by subscribing to roughly 20 Sitecore related blogs, chatting with people who work for Sitecore, chatting with other consultants who work for Sitecore partners and then as the last resort - the ultimate Sitecore tool; Reflector. I've even seen Sitecore support personnel themselves point developers in the direction of Reflector as a source for information. And while it's a great tool, I can't really believe this is how it is meant to be. Notice btw, how I deliberately didn't include SDN5 as a common source of information for me. While I go there, it's very rare that I am able to successfully find what I'm looking for there - so the fallback option is Sitecore Support. Fortunately I find Sitecore Support very good, at least :-) Enough about documentation however.

The second thing I would ask for, is for Sitecore to start eating their own dogfood. If it's not good enough for you to run your own site on, it's certainly not good enough for the very large developer base out here who has very limited access to documentation on the product. Update notes and changelogs don't really tell a useful story as I think most who work with the product will know. I've never seen a changelog saying that the "admin" user no longer comes with Danish locale as default in a blank Sitecore 6 - and that this breaks the way some fields work in Sitecore - nor have I seen a note that it was fixed. It might exist somewhere; see "searching" above ;-) Or what about Page Editor acting up and throwing exceptions if you happen to run Sitecore in a non "en" language context? Or breaks if you don't set a layout for the Print device? I would much rather wait an additional 3 months for any Sitecore release, if it means it has been fully implemented throughout the Sitecore organisation before being released to the general public. This will not cover all usage scenarios of Sitecore as a product - but it WILL get rid of "all those little annoyances" that are so very hard to explain to our customers. "Why does my image links break, if I do it this way instead of that way?" and so on. Fortunately, level 1 support is behind me - but then.. should we accept these kinds of issues at all?

Finally, and just for good measure, I will throw in a product feature comment. The Page Editor. Brilliant, but hugely underestimated right now. Find a good way to solve the problems of configurable controls (a.la SharePoint WebParts) getting easily added to layouts - and come up with some form of layout or rendering inheritance as pointed out by another commentator. Not necessarily saying the suggested approach is the best one (although it very well might be), but it certainly is something that I would like to see addressed and solved. In summary:

1. Document. Love your developer base, please ;-) Take a look at what the competition is doing, maybe expecting a new MSDN is asking too much after all ;-)
2. Eat it yourselves. First.
3. Keep evolving the Page Editor, which I honestly believe can be one of the absolute killer features and opens up a lot of opportunities for new development areas.

That's not asking for much is it? ;-) We were invited however.

Sitecore Packager throwing System.IO.IOException: The file exists.

Just had a bit of a curiosity when attempting to create a package using the Sitecore Packager.

Essentially it would give up on me, and tell me that the file existed. Stacktrace revealed:

[IOException: The file exists.] System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) +2056933 System.IO.__Error.WinIOError() +30 System.IO.Path.GetTempFileName() +2715024 Sitecore.Shell.Applications.Install.Commands.BuildPackageCommand.Execute(CommandContext context) +106

And so on. So looking at this, turns out the packager isn't really to blame here. Except maybe for the fact that it should have cought this exception and given a better error message :P

Looking into my C:\WINDOWS\TEMP folder revealed:

And further looking at MSDN reveals:

The GetTempFileName method will raise an IOException if it is used to create more than 65535 files without deleting previous temporary files.

The GetTempFileName method will raise an IOException if no unique temporary file name is available. To resolve this error, delete all unneeded temporary files.

A bit of housekeeping, and voila

All good again :-)

CorePoint.DomainObjects Update 1

Just posted a minor update of CorePoint.DomainObjects onto Sitecore Trac. Not much new under the sun, a bug fix and a few simple helper methods. 1) It is now possible to create a field such as: [Field("myImageField")] public Sitecore.Data.Fields.ImageField MyImageField { get; set; } This was always the intention, but implementation was faulty. It now works - knock on wood ;-) 2) StandardTemplate knows a couple of new tricks. .IsAncestorOf and .IsDescendantOf. They have similar functionality as Sitecore's own versions of same methods. Update 1 for Sitecore 5 Update 1 for Sitecore 6 Enjoy :-)

Update-3 (080912) Released

While doing my rounds on SDN I came across a new update for Sitecore 6. Unfortunately it doesn't appear to fix many (if any) of the issues that has been noted - some of which are showstoppers in my opinion - but it's better than nothing I assume ;-) Hopefully the list of known issues will diminish and we'll see Sitecore 6 reaching bug convergence this year.

Return to the not-so cacheable Control in Sitecore 5.3

Ok so,

It would appear that my previous post on this subject somehow managed to misdirect the focus from what was the original intention; point out the very troublesome issue of Sitecore Support's policy of recommending Sitecore 6 upgrades for Sitecore 5 problems.

The issue at hand was that of a certain Control not being cached. Well it was, but the code was also executed. Although asked specifically for workarounds, no alternatives to the upgrade route was presented. A short while spent in Reflector, a few config files and so on, leads me to believe there could be a workable solution.

This has NOT been tested outside my sandbox testing environment however, any feedback would be appreciated.

The "CachingEnabledControl". Inherit from this one and your work is 90% done.

Now all you need to do, in top of your CreateChildControls() method, is check if Sitecore has a cached copy of your control that it intends to show.

public class MyCacheableControl : CachingEnabledControl
{
 protected override void CreateChildControls()
 {
  if ( IsSitecoreCached )
   return;

  // Normal functionality here

Given the same setup as described in the original post, we now no longer incur the long delay (.Sleep()), and Sitecore outputs it's cache just fine.

Please DO keep in mind, this is probably UNSUPPORTED and MIGHT NOT WORK FOR YOU AT ALL - for whatever reason :P

Here's the source file. MyCacheableControl.cs

How do you DO?

As someone recently commented, this blog is not overall a very positive one. And while it was never my intention to write a fanboi blog, it doesn't have to be all negative either ;-)

Overall, I have grown to like the Sitecore product quite a lot. I find, that I am able to apply it to most any scenario my clients throw at me (though there was this one client of my former employer, who wanted the CMS to realtime translate her articles into all of the other installed languages... :P) - It helps me get the job done.

Now I am sure that most people who work implementing Sitecore solutions on a regular basis has adapted a way of their own. Either in the form of socalled "Helper" libraries, or maybe a drawer full of ready to go renderings for topmenus, left menus, breadcrumbs and whatnot. Me, as I often work with Sitecore jobs that involve tight integration with Sitecore data and external data (either in form of added Sitecore functionality, or migration from third party systems for instance), my most important tool in the drawer is the ability to work following modern OO practices - Domain Objects if you will.

I decided it is time for me to share something with the Sitecore community that makes it easy to do just that. Basically, this is my "5th generation" data layer if you will, although I would probably huff over anyone classifying this as generic "DAL".

Not only is it being shared, it is being shared in full. It is being released under the GNU General Public License v3, sources and all, in the hope there are other people out there that can see the benefit of the approach I suggest and will find it useful.

Here's a quick snippet of how Sitecore integration looks, when using it:

Enough rambling for now. Head on over to the official site in the Sitecore Shared Source Library to grab your copy, and to have a look around.

• Trac space for CorePoint.DomainObjects
• Getting started guide
• Release blog post from Jimmie Overby, Sitecore Shared Source Coordinator
• Entry point on SDN

Say goodbye to Sitecore 5.3

No really. Say goodbye. What, just 3 weeks ago, was "designed to drive the largest sites on the planet. ... ... deliver compelling user experiences at lightning speeds using a combination of high availability options, intelligent caching controls, and the power of ASP .NET 2.0" (ref: http://www.sitecore.net/Products/Sitecore%20CMS.aspx) is now yesterday's news.

As I am sure any reader of this blog will know, the new Sitecore CMS flagship, Version 6 (formerly or maybe still known as Crestone) launched on 30th of July 2008, and everyone at Sitecore seems well pleased with it. They have every right to be, don't get me wrong... After what happened "back in the day" when Sitecore 5 was first launched, the transition to Sitecore 6 promises to be a much smoother ride indeed. It looks good, congratulations :-)

This post isn't about Sitecore 6 however. It's about what happens now, to you, to your clients, to your existing Sitecore 5.x solutions.

If it wasn't for the fact that I don't particularly want to spend my entire time blogging on potential and concrete issues in Sitecore 5.x; I could probably still make this a full time profession. Known issues, hotfixes, regression and non-regression tested 5.3 builds...

The question I'm asking is; will we ever get to see a stable 5.3 release now that all the spotlight has moved on to the new emperor's clothes? Surely there must be hundreds if not thousands of 5.x sites out there, and surely one cannot reasonably expect all of them to be willing to invest quite substantially in the upgrade to 6.x just to get a bugfree (or well... tolerable) product?

First line of defense from Sitecore Support seems to see this as perfectly reasonable. Take this recent scenario that played out. You can reproduce this yourself very easily. I tested this on Sitecore 5.3.1 Build 071114 - the only current Sitecore "Recommended Release" (you running 5.3.2?....)

Make yourself a new control, like this:

Simplest control ever. It outputs the current time. Now register this control on a sublayout.

Go to your Layout settings on the sublayout you registered the control on, and make sure caching is enabled (you DO do this by default, I take it..?).

Make sure you've published, and go take a look. Not surprisingly, the Sleep( 10000 ) in there causes quite a delay. Pretend this is your extremely heavy backend integration code executing. After the delay has passed, your page comes up.

Quite as expected. Now click refresh... This is where you start speculating if you did remember to configure caching correctly. I mean, after all, it is quite an involved process with lots of little things to tweak.

The 10 seconds will pass, and voila... Erm.... Well.... there you go, your cached output - again.

So Sitecore realises, AFTER deciding to render your control again (and therefore, running your extremely heavy backend integration code once again), that "aaah... ok let's disregard, and just use what's in cache instead.

Makes a lot of sense doesn't it? ;-)

In case you didn't follow. Sitecore will execute everything, thereby allowing your webserver to consume expensive resources (and even stop and wait for the result), then dump the result and use what it has in cache already.

Mmhmm.

Admittedly, caching and the way Sitecore has to implement it is very tricky stuff indeed. Am not disputing that there's a bug here (... but has this ever worked?). What really got me ticking was the response I got from Sitecore Support on this matter.

"Hi Mark,I have checked this and indeed Sitecore will return cached output but control code will be executed. This was fixed in Sitecore 6.Best regards"

To which, I imagine, there can only be one response; "So?". I have requested information on when a 5.3 fix will be available. Was told this has been put forward to developers, so we'll see what comes of it.

I am stunned however, at how easily the "support decision" was to proclaim that a switch between major versions was the way to go. I don't know about you guys, but I would hate having to go to my customer and go... "You know... I know that 3 weeks ago, Sitecore was the king of kings, intelligent caching and so on. But you see... well... it doesn't really work like intended. Don't worry however, your £75.000 investment is safe and sound. All you need to do, is invest in Sitecore 6 training for your 8 editors (who only NOW are getting to know the system), a simple quick £10.000 upgrade project, possibly a small license increase and THEN we're in business. Really, we are."

Sitecore Support was right, just for the record. The problem described here does not manifest in Sitecore 6.

Is the new mantra really; "Our website is running on it [Sitecore v6] - Why isn't yours?" ?

is not a valid Win32 application. (Exception from HRESULT: 0x800700C1)

While tranferring my development environment onto my newly installed Windows Vista x64, I came across the above somewhat cryptical error. Exception Details didn't exactly prove useful either: Exception Details: System.BadImageFormatException: is not a valid Win32 application. (Exception from HRESULT: 0x800700C1) A little Yahooing quickly provided some insight. Sitecore 5.3 doesn't cope all to well when being executed as an x64 IIS Application. The essence of the article, came down to just this: cscript %SystemDrive%\inetpub\AdminScripts\adsutil.vbs set w3svc/AppPools/Enable32bitAppOnWin64 1 However, the article is fairly old. And the admin scripts he is referring would not normally be part of your Vista IIS 7 installation. You need to install the IIS 6 Compatibility scripts for it.

Executing the command line and a quick IISRESET just for good measure, and all the problems went away. Now I doubt running Sitecore 5.3 in this way is really supported, but it gets the job done. I will keep a lookout for side effects.

Web Application Project revisited

The most recent Sitecore project I was involved in, was quite a beast. Not in terms of the project itself, but how we had to cope to handle it in our daily working environment. The client had a handful of sites in the solution, all of them sharing functionality to some extent, and then each site had its own unique functionality bits as well. Now as I'm sure most anyone who's ever worked with Web Application Project in a VSS environment can imagine; a solution such as the one we had our hands on there, gets fairly big. And then imagine 2-3 separate teams working on the solution at the same time - getting exclusive access to the ever so central Project File becomes problematic in ways that most of all remind me of The Dining Philosophers. I am no fan of the Web Application Project, never have been and never will be. So I thrawled through SDN, just to see if there were any new developments in this area. I came across this one; VS2005 and Sitecore 5.3 setup without Web Application Project. Nice. I'll summarize what it says - basically it instructs you to move the "problematic" areas of your Sitecore solution outside your project scope (much like your /Data folder), and voila. I don't know though, and the article points this out as well... How will various modules and tools react to this somewhat dramatic change in structure? As I don't really have the time to test this out in any detail, I decided to just give something a shot. I pulled this tip off the web some time ago (I forgot where, sorry). If, as the SDN article states, the problem is isolated to two folders; "/App_Config" and "/Sitecore", there is a fairly quick and simple workaround that appears to be working - and is by far, less intrusive than moving the folders out of the directory structure entirely.

1. Configure your explorer (not IE, your Windows Explorer) to "Show Hidden Files and Folders" (I won't post screenshots, as my current Vista installation is in Danish)
2. Go to your root "/Website" directory for your solution. Mark the two folders; "App_Config" and "Sitecore", rightclick, and check the box "Hidden" (found right below readonly). It will ask you if this should apply to subitems as well, just say no to that.
3. Fire up your favorite Visual Studio (I just tried this with Visual Studio Express 2008 and it worked fine), go "Open Website" (or "Open Website in Existing Folder", all depends a bit on what VS version you are using)

And there we go.

From here, I rightclicked the "Website root" and selected "Add ASP.NET Folder" -> "App_Code", tossed a few class files in there, made up a few references to this code from both .aspx pages and .ascx sublayouts... it all ran fine, and I never needed to manually compile (ah, the good old CTRL-SHIFT-B) once.

Does this qualify as conclusive evidence this will work? Absolutely not. But if the method of moving the folders out is "under evaluation", maybe this should be as well? Seems to me to be very straightforward.

I'll work with it for a while, see if I bump into any unexpected mishaps.