Disabling Lucene indexes (programatically)

Over the past 18 months or so, my work has taken me around a great deal of integration jobs. As those of you who follow this blog will know, integration of external data into Sitecore is something that has often been explored (and still is).

Recently, while faced with the task of importing, cross-referencing, IDTable mapping – about 14.000 items on a regular (scheduled) basis, I began to look for even further ways of improving integration performance.

One thing I put my attention towards, was entries like these in the log files, while the import was running:

3796 10:34:59 INFO  'ImportCategories' Executing
ManagedPoolThread #2 10:35:13 INFO  Starting update of index for the database 'master' (7 pending).
ManagedPoolThread #2 10:35:13 INFO  Update of index for the database 'master' done.
ManagedPoolThread #5 10:35:13 INFO  Starting update of index for the database 'master' (8 pending).
ManagedPoolThread #5 10:35:13 INFO  Update of index for the database 'master' done.
ManagedPoolThread #4 10:35:13 INFO  Starting update of index for the database 'master' (9 pending).
ManagedPoolThread #4 10:35:14 INFO  Update of index for the database 'master' done.
ManagedPoolThread #4 10:35:14 INFO  Starting update of index for the database 'master' (10 pending).
ManagedPoolThread #4 10:35:14 INFO  Update of index for the database 'master' done.
ManagedPoolThread #4 10:35:14 INFO  Starting update of index for the database 'master' (1 pending).
ManagedPoolThread #4 10:35:14 INFO  Update of index for the database 'master' done.
ManagedPoolThread #4 10:35:14 INFO  Starting update of index for the database 'master' (1 pending).
ManagedPoolThread #4 10:35:14 INFO  Update of index for the database 'master' done.

So basically, as the items were being imported, the indexer ran along in the background desperately trying to catch up. Well maybe not desperately… ;-)

I started looking around for an official way to disable these indexes. The idea being, disable the indexes – run the integration – enable them again and let them catch up. Has to be more effective yea?  Kind of like harddisk trashing otherwise, for those of you who still remember that phenomenon ;-)

Alex Shyba came up on top of most searches I did, describing how to disable Lucene indexes. The post is minded towards how to improve performance in the production environments however, and not to mention it’s well over a year old by now. Sitecore has been doing a lot of work on the Lucene/Sitecore search API, and odds are that things may have changed in the year that passed.

Besides, I needed a way to do it programatically.

I started looking into ways of achieving this. Basically exploring APIs, configuration settings, even considering … patching-via-reflection into the Sitecore Patching Configuration API to comment out sections of Sitecore Configuration *grin*.  Fortunately, before exploring any of these options in great detail, I had the common sense to approach the excellent Sitecore Support team to get a better (and more importantly, supported) solution. And fortunately, as it turns out, there is one. Clean, and very very simple. Thanks guys :-)

Sitecore.Configuration.Settings.Indexing.Enabled = false;

Couldn’t be simpler :-)

And the configuration equivalent exists as well, even if it is not shown in the 6.3 default web.config.

<setting name="Indexing.Enabled" value="false" />

Sitecore 6.3, ComponentArt, WebForms For Marketers, and you

Apologies for the almost complete silence in this blog, but Real Life(tm) has been taking me for quite a spin these past months.

Just a quick heads-up, in case you run into the same issue that I just had.

After installing Sitecore 6.3 (100716), WebForms For Marketers 2.1 (100629), I ran into the following issue when attempting to edit values on a DropList Field.

“Could not load file or assembly ‘ComponentArt.Web.UI, Version=2007.1.1617.2,Culture=neutral,PublicKeyToken=9bc9f846553156bb’ or one of its dependencies. HRESULT: 0x80131040”

It seems, Sitecore 6.3 comes with an upgraded version of the ComponentArt library, and version information on this new assembly reads 2010.1.2637.35. Also seems that WFFM is compiled against the previously distributed version, and fails because of it.

The fix I applied is relatively straight forward, and should work for you as well. Add a new assembly dependency mapping in your web.config as

<dependentAssembly>
  <assemblyIdentity name="ComponentArt.Web.UI" publicKeyToken="9bc9f846553156bb" />
  <bindingRedirect oldVersion="2007.1.1617.2" newVersion="2010.1.2637.35" />
</dependentAssembly>

And the problem goes away. I don’t know if the assembly versions are fully compatible, so don’t come banging on my door if this fix doesn’t make all potential problems go away ;-)   It appears to work like a charm here, in our solution.

Issue has been raised with Sitecore Support (331878).

Partial Cache Clearing in Sitecore

For a very long time, some very serious performance issues have affected certain types of Sitecore deployments. It has to do with scaling a Sitecore solution into webfarms (multiple content delivery servers) and the use of the Sitecore Staging Module.

 

In very short summary, if the website is updated frequently (say, for instance, an editorial staff of 10 or so, posting relevant market updates) AND the website is under heavy load, most practical uses of the tool Sitecore recommends and supports will more or less kill performance on your SQL Servers or whatever storage mechanism you have in place.

 

I’ve never blogged in detail about this issue, as I didn’t have a client myself that was affected directly by this issue. Paul George has blogged about it in detail however, and if you want to learn more about what this issue is and if it could be affecting you, take a look at these posts:

 

• Problems with Sitecore and the staging module part 1
• Problems with the Sitecore 6 staging module part 2 – extranet user loses session

 

So why write about it now?

 

The good news

Alex Shyba posted about a new Sitecore Shared Source module that was just made publically available; SitecoreStager – Sitecore Partial (item) Cache Clearing Module.

 

In short, instead of just uncritically clearing the entire cache on your target server, dropping user sessions, putting out the cat and so on – the SitecoreStager will instead execute a partial cache clear and in essence just clear items from the cache that were affected by the publishing operation.

 

This is very good news indeed, and for those clients who have been affected by this problem I am sure it’s tipped hats all ‘round :-)

 

The bad news

This is actually something that has been nagging me a bit for some time now, and has just been refreshed by this release.

 

It is Shared Source.

 

Don’t get me wrong. I think the Sitecore Shared Source Library is an excellent idea. I have code in there myself, and it’s a perfect place to go look for those “there has to be someone else who had this problem and solved it” code snippets and field types and whatever it may be.

 

But it is also, for obvious reasons, unsupported by Sitecore themselves. I mean, how could they?  Half the modules and code pieces in there come from independent sources like myself – and it wouldn’t be reasonable to expect Sitecore to support them.

 

But what about the code Sitecore release to the library themselves?

 

Would you not agree; “With Sitecore, you can have a team of editors publishing content to an enterprise level site, and performance will still rock” is a statement you’ll find (albeit probably not verbatim) in Sitecore marketing material?   But it is unsupported?

 

I feel somewhat the same for the Multiple Sites Manager…  shouldn’t a standard Sitecore have a solution for regular (advanced) editors to set up new websites without having to edit configuration files?  Or is that limited to Foundry licenses only?

 

What I mean is

Sitecore marketing materials doesn’t exactly help anyone much, when they boast “Comes with Blog Modules, Wiki Modules and even Microsoft Dynamix and Microsoft BizTalk integration (yes…)”; yet offers little or no actual support other than pointing the users, the licensees, the Sitecore implementors (like myself) in the direction of the Shared Souce Library, shrug and go… “From here, you’re on your own”.

 

Kudos for making this module. I think this belongs in the core package however, fully supported and updated with every Sitecore release. That’s my 2 cents worth.

 

 

.

Code Monkeys?

Before I begin, I’m breaking routine here – this post is in no way Sitecore related.

 

So here I was, reading through my blogroll, catching up on bits and bobs from around the world. And then one of the sites decides to confront me with the following ad:

 

 

I am, by own admission, quite possibly a bit over sensitive to the idea that developers are interchangeable code monkeys. I’ve worked in the internet industry pretty much since the inception in the mid 90s, and I will never subscribe to the notion that one developer (me) can easily be replaced by a developer working offshore for as low as $ USD 7.00 a day.

 

But then again, maybe that’s just me ;-)

 

Am I missing a point here?