Feed on
Posts
Comments

Marketing Punctuated Equilibrium

For all you, developers, techies, web analysts, designers, quality engineers and project managers in the internet industry. The following page is a real must read! As you can see on this picture, the marketing organizations are on loose ground. It’s time to help them, but before we can do that, we probably need to dive a bit into their world…

http://chiefmartec.com/2013/03/agile-marketing-for-a-world-of-constant-change/

The easiest way to rebuild the indexes of your DMS database is done in the following way:

SQL Script
  1. SET NOCOUNT ON
  2. GO
  3. –Set the fillfactor
  4. DECLARE @FillFactor TINYINT
  5. SELECT @FillFactor=80
  6. DECLARE @StartTime DATETIME
  7. SELECT @StartTime=GETDATE()
  8. if object_id('tempdb..#TablesToRebuildIndex') is not null
  9. begin
  10. drop table #TablesToRebuildIndex
  11. end
  12. DECLARE @NumTables VARCHAR(20)
  13. SELECT
  14. s.[Name] AS SchemaName,
  15. t.[name] AS TableName,
  16. SUM(p.rows) AS RowsInTable
  17. INTO #TablesToRebuildIndex
  18. FROM
  19. sys.schemas s
  20. LEFT JOIN sys.tables t
  21. ON  s.schema_id = t.schema_id
  22. LEFT JOIN sys.partitions p
  23. ON  t.object_id = p.object_id
  24. LEFT JOIN sys.allocation_units a
  25. ON  p.partition_id = a.container_id
  26. WHERE
  27. p.index_id IN ( 0, 1 ) – 0 heap table , 1 table with clustered index
  28. AND p.rows IS NOT NULL
  29. AND a.type = 1  – row-data only , not LOB
  30. GROUP BY
  31. s.[Name],
  32. t.[name]
  33. SELECT @NumTables=@@ROWCOUNT
  34. DECLARE RebuildIndex CURSOR FOR
  35. SELECT
  36. ROW_NUMBER() OVER (ORDER BY ttus.RowsInTable),
  37. ttus.SchemaName,
  38. ttus.TableName,
  39. ttus.RowsInTable
  40. FROM
  41. #TablesToRebuildIndex AS ttus
  42. ORDER BY
  43. ttus.RowsInTable
  44. OPEN RebuildIndex
  45. DECLARE @TableNumber VARCHAR(20)
  46. DECLARE @SchemaName NVARCHAR(128)
  47. DECLARE @tableName NVARCHAR(128)
  48. DECLARE @RowsInTable VARCHAR(20)
  49. DECLARE @Statement NVARCHAR(300)
  50. DECLARE @Status NVARCHAR(300)
  51. FETCH NEXT FROM RebuildIndex INTO @TableNumber, @SchemaName, @tablename, @RowsInTable
  52. WHILE ( @@FETCH_STATUS = 0 )
  53. BEGIN
  54. SET @Status='Table '+@TableNumber+' of '+@NumTables+': Rebuilding indexes on '+@SchemaName+'.'+@tablename + ' ('+@RowsInTable+' rows)'
  55. RAISERROR (@Status, 0, 1) WITH NOWAIT  –RAISERROR used to immediately output status
  56. SET @Statement = 'ALTER INDEX ALL ON ['+@SchemaName+'].['+@tablename +'] REBUILD WITH (FILLFACTOR = '+CONVERT(VARCHAR(3), @FillFactor)+' )'
  57. EXEC sp_executesql @Statement
  58. FETCH NEXT FROM RebuildIndex INTO @TableNumber, @SchemaName, @tablename, @RowsInTable
  59. END
  60. CLOSE RebuildIndex
  61. DEALLOCATE RebuildIndex
  62. drop table #TablesToRebuildIndex
  63. Print 'Total Elapsed Time: '+CONVERT(VARCHAR(100), DATEDIFF(minute, @StartTime, GETDATE()))+' minutes'
  64. GO

This script is particular usable if you have a huge database since it optimizes the small ones first.

Example output:

Code Snippet
  1. Table 22 of 37: Rebuilding indexes on dbo.ReferringSites (40474 rows)
  2. Table 23 of 37: Rebuilding indexes on dbo.Locations (131276 rows)
  3. Table 24 of 37: Rebuilding indexes on dbo.Keywords (267301 rows)
  4. Table 25 of 37: Rebuilding indexes on dbo.UserAgents (337697 rows)
  5. Table 26 of 37: Rebuilding indexes on dbo.GeoIps (1573434 rows)
  6. Table 27 of 37: Rebuilding indexes on dbo.Cache_TrafficByDay (1755931 rows)
  7. Table 28 of 37: Rebuilding indexes on dbo.VisitorTags (1985036 rows)
  8. Table 29 of 37: Rebuilding indexes on dbo.AutomationStates (1987782 rows)
  9. Table 30 of 37: Rebuilding indexes on dbo.Profiles (3315763 rows)

Enjoy!

Inside Sitecore we work with Scrum. This simply means that we have the business prioritize business cases and technical teams try to implement them as good as possible. Over the years we’ve learned a lot about application development and one of those things was that we’ve to prioritize technical tasks in a similar fashion as business cases. Obviously these don’t go always hand in hand. A simple update of the Sitecore platform(for example Update-1 to Update-3) isn’t an obvious task for the business. In this particular case, there were only two new features in the platform:

  • New Channel Performance report inside the Insight Dashboards which shows which channels drive most value.
  • The Content Editor allows you now to display [shared] and [unversioned] to end users, to simplify translation projects

When we did the upgrade, we saw a decrease of performance on our production environment. I’ve heard from many sides internally the unfunded conclusion that ‘6.6 Update-3’ is a bad and slow release. Interesting thought… But reality was different. Last Monday we released another version of the website, but with the same version: 6.6 Update-3. In the end of the story, if you give people new features, you can never roll back.

The cause seemed to be in the combination of some new relative slow features, infrastructural problems and deprioritized technical tasks. Let’s focus on the last ones. Let’s assume that the Technical Task was something along the lines ‘optimize the custom item manager’. As a business user it’s quite hard to judge whether this adds value. A developer never tells you that it’s causing a 25% decrease of your website.

So what can you do to solve this challenge? I guess we all do know that we can do performance testing. But that in itself doesn’t change the mind of the business owner, who wants to have a quick time-to-market. The solution for that is to agree on the experience. A simple way is to decide together with your business owner what is allowed. For example: every server page needs to be generated and transferred in less than 500ms. This allows a browser load of less than 3 seconds.

Out of the sudden, this small technical task makes sense for the business user, because a developer talks his language. It’s a no-brainer but it happens a lot. It’s not related to the project process you’re using. Scrum has a Definition of Done while using Prince II, you’ll talk about deliverables.

Still it doesn’t justify refactoring of your code. It’s obvious that a business owner doesn’t see the value in that. It can even slow you down a lot! There are easy ways to show technical debt like this in relation to team performance, but you need to have a lot of tooling before you can make this visible. Most of the time you don’t have those, so let’s make it more concrete.

Refactoring is all based on making the same thing do the same job, but in a more maintainable way. If you’re doing code and performance optimizations at the same time, you’ll end up in the situation where the system performs well, but you’ve broken all functionality. So split it. Refactoring shouldn’t optimize stuff, but is polishing for you and your team mates.

Still it doesn’t justify why you’re doing it. But it makes it easier to explain to your boss or business owner: I made it maintainable.
Interestingly enough, that’s enough explanation for your boss or business owner. As part of your work, you’ve introduced 3 things:

  • Meeting the desired functionality (the feature works)
  • Meeting the desired quality standards (it performs well, it looks good, it validates, etc)
  • It’s ready to enhance further (because it’s maintainable)

So let’s assume they still don’t take it. Then make a list of the crappy stuff and explain in this list why you didn’t do it. Every time in the near future when you’ve to touch an area which is already build, show this list and show what you need to do to start working on it. In the end of the story it’s crappy code you’ve to work with.

Estimations will increase when you touch it again. The shock effect when you explain this is big. You can see it in the eyes of anyone who owns the budget where you are working with. You can go in the ‘told you so mode’. You’re allowed to do this once. Next time, propose to do it straight away, but as part as a simple flow when creating a new feature:

  1. First: Meet desired functionality
  2. Meet desired quality
  3. Meet desired code quality
  4. Do the check-in

Note: I make some shortcuts here. I do ignore all stuff like Test Driven Development, BDD, Pair Programming, etc. Why? Because they don’t add value to your business. Tests are cool for you. Because you like Green Checkboxes. The Business likes money. So what? Meet each other in the middle. Explain to them in a simplified way why you do stuff. Why Technical Debt matter. Why performance agreements matter. Make it visible. Fail on purpose. Learning is done by failing, not by showing I did according to the book. It’s sad but it’s damn true. But please, try to fail before you go to production Winking smile

 

Thoughts?

In the operations around Sitecore.net we often get contacted by someone who has a problem with a link. Recent example of that is when we started to enable the CSRF-extensions for Sitecore 6.6(Update-3). All kinds of redirect problems were caught, but how to figure out to which exact mailing they are related?

Don’t worry, that’s easy… As long as you have database access… Smile

So let’s assume we have the following link:

http://www.yoursite.net/sitecore/RedirectUrlPage.aspx?ec_url=http://www.sitecore.net&ec_camp=15A4275B9C284B62BC8379D4691A5D64&ec_as=00000000000000000000000000000000

The steps to get to the Campaign:

  1. Take the ec_as-query string parameter
  2. Execute the following script to the database. Where @guid is your ec_as-parameter

    DECLARE @guid VARCHAR(50)
    SET @guid = ’00000000000000000000000000000000′

    SELECT [AutomationId]
      FROM [Sitecore_DMS].[dbo].[AutomationStates]
      WHERE [AutomationStateId] = CAST(
            SUBSTRING(@guid, 1, 8) + ‘-’ + SUBSTRING(@guid, 9, 4) + ‘-’ + SUBSTRING(@guid, 13, 4) + ‘-’ +
            SUBSTRING(@guid, 17, 4) + ‘-’ + SUBSTRING(@guid, 21, 12)
            AS UNIQUEIDENTIFIER)

  3. Now you have the AutomationId which is the same as your Automation Plan in the back-end. You can simply search for the Item in the Content Editor.
  4. Once you’ve found this item, you can got to the Navigate Tab –> Links –> Referrers –> All campaigns that use this automation

So basically what have we done? We used the Automation State to get back to the original Campaign. Sounds easy doesn’t it Smile?

As you could have been reading in my previous post, Sitecore’s major product Sitecore .NET Web Content Management System is affected by the ASP.NET Vulnerability 2416728. Today, Microsoft has released an update to all the Windows Update Services(WSUS and WU). You can find all the details in Scott Guthrie’s post.

What could potentially happen to your Sitecore installation when applying this patch?

  1. A couple of users get logged out. That’s only really notable for the user who use the check box ‘Remember me’. Just login again. That’s the deal. According to my checks on 6.2, this does not happen.
  2. Potentially you could get an exception for people who have a open session while updating Sitecore. According to my checks, this doesn’t happen.

So it seems that you can install this patch without any unexpected side effect. I’ve checked it on my machines and tested it there(so this is an unofficial Sitecore  statement). If we come across issue, we will let you know. If you come across issues, please let our support team know.

Update: Sometimes an ‘HttpException: Unable to validate data’ happens when people are already authenticated. I couldn’t reproduce this. But it seems to be there. In that case, please review this solution on SDN.

Sitecore’s major product Sitecore ASP.NET Web CMS is affected by the ASP.NET Vulnerability 2416728. Scott Guthrie describes all the nifty details about this threat in this post. Our Support-team has been working intensively with our Product team to get a well tested solution out. It’s available now. 

For more information, please review this article on SDN.

Sitecore is moving a lot these days and releasing almost on a monthly base new and cool functionalities. Not everybody has the ability to work with Sitecore on a daily base an keep up with this new exciting functionality. So time for me to step in and guide you back to the highway by creating small and very focused blog posts. This is the first one about Webforms for Marketers and the Dynamics CRM Campaign Integration.

The following diagram shows how the CMS, OMS, Webforms for Marketers and Dynamics CRM Campaign Integration glue together: CRM Integratie

On top of the Sitecore Web CMS you can find the Sitecore Online Marketing Suite(OMS) which is used for gaining more details out of the visitor experience.

The Webforms for Marketers module is added on top of the OMS. It relies heavily on the OMS, so I’ve decided to add it there.
Eventually you can run this module without the OMS, but you’ll definitely not gain all the potential out of it.

For integration with MS Dynamics CRM, we have our new Dynamics CRM Campaign Integration module. This module consists of 2 parts:

  1. Save Actions: An extension to the Webforms for Marketers module. It allows any business user to import data into any CRM entity. All WYSIWYG. So no coding required.
  2. Security Provider: Share Contacts in Dynamics CRM as Users in Sitecore. This means that a contact in CRM can login into your Sitecore website for extranet purposes.

Let’s assume you want to start developing on a similar solution. Then it’s highly advisable for you to setup a sandbox environment. You need the following components:

Then you’re ready to rock!

So far so good for today. We’ve got plenty more to talk about. But that can wait for later.

A very common question I receive is that people don’t know anymore where to setup Insert Options. Especially with the new possibilities of the Sitecore 6.1 content management system.

Here’s my view on this:

  1. Individual item
    Rarely, you want to setup Insert Options on individual items. Something when you have a specific folder or section in your website, you might chose for this option. Keep in mind that this you always should have a valid reason.
    For example: For all my meta data I use the /Common/Folder template to store my item underneath. As these folders have only 1 specific setting(Insert Options), I’ve decided not to Subtemplate(create a new template and inherit from the Folder-template) the Folder-template, because of maintainability. Consider in that case Insert Option Rules.
  2. Branch Templates
    It’s a bad practice to set any Insert Options on Branch templates.
  3. Standard Values
    95% of the cases setup your Insert Options on Standard Values as it’s applied everywhere when you use the specific template.
  4. Insert Options Rules
    When you need to setup global conditions, like ‘All the items based on the Folder template containing the word ‘Blog’ should add the Blog Entry template as Insert Option’.
    We use it ourselves in the Marketing Center. We allow everybody to create folders, but we want them to create specific items underneath.
    Before setting up a rule, figure out if you can’t do it with Standard Values.

Trimming the list afterwards can always be done by settings up Insert Rules.
For an advanced Reference regarding these Content Infrastructure topics, please refer to Data Definition Reference.

PDC09  has started. Unfortunatelly I’m not there. But no worries! Everythings get recorded by the Channel 9 chaps.
I thought: let’s share my recommended movies / the movies I’ll watch :).

So here we go:

  1. Data Programming and Modeling for the Microsoft .NET Developer
    Don Box! The main guy behind Oslo, someone you should always watch!
  2. Microsoft ASP.NET 4 Core Runtime for Web Developers
    That’s my job isn’t it? ASP.NET 4 is upcoming, .NET 4 is upcoming, so we as Sitecore should remain running!
  3. Code Visualization, UML, and DSLs
    Very interesting way of looking at code is to Visualize it and make it more human.
    Definitelly one to watch.
  4. Manycore and the Microsoft .NET Framework 4: A Match Made in Microsoft Visual Studio 2010
    Many core is the main issue these days. This video brings a lot of the great work done by MSFT together.

All together 4 hours of watching. Have fun :)

Updated: Now with working links. Sorry!

An annoying issue which came across twice this week: Sitecore was throwing “Specified cast is not valid”-Exceptions:

"Specified Cast is not valid"-exception

And seriously, if there’s one thing I hate about .NET in general, it are exceptions. 95% of the time they are non self explaining and sometimes even nonsense. This exception would normally take me 20 minutes to solve at least. But not this time. So how to solve this one?

Basically, this is nothing more than a Database Connection error… When I added /sitecore behind my url, I received a generic connection failure exception. I had to do some work on my connection string(or start my SQL Server). That’s all. But Alex, where did it come from? And why does the login page show something different?

Chicken and the egg issue: when accessing an Item we need a (SQL)-Database, but before allowing to access an Item, we need a User, a User needs a (SQL)-Database. And of course, this dependancy became stronger when we introduced Sitecore v6 with the Membership Providers.

So what can we learn from this? Always consult both pages: / and /sitecore. Often, there’s a different message. Sitecore is all based on context(yes, Sitecore.Context…). So when you have a problem, always try different entry points to minimize the Context. For example, Sitecore has by default 2 active websites running: ‘website’ and ‘shell’. In this case, I tried to access both of them.

Happy coding out there :)

Older Posts »