All posts by rangler2

Customising Sitecore Experience Analytics Dashboard

We had a request to customise the default Experience Analytics dashboard on a Sitecore 8.1 site. With a little guidance from Sitecore we have been able to do this easily without development, but it is a bit of a convoluted process so I thought I would outline it here.

If possible, get your site live with Analytics configured correctly and capturing data well before you start looking at customising it, as you can then browse around the default analytics pages and see the real data that is captured. You can then give an analyst access and ask them what changes would be useful.

The default dashboard looks a little like this, on a site that hasn’t had any Analytics customisation done:

All the blocks apart from are ‘by value’ which means there is no data because no profiles / personas / experience scoring has been configured. Therefore, our analyst asked if the dashboard could be changed to include more common analytics like you might find on a Google Analytics dashboard.

  • Total Visits (line chart)
  • Top Referring Site (bar graph)
  • Top Entry Pages (bar graph)
  • Pageviews + Pageview/Visit (line chart)
  • Top pages (bar chart)
  • Top pages over time (line chart)
  • Top 10 pages (table)

These are all graphs that she found in other Analytics report pages, so all we had to do is figure out how to remove the existing charts from the dashboard and copy these from the other pages, creating a new dashboard layout.

The most relevant Sitecore documentation discusses how to create a new report page with a single chart. Using this and a bit of Sitecore knowledge you can figure out how to edit an existing complex dashboard.

The analytics reports structure is stored as Sitecore items in the core database. You can view it by using Sitecore Desktop (link from the Sitecore launchpad) and switch to Core DB using the link at the bottom right:

Then open Content Editor and browse to /sitecore/client/Applications/ExperienceAnalytics/Dashboard

You can see that beneath Dashboard are all the report folders (Audience, Behaviour, etc – which are just common Folders) and report pages (Overview, Devices, etc – using template /sitecore/client/Speak/Templates/Pages/Speak-DashboardPage)

Each report page has a PageSettings folder (template = /sitecore/client/Speak/Templates/Pages/PageSettings) that contains an item underneath it for every chart on the page. These items allow you to set the chart title and any other settings such as which metrics / dimensions to use on the chart.

Review the Layout Details for any page with multiple charts and you can see it is made up of a series of Rows and Columns which define the page layout, and renderings for each chart.

Click Edit and you can see the placeholder IDs used – now you can see how the page is put together. RowPanels are rendered one above the other (in Main.Content placeholder) and each RowPanel automatically defines a placeholder “RowPanel X.Content” where X is the row number. Then ColumnPanels are inserted into the RowPanel placeholders to set up however many columns you want to break that row into. In this example, one full width row, then two 50/50 rows.

The chart renderings themselves are inserted into the ColumnPanels using autogenerated placeholders based on the row and column number e.g. “ColumnPanel 3.2.Content” means this rendering will be shown in row 3 column 2.

To edit this becomes more interesting because you can’t simply edit this in Sitecore – Edit gives you a server error, and Change gives you an empty list of renderings:

So based on the Sitecore article we have to switch to Sitecore Rocks which is able to edit these. Install and connect to the Sitecore instance, then expand Core database to the same location. Select a report page, Right-click, Tasks, Design Layout:

Now you can edit the configuration of rows/columns by moving/adding/removing RowPanel and ColumnPanels, editing the Placeholder of each. Note that each ColumnPanel has a ‘GridColumns’ field set to 12 for full width, 6 for half width, 4 for 1/3 width, etc. Each RowPanel seems to be used for a single Row, not floating multiple charts into it.

The charts themselves use one of a number of Renderings all starting with ‘ExperienceAnalytics’ – you can copy/paste these using right click options from one page’s layout to another, or Add Rendering to pick one to add from scratch:

Each chart needs to have its Datasource set to a PageSettings child item using the matching parameters template:

Easiest is to find and copy the existing settings item from another report page, under the dashboard page, and then edit the datasource of the chart rendering item to use the new settings item.

Save regularly and reload the dashboard page, to see your changes. Here is my modified dashboard page where I have different charts:

Umbraco setup with media folder on an Azure File Share

We had a client who is moving their legacy Umbraco 4 and 6 sites to Azure VMs, and wanted to use Azure File Share for the shared media folder, rather than the recommended DFS setup.

After some trial and error we got this working (starting from a vanilla Umbraco site), by making sure the following:

  • Create a local Windows user account with the same credentials as the Azure File Share user and password
  • Set the IIS app pool user to the Azure File Store local user  – So Umbraco could access/write to the media folder
  • Set website > authentication > anonymous auth – to use App pool user (not IUSR) – So I could browse to files stored in the media folder
  • Create an IIS ‘Application’ for “media”  (not a virtual directory) – so I can stop the Umbraco web.config from applying to the media folder (see error below)
    • The path should be set to the Azure File Share folder e.g. \\myfilestorage.file.core.windows.net\myfileshare\media (you should map a network drive and create a ‘media’ folder first)
  • Update the web.config and wrap all the Umbraco stuff with <location path=”.” inheritInChildApplications=”false”></location> – to stop the Umbraco web.config from applying to the media folder (see error below)
    • The <configSections> and <runtime> sections must be kept outside of the <location> section.

We have uploaded a bunch of media without error and performed a stress test on the loading of media item via the load balanced front end, without any issues.

We have also checked this works with Umbraco 7 although when we upgrade to this we intend to use Blob storage instead. The Umbraco 7 default web.config includes some <location> sections which must also be moved outside the new <location> tag, and I had to also remove the legacy <httpModules> section to stop some wierd errors…

If you don’t do the last two steps, then it will work in Umbraco but when browsing to a media item you will see this error:

If you forget to set the site authentication to use App Pool Identity, then you will instead get a wierd error code 414 to do with the request URL being too long!

 

Using environment variables for configuration with VSTS build and release

29/09/17 – Updated to cover new build/release editor. Screenshots still show the old editor. Also sorry about the double quotes in the code snippets, they are being replaced with pretty ones and you have to use normal ones, I’ll sort it out sometime.

We are using Visual Studio Team Services online for source code, task and release management.

For build and deployment we typically create multiple configurations in the project for each environment (e.g. DevCMS, DevWWW, etc) and use config transforms to make changes to the config. While it is easy to set up and maintain, there are a few downsides to this:

  • Potentially sensitive configuration settings are checked into source control (passwords, connection strings)
  • A separate build is required for each environment

Azure websites do allow you to override app settings and connection strings at runtime via the portal, but this doesn’t help with other settings or files. Better would be to use environment variables within the release process. VSTS releases allow just this, in the release definition you can set up variables and set them per each environment:

The question is, how to get the variables set here to update your config files at deploy time. One way I have found to do this is as follows.

Step 1. Add Parameters.xml file to the website project.

This file defines the values that need to be updated. This can replace tokens in any text file, xml files have better support using xpath to target attributes/values to replace.

<?xml version=1.0” encoding=utf-8 ?>
<parameters>
  <parameter name=DemoEnv” description=DemoEnv” defaultValue=#{DemoEnv}#” tags=“”>
    <parameterEntry
kind=XmlFile
                    scope=obj\\Release\\Package\\PackageTmp\\Web\.config$
                    match=/configuration/appSettings/add[@key=’DemoEnv’]/@value />
</parameter>
</parameters>

For the defaultValues, set the value to a token that can be replaced during release. In this example I am using the default token format used by the Replace Tokens utility described later.

I have to credit this post from Colins ALM for explaining this, although it is about TFS the first half still applies.

http://colinsalmcorner.com/post/webdeploy-and-release-management–the-proper-way

Note that connection strings are special and the values are replaced with tokens automatically during package creation. The original value is placed into the SetParameters.xml file. So to get your own token in there, you either need to add a config transform and replace your local connection string with a token, or follow the link above and set up a publish profile that defines a connection string token that will be used in the package.

Step 2. Configure the build to generate a web deploy package

This you can achieve by using a build task with MSBuild arguments as follows:

/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation=”$(build.artifactstagingdirectory)\\”

On build it will generate a zip file and a SetParameters.xml file which has just the values from the Parameters.xml that you created earlier:

<?xml version=1.0” encoding=utf-8?>
<parameters>
<setParameter name=DemoEnv” value=#{DemoEnv}# />
</parameters>

This is ideal to replace at release time with a simple token replace task.

Step 3. Set up release definition

In the release definition, add a separate environment for each environment.

Each environment has a list of tasks. First I have added a Replace Tokens task using this add-on from the marketplace.

This is configured to set the root directory to the drop folder from the build, and target files set to the name of the SetParameters.xml file output by the build. The

This will replace any variables defined with the values set in the release variables, which you can edit via Configure variables (old editor only):

Or on the Variables tab as shown at the top of this post (old and new editor – for old editor, switch using the dropdown at the top right from Release variables to Environment variables).

Step 4. Set up deployment step

The deploy step for us is a normal Azure website deployment which uses web deploy. This defines the subscription, web app, slot, and web deploy package.

The important thing to set is the SetParameters file additional option which needs to point to the SetParameters.xml file that you did the token replacement on:

NOTE: You have to tick ‘Publish using Web Deploy’ for the SetParameters box to appear.

If you have issues with the token replacement, you can use a command line “Type” command to output the replaced version of the SetParameters.xml file to the release log that you can check has been replaced properly.

References:

http://www.colinsalmcorner.com/post/config-per-environment-vs-tokenization-in-release-management

http://colinsalmcorner.com/post/webdeploy-and-release-management–the-proper-way

One thing to note, I found that if you change the environment variables, and trigger a release, it won’t overwrite the web.config. It requires a new build. I think this is because web deploy uses the file date/size from the package to compare against the site, and this isn’t changing, so it doesn’t replace it. If anyone knows a way around this, let me know!

Download and Combine all Rackspace Cloud Files/CDN access logs

Rackspace Cloud Files / CDN can have access logging enabled on a container, which will log to another container named “.CDN_ACCESS_LOGS” in a folder structure for year/month/date/hour and multiple gzipped log files in each hour folder.

This post gives a bash script to do this but on Windows without a bash interpreter or Python etc I wanted an alternative.

To review these all in one file, use https://cyberduck.io/ to log in to browse the files, using username and API key. Search for .cdn to expose the hidden access logs folder.

081216_0929_Downloadand1.png

Double click to Open it and choose the sub folder you need to review. Right click, Download To, and choose somewhere to save it.

Then you’ll have the folder structure on your machine. Search in the folder for *.gz and select all, move to a single folder so they are all in one place.

Select all .gz files, right click, 7-Zip, Extract to here.

Now you have all the .log files in one folder. Review with your favourite log parsing tool, or to combine into one .log file, open a command prompt to the folder and enter “type *.log > c:\temp\all.log” which will crudely combine them into one file. Not in any order mind.

 

How to shrink a Sitecore database

We had some databases that were 25 gb each due to some large media items.

To shrink the database, it is not as simple as deleting the large items.

After you have identified and deleted any unnecessary items from Sitecore, do this on each database (or publish the parent section) and then Empty the Sitecore recycle bin (in each database, via the Sitecore Desktop menu).

Then you have to run the Sitecore Clean Up Database wizard. This will remove the orphaned items from the DB. Finally you can shrink the databases and the space will actually be reclaimed.

DLL assembly versioning in TeamCity builds

A nice thing to do when deploying code from a CI server is to embed the build version number into the DLLs, so if someone isn’t sure what code ended up in that environment, they can check the version rather than decompiling the DLL.

This is very easy in TeamCity, Add the “AssemblyInfo patcher” build feature, with the default configuration:

tc1

Any confusion as to what build is deployed could then check the DLL version against TeamCity, look up that build and check what changes / revision were included.

Some of our clients require us to supply packages which they deploy themselves. We supply web deploy packages. For these we provided numbered releases, and this is particularly useful in this scenario.

For these we also manually set the Build number to include the release number as well as the default build.counter e.g.

tc2

Then the DLL version will be like 5.0.42 which would mean release 5.0 built from TeamCity build# 42.

Alternatively you could embed the vcs revision number.

Capture querystring on New Relic .NET Agent

New Relic by default only captures request URL excluding querystring. This shows on transaction traces and errors etc.

Just spent ages reading the docs to figure out how to get it to include querystring, which is confusingly referred to as Request Parameters (which I thought referred to things such as server variables)

Anyway it is as simple as adding this into the newrelic.config under <configuration> :

<attributes enabled="true">
<include>request.parameters.*</include>
</attributes>

Then all querystring parameters are recorded. I think instead of * you can specify particular named parameters if you need to. Not sure if this captures POST parameters too so be careful not to record sensitive data (maybe don’t do this on an ecommerce site for example).

How to migrate users and passwords between Sitecore instances

If you ever package users from one Sitecore instance to another you will find that upon install the users are disabled and the passwords need to be reset.

What if you were moving them from one version to another and need to keep the same passwords?

Fear not, just run this SQL script. It will update the passwords of all matching usernames to the original passwords, and enable the accounts. The databases must be on the same instance.

/******

 SELECT Upgraded Core DB in SQL Studio.

 In script below, Replace Old_Core with name of old DB

 First run the SELECT only to see what it's going to do, then run the update script.

 ******/
SELECT TOP 1000
      m.[UserId],
      om.userid,
      m.[Password],
      om.password,
      m.[PasswordSalt],
      om.passwordsalt,
      u.username,
     ou.username,
      m.comment,
     m.islockedout,
     om.islockedout,
     m.isapproved,
     om.isapproved
  FROM [aspnet_Membership] m
  inner join [aspnet_Users] u ON m.UserID = u.UserID
  inner join Old_Core.dbo.aspnet_Users ou ON ou.username = u.username
  inner join Old_Core.dbo.aspnet_Membership om ON ou.userid = om.userid

   UPDATE [aspnet_Membership]
  SET
  --comment = ou.username,    -- TEST!
  Password = om.Password,
  PasswordSalt = om.PasswordSalt,
  IsLockedOut = om.IsLockedOut,
  IsApproved = om.IsApproved
  FROM [aspnet_Membership] m
  inner join [aspnet_Users] u ON m.UserID = u.UserID
  inner join Old_Core.dbo.aspnet_Users ou ON ou.username = u.username
  inner join Old_Core.dbo.aspnet_Membership om ON ou.userid = om.userid

How to get Azure web app deployment to not delete

If you’ve tried the ‘new scriptable build system’ on Visual Studio Online and you use the wizard to set up an Azure Website build/deployment you might find that by default it will delete any existing files/folder upon deployment. If you rely on any uploaded app_data files etc being kept in the wwwroot you might want to disable this.

So if you set up a build using this button:

Choose Azure Website:

Then when configuring your deploy build step make sure to add -doNotDelete into the Additional Arguments:

Otherwise it will delete existing files!

Sitecore 8 Federated Experience Manager

I recently posted a short introduction to the Sitecore FXM on the e3 blog, below is a more detailed version!

One new feature in Sitecore 8 is the Federated Experience Manager (FXM). This is a brand new tool designed to allow you to apply Sitecore’s marketing features to existing non-Sitecore sites. This would be ideal for clients who are investing in the Sitecore platform but also have legacy sites which aren’t being redeveloped into Sitecore. For example, associated WordPress blogs, Drupal communities, even flat microsites could contribute to a user’s Sitecore Experience profile and affect personalisation across all these sites. I’ve had a chance to experiment with this tool and this is what I found.

This new tool is an option in the new Sitecore Launchpad. It presents you with a list of sites that you have added already, along with number of visits that have been tracked through the FXM:

After creating a federated site they show in this list. For testing purposes I have added the e3 test website as a federated site. You need to enter the site’s domain and then paste some tracking script onto every page of the target site. The details of my federated site are as follows:

Now comes the fun bit. Clicking on “Open in Experience Editor” brings up a new version of the Experience Editor, geared towards being able to edit federated sites. In my instance, here you can see the e3 homepage showing up within the experience editor of my fresh Sitecore 8 instance:

From this visual editor you are able to do various things such as:

  • Attach Sitecore functionality to links / button clicks (such as triggering a goal)
  • Assign goals/campaigns/events to trigger when certain page(s) are viewed
  • Add Sitecore placeholders at any point within the existing site, to inject new content or replace existing content

As a basic example, we could trigger a goal when a user views a case study and then display a personalised block on the homepage to highlight a particular case study, such as this one about Unicef Launchpad.

The first thing you must do is add a Page Filter. This is how Sitecore will target your personalisation to decide which page(s) it affects. Here we create a filter to match all case study pages, and assign a custom “Viewed Case Study” goal I made earlier:


Next, for the personalisation on the Homepage, I can click “Add Placeholder” and then click on a part of the page. This selects the component I have clicked and gives a choice of adding before or after that component:

In this case I have chosen “Add before” as I want to add a new component in above “Our Latest Work”. Once the placeholder is added, I can add any existing Sitecore layout in the new placeholder (for example any design component you already have built for your Sitecore site), and display the desired content. The federated site may need updated CSS to correctly display design elements from the Sitecore site, otherwise they might appear unstyled. Once I’ve finished adding the new component it looks like this:

By default this new content will display all the time. The final step is to apply some personalisation to the new block. To do this, I can select the block and choose Edit personalisation from the floating toolbar:

The following personalisation settings will show the custom content if the user has viewed a case study, otherwise hide the component. You could add multiple conditions in here to display different content depending on the user’s activity.

Once all this is done and published, I can browse to the target site and it appears as normal. If I browse to any case study and then back to the homepage, I can see the new component has been injected into the page’s content:

There are many possibilities in terms of the personalisation logic you could apply, and this new ability to apply marketing features to external sites really adds a new dimension to Sitecore’s experience platform.