Umbraco setup with media folder on an Azure File Share

We had a client who is moving their legacy Umbraco 4 and 6 sites to Azure VMs, and wanted to use Azure File Share for the shared media folder, rather than the recommended DFS setup.

After some trial and error we got this working (starting from a vanilla Umbraco site), by making sure the following:

  • Create a local Windows user account with the same credentials as the Azure File Share user and password
  • Set the IIS app pool user to the Azure File Store local user  – So Umbraco could access/write to the media folder
  • Set website > authentication > anonymous auth – to use App pool user (not IUSR) – So I could browse to files stored in the media folder
  • Create an IIS ‘Application’ for “media”  (not a virtual directory) – so I can stop the Umbraco web.config from applying to the media folder (see error below)
    • The path should be set to the Azure File Share folder e.g. \\myfilestorage.file.core.windows.net\myfileshare\media (you should map a network drive and create a ‘media’ folder first)
  • Update the web.config and wrap all the Umbraco stuff with <location path=”.” inheritInChildApplications=”false”></location> – to stop the Umbraco web.config from applying to the media folder (see error below)
    • The <configSections> and <runtime> sections must be kept outside of the <location> section.

We have uploaded a bunch of media without error and performed a stress test on the loading of media item via the load balanced front end, without any issues.

We have also checked this works with Umbraco 7 although when we upgrade to this we intend to use Blob storage instead. The Umbraco 7 default web.config includes some <location> sections which must also be moved outside the new <location> tag, and I had to also remove the legacy <httpModules> section to stop some wierd errors…

If you don’t do the last two steps, then it will work in Umbraco but when browsing to a media item you will see this error:

If you forget to set the site authentication to use App Pool Identity, then you will instead get a wierd error code 414 to do with the request URL being too long!

 

Using environment variables for configuration with VSTS build and release

29/09/17 – Updated to cover new build/release editor. Screenshots still show the old editor. Also sorry about the double quotes in the code snippets, they are being replaced with pretty ones and you have to use normal ones, I’ll sort it out sometime.

We are using Visual Studio Team Services online for source code, task and release management.

For build and deployment we typically create multiple configurations in the project for each environment (e.g. DevCMS, DevWWW, etc) and use config transforms to make changes to the config. While it is easy to set up and maintain, there are a few downsides to this:

  • Potentially sensitive configuration settings are checked into source control (passwords, connection strings)
  • A separate build is required for each environment

Azure websites do allow you to override app settings and connection strings at runtime via the portal, but this doesn’t help with other settings or files. Better would be to use environment variables within the release process. VSTS releases allow just this, in the release definition you can set up variables and set them per each environment:

The question is, how to get the variables set here to update your config files at deploy time. One way I have found to do this is as follows.

Step 1. Add Parameters.xml file to the website project.

This file defines the values that need to be updated. This can replace tokens in any text file, xml files have better support using xpath to target attributes/values to replace.

<?xml version=1.0” encoding=utf-8 ?>
<parameters>
  <parameter name=DemoEnv” description=DemoEnv” defaultValue=#{DemoEnv}#” tags=“”>
    <parameterEntry
kind=XmlFile
                    scope=obj\\Release\\Package\\PackageTmp\\Web\.config$
                    match=/configuration/appSettings/add[@key=’DemoEnv’]/@value />
</parameter>
</parameters>

For the defaultValues, set the value to a token that can be replaced during release. In this example I am using the default token format used by the Replace Tokens utility described later.

I have to credit this post from Colins ALM for explaining this, although it is about TFS the first half still applies.

http://colinsalmcorner.com/post/webdeploy-and-release-management–the-proper-way

Note that connection strings are special and the values are replaced with tokens automatically during package creation. The original value is placed into the SetParameters.xml file. So to get your own token in there, you either need to add a config transform and replace your local connection string with a token, or follow the link above and set up a publish profile that defines a connection string token that will be used in the package.

Step 2. Configure the build to generate a web deploy package

This you can achieve by using a build task with MSBuild arguments as follows:

/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation=”$(build.artifactstagingdirectory)\\”

On build it will generate a zip file and a SetParameters.xml file which has just the values from the Parameters.xml that you created earlier:

<?xml version=1.0” encoding=utf-8?>
<parameters>
<setParameter name=DemoEnv” value=#{DemoEnv}# />
</parameters>

This is ideal to replace at release time with a simple token replace task.

Step 3. Set up release definition

In the release definition, add a separate environment for each environment.

Each environment has a list of tasks. First I have added a Replace Tokens task using this add-on from the marketplace.

This is configured to set the root directory to the drop folder from the build, and target files set to the name of the SetParameters.xml file output by the build. The

This will replace any variables defined with the values set in the release variables, which you can edit via Configure variables (old editor only):

Or on the Variables tab as shown at the top of this post (old and new editor – for old editor, switch using the dropdown at the top right from Release variables to Environment variables).

Step 4. Set up deployment step

The deploy step for us is a normal Azure website deployment which uses web deploy. This defines the subscription, web app, slot, and web deploy package.

The important thing to set is the SetParameters file additional option which needs to point to the SetParameters.xml file that you did the token replacement on:

NOTE: You have to tick ‘Publish using Web Deploy’ for the SetParameters box to appear.

If you have issues with the token replacement, you can use a command line “Type” command to output the replaced version of the SetParameters.xml file to the release log that you can check has been replaced properly.

References:

http://www.colinsalmcorner.com/post/config-per-environment-vs-tokenization-in-release-management

http://colinsalmcorner.com/post/webdeploy-and-release-management–the-proper-way

One thing to note, I found that if you change the environment variables, and trigger a release, it won’t overwrite the web.config. It requires a new build. I think this is because web deploy uses the file date/size from the package to compare against the site, and this isn’t changing, so it doesn’t replace it. If anyone knows a way around this, let me know!

Download and Combine all Rackspace Cloud Files/CDN access logs

Rackspace Cloud Files / CDN can have access logging enabled on a container, which will log to another container named “.CDN_ACCESS_LOGS” in a folder structure for year/month/date/hour and multiple gzipped log files in each hour folder.

This post gives a bash script to do this but on Windows without a bash interpreter or Python etc I wanted an alternative.

To review these all in one file, use https://cyberduck.io/ to log in to browse the files, using username and API key. Search for .cdn to expose the hidden access logs folder.

081216_0929_Downloadand1.png

Double click to Open it and choose the sub folder you need to review. Right click, Download To, and choose somewhere to save it.

Then you’ll have the folder structure on your machine. Search in the folder for *.gz and select all, move to a single folder so they are all in one place.

Select all .gz files, right click, 7-Zip, Extract to here.

Now you have all the .log files in one folder. Review with your favourite log parsing tool, or to combine into one .log file, open a command prompt to the folder and enter “type *.log > c:\temp\all.log” which will crudely combine them into one file. Not in any order mind.

 

How to shrink a Sitecore database

We had some databases that were 25 gb each due to some large media items.

To shrink the database, it is not as simple as deleting the large items.

After you have identified and deleted any unnecessary items from Sitecore, do this on each database (or publish the parent section) and then Empty the Sitecore recycle bin (in each database, via the Sitecore Desktop menu).

Then you have to run the Sitecore Clean Up Database wizard. This will remove the orphaned items from the DB. Finally you can shrink the databases and the space will actually be reclaimed.

DLL assembly versioning in TeamCity builds

A nice thing to do when deploying code from a CI server is to embed the build version number into the DLLs, so if someone isn’t sure what code ended up in that environment, they can check the version rather than decompiling the DLL.

This is very easy in TeamCity, Add the “AssemblyInfo patcher” build feature, with the default configuration:

tc1

Any confusion as to what build is deployed could then check the DLL version against TeamCity, look up that build and check what changes / revision were included.

Some of our clients require us to supply packages which they deploy themselves. We supply web deploy packages. For these we provided numbered releases, and this is particularly useful in this scenario.

For these we also manually set the Build number to include the release number as well as the default build.counter e.g.

tc2

Then the DLL version will be like 5.0.42 which would mean release 5.0 built from TeamCity build# 42.

Alternatively you could embed the vcs revision number.

Capture querystring on New Relic .NET Agent

New Relic by default only captures request URL excluding querystring. This shows on transaction traces and errors etc.

Just spent ages reading the docs to figure out how to get it to include querystring, which is confusingly referred to as Request Parameters (which I thought referred to things such as server variables)

Anyway it is as simple as adding this into the newrelic.config under <configuration> :

<attributes enabled="true">
<include>request.parameters.*</include>
</attributes>

Then all querystring parameters are recorded. I think instead of * you can specify particular named parameters if you need to. Not sure if this captures POST parameters too so be careful not to record sensitive data (maybe don’t do this on an ecommerce site for example).

How to migrate users and passwords between Sitecore instances

If you ever package users from one Sitecore instance to another you will find that upon install the users are disabled and the passwords need to be reset.

What if you were moving them from one version to another and need to keep the same passwords?

Fear not, just run this SQL script. It will update the passwords of all matching usernames to the original passwords, and enable the accounts. The databases must be on the same instance.

/******

 SELECT Upgraded Core DB in SQL Studio.

 In script below, Replace Old_Core with name of old DB

 First run the SELECT only to see what it's going to do, then run the update script.

 ******/
SELECT TOP 1000
      m.[UserId],
      om.userid,
      m.[Password],
      om.password,
      m.[PasswordSalt],
      om.passwordsalt,
      u.username,
     ou.username,
      m.comment,
     m.islockedout,
     om.islockedout,
     m.isapproved,
     om.isapproved
  FROM [aspnet_Membership] m
  inner join [aspnet_Users] u ON m.UserID = u.UserID
  inner join Old_Core.dbo.aspnet_Users ou ON ou.username = u.username
  inner join Old_Core.dbo.aspnet_Membership om ON ou.userid = om.userid

   UPDATE [aspnet_Membership]
  SET
  --comment = ou.username,    -- TEST!
  Password = om.Password,
  PasswordSalt = om.PasswordSalt,
  IsLockedOut = om.IsLockedOut,
  IsApproved = om.IsApproved
  FROM [aspnet_Membership] m
  inner join [aspnet_Users] u ON m.UserID = u.UserID
  inner join Old_Core.dbo.aspnet_Users ou ON ou.username = u.username
  inner join Old_Core.dbo.aspnet_Membership om ON ou.userid = om.userid

How to get Azure web app deployment to not delete

If you’ve tried the ‘new scriptable build system’ on Visual Studio Online and you use the wizard to set up an Azure Website build/deployment you might find that by default it will delete any existing files/folder upon deployment. If you rely on any uploaded app_data files etc being kept in the wwwroot you might want to disable this.

So if you set up a build using this button:

Choose Azure Website:

Then when configuring your deploy build step make sure to add -doNotDelete into the Additional Arguments:

Otherwise it will delete existing files!

Sitecore 8 Federated Experience Manager

I recently posted a short introduction to the Sitecore FXM on the e3 blog, below is a more detailed version!

One new feature in Sitecore 8 is the Federated Experience Manager (FXM). This is a brand new tool designed to allow you to apply Sitecore’s marketing features to existing non-Sitecore sites. This would be ideal for clients who are investing in the Sitecore platform but also have legacy sites which aren’t being redeveloped into Sitecore. For example, associated WordPress blogs, Drupal communities, even flat microsites could contribute to a user’s Sitecore Experience profile and affect personalisation across all these sites. I’ve had a chance to experiment with this tool and this is what I found.

This new tool is an option in the new Sitecore Launchpad. It presents you with a list of sites that you have added already, along with number of visits that have been tracked through the FXM:

After creating a federated site they show in this list. For testing purposes I have added the e3 test website as a federated site. You need to enter the site’s domain and then paste some tracking script onto every page of the target site. The details of my federated site are as follows:

Now comes the fun bit. Clicking on “Open in Experience Editor” brings up a new version of the Experience Editor, geared towards being able to edit federated sites. In my instance, here you can see the e3 homepage showing up within the experience editor of my fresh Sitecore 8 instance:

From this visual editor you are able to do various things such as:

  • Attach Sitecore functionality to links / button clicks (such as triggering a goal)
  • Assign goals/campaigns/events to trigger when certain page(s) are viewed
  • Add Sitecore placeholders at any point within the existing site, to inject new content or replace existing content

As a basic example, we could trigger a goal when a user views a case study and then display a personalised block on the homepage to highlight a particular case study, such as this one about Unicef Launchpad.

The first thing you must do is add a Page Filter. This is how Sitecore will target your personalisation to decide which page(s) it affects. Here we create a filter to match all case study pages, and assign a custom “Viewed Case Study” goal I made earlier:


Next, for the personalisation on the Homepage, I can click “Add Placeholder” and then click on a part of the page. This selects the component I have clicked and gives a choice of adding before or after that component:

In this case I have chosen “Add before” as I want to add a new component in above “Our Latest Work”. Once the placeholder is added, I can add any existing Sitecore layout in the new placeholder (for example any design component you already have built for your Sitecore site), and display the desired content. The federated site may need updated CSS to correctly display design elements from the Sitecore site, otherwise they might appear unstyled. Once I’ve finished adding the new component it looks like this:

By default this new content will display all the time. The final step is to apply some personalisation to the new block. To do this, I can select the block and choose Edit personalisation from the floating toolbar:

The following personalisation settings will show the custom content if the user has viewed a case study, otherwise hide the component. You could add multiple conditions in here to display different content depending on the user’s activity.

Once all this is done and published, I can browse to the target site and it appears as normal. If I browse to any case study and then back to the homepage, I can see the new component has been injected into the page’s content:

There are many possibilities in terms of the personalisation logic you could apply, and this new ability to apply marketing features to external sites really adds a new dimension to Sitecore’s experience platform.

Automated Site Setup Scripts (Sitecore / Umbraco)

We wanted a simple way to get a new environment set up on a developer machine, but without the hassle of Virtual Machines.

Enter, Microsoft’s appcmd.

This lets you manipulate IIS7 via command line calls. There is lots of documentation on the IIS site but it takes a while to figure out which commands you need to do everything to set up a site. So I’ve put together a selection of the useful commands below in a sample site setup script.

We save these as batch files (.bat) and put them in a shared folder.

Prerequisites:

  • You work on Windows using IIS7
  • (if Sitecore) You have all relevant Sitecore installation zips in a shared network folder
  • (if Umbraco) You use Nuget package to distribute Umbraco CMS (or you could do the zip thing)
  • You run sites locally with custom host name, using a shared DB
  • All team members setup sites in the same folder path e.g. c:\work
  • You have admin rights on the local machine

Sample setup script:

Put this in a .bat file and save in a shared folder. Keep as many steps as required, and replace the relevant placeholders. To run, right click Run as Administrator.

@ECHO OFF

ECHO *** MUST BE RUN AS ADMIN - IF NOT PLEASE CLOSE AND RUN AGAIN, OTHERWISE: ***
PAUSE

ECHO *** CHECKING OUT FROM SVN ***

IF not exist c:\PATH_TO_TRUNK svn checkout -q https://SVN_PATH_TO_TRUNK c:\PATH_TO_TRUNK

ECHO *** EXTRACTING SITECORE ***

"C:\Program Files\7-Zip\7z" x -y "PATH_TO_SITECORE_ZIPS\Sitecore 7.0 rev. 140120.zip" -oc:\PATH_TO_TRUNK "Sitecore 7.0 rev. 140120\Data" "Sitecore 7.0 rev. 140120\Website"

ECHO *** RENAMING SITECORE ***

rename "c:\PATH_TO_TRUNK\Sitecore 7.0 rev. 140120" www

ECHO *** COPY LICENSE ***

copy PATH_TO_LICENSE\license.xml c:\PATH_TO_TRUNK\www\Data

ECHO *** CREATING APP POOL ***

C:\Windows\System32\inetsrv\appcmd add apppool /name:"APP_POOL_NAME" -managedRuntimeVersion:v4.0

ECHO *** CREATING SITE ***

C:\Windows\System32\inetsrv\appcmd add site /name:"APP_POOL_NAME" /bindings:http://site.localhost:80 /physicalPath:"c:\PATH_TO_TRUNK\www\Website" -applicationDefaults.applicationPool:APP_POOL_NAME

ECHO *** CREATING VIRTUAL DIRECTORIES ***

C:\Windows\System32\inetsrv\appcmd add vdir /app.name:"APP_POOL_NAME/" /path:/siteAssets /physicalPath:c:\PATH_TO_TRUNK\siteAssets

ECHO *** ADDING TO HOSTS FILE ***

find /c "	site.localhost" c:\windows\system32\drivers\etc\hosts
if %errorlevel% equ 1 goto notfound
echo Already there
goto done
:notfound
ECHO 127.0.0.1	site.localhost >> c:\windows\system32\drivers\etc\hosts
goto done
:done
  
ECHO *** BUILDING SOLUTION ***

C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe c:\PATH_TO_TRUNK\src\Site.sln /verbosity:quiet

ECHO *** OPENING SITE ***

start http://site.localhost/

PAUSE

Hope that’s helpful to someone!