Dear Umbraco… I Can Haz Content?!

Dear Umbraco… I Can Haz Content?!

Basic mission; code not my own.

Create an advert type; consisting of URL and Image. Additional requirement created setting the type of advert (using uComponents) An AdvertContainer (Doc Type)

With two 'types' of advert (i.e. 3rdparty/mpu)

ThirdParty contains JS (no image/URL)
MPU contains Image/URL

var nodes = GetMultiNodeSelection(1101, "adverts");

Foists off into some god awful linq abuse…

private IEnumerable<IContent> GetMultiNodeSelection(int id, string property)
{
    var node = _contentService.GetById(id);
    if (node.HasProperty(property))
    {
        var nodeProperty = node.Properties[property];
        var nodeIds = nodeProperty.Value.ToString().Split(',');
        return nodeIds.Select(nodeId => _contentService.GetById(int.Parse(nodeId))).ToList();
    }
    return null;
}

Apparently the between the methods this creates 646,908 hits in 44,111 hidden methods without source?! Most of this crap appears to be between .net and IIS having a bun fight.

Tripped and fell into Umbraco

public IContent GetById(int id)
{
    using (IContentRepository repository = this._repositoryFactory.CreateContentRepository(this._uowProvider.GetUnitOfWork()))
    {
        return repository.Get(id);
    }
}

Feel like a kid on a long journey; are we there yet. I know theres a fucking database in here somewhere.

public TEntity Get(TId id)
{
    Guid guid = (id is int) ? this.ConvertIdToGuid(id) : this.ConvertStringIdToGuid(id.ToString());  -- 41 Hits!? 
    IEntity byId = this._cache.GetById(typeof(TEntity), guid);
    if (byId != null)
    {
        return (TEntity) byId;
    }
    TEntity entity = this.PerformGet(id);
    if (entity != null)
    {
        this._cache.Save(typeof(TEntity), entity);
    }
    return entity;
}

INTO

protected override IContent PerformGet(int id)
{
    Sql sql = this.GetBaseQuery(false).Where(this.GetBaseWhereClause(), new object[] { new { Id = id } })
                .Where<DocumentDto>(x => x.Newest).OrderByDescending<ContentVersionDto>(x => x.VersionDate);
    DocumentDto dto = base.Database.Fetch<DocumentDto, ContentVersionDto, ContentDto, NodeDto>(sql).FirstOrDefault<DocumentDto>();
    if (dto == null)
    {
        return null;
    }
    return this.CreateContentFromDto(dto, dto.ContentVersionDto.VersionId);
}

Getting bored now, how fucking deep is this well

Umbraco.Core.Persistence.Repositories.RepositoryBase<TId,TEntity>.Get(TId id)
private IContent CreateContentFromDto(DocumentDto dto, Guid versionId)
{
    IContentType contentType = this._contentTypeRepository.Get(dto.ContentVersionDto.ContentDto.ContentTypeId);//The slowest bit
    IContent content = new ContentFactory(contentType, this.NodeObjectTypeId, dto.NodeId).BuildEntity(dto);
    if (dto.TemplateId.get_HasValue() && (dto.TemplateId.Value > 0))
    {
        content.Template = this._templateRepository.Get(dto.TemplateId.Value);
    }
    content.Properties = this.GetPropertyCollection(dto.NodeId, versionId, contentType);
    ((ICanBeDirty) content).ResetDirtyProperties();
    return content;
}

erm am I in a fucking hole within a hole here or what

public TEntity Get(TId id) // See above

Ah fun so now were in a different PerformGet

Umbraco.Core.Persistence.Repositories.ContentTypeRepository.PerformGet(int id)
protected override IContentType PerformGet(int id)
{
    Func<DocumentTypeDto, ITemplate> selector = null;
    Sql baseQuery = this.GetBaseQuery(false);
    baseQuery.Where(this.GetBaseWhereClause(), new object[] { new { Id = id } });
    baseQuery.OrderByDescending<DocumentTypeDto>(x => x.IsDefault);
    DocumentTypeDto dto = base.Database.Fetch<DocumentTypeDto, ContentTypeDto, NodeDto>(baseQuery).FirstOrDefault<DocumentTypeDto>();
    if (dto == null)
    {
        return null;
    }
    IContentType type = new ContentTypeFactory(this.NodeObjectTypeId).BuildEntity(dto);
    type.AllowedContentTypes = base.GetAllowedContentTypeIds(id);
    type.PropertyGroups = base.GetPropertyGroupCollection(id);
    ((ContentType) type).PropertyTypes = base.GetPropertyTypeCollection(id);
    List<DocumentTypeDto> source = base.Database.Fetch<DocumentTypeDto>("WHERE contentTypeNodeId = @Id", new object[] { new { Id = id } });
    if (Enumerable.Any<DocumentTypeDto>(source))
    {
        if (selector == null)
        {
            selector = template => this._templateRepository.Get(template.TemplateNodeId);
        }
        type.AllowedTemplates = source.Select<DocumentTypeDto, ITemplate>(selector).ToList<ITemplate>();
    }
    foreach (ContentType2ContentTypeDto dto2 in base.Database.Fetch<ContentType2ContentTypeDto>("WHERE childContentTypeId = @Id", new object[] { new { Id = id } }))
    {
        type.AddContentType(base.Get(dto2.ParentId));
    }
    ((ICanBeDirty) type).ResetDirtyProperties();
    return type;
}

Now theres a name ICanBeDirty … funny, the deeper this goes the dirtier I feel.

Umbraco.Core.Persistence.Repositories.ContentTypeBaseRepository<TId,TEntity>.GetPropertyGroupCollection(int id)
protected PropertyGroupCollection GetPropertyGroupCollection(int id)
{
    Sql sql = new Sql();
    sql.Select(new object[] { "*" }).From<PropertyTypeGroupDto>().LeftJoin<PropertyTypeDto>().On<PropertyTypeGroupDto, PropertyTypeDto>(left => left.Id, right => right.PropertyTypeGroupId, new object[0]).LeftJoin<DataTypeDto>().On<PropertyTypeDto, DataTypeDto>(left => left.DataTypeId, right => right.DataTypeId, new object[0]).Where<PropertyTypeGroupDto>(x => (x.ContentTypeNodeId == id)).OrderBy<PropertyTypeGroupDto>(x => x.Id);
    List<PropertyTypeGroupDto> dto = base.Database.Fetch<PropertyTypeGroupDto, PropertyTypeDto, DataTypeDto, PropertyTypeGroupDto>(new Func<PropertyTypeGroupDto, PropertyTypeDto, DataTypeDto, PropertyTypeGroupDto>(new GroupPropertyTypeRelator().Map), sql);
    PropertyGroupFactory factory = new PropertyGroupFactory(id);
    return new PropertyGroupCollection(factory.BuildEntity(dto));
}

Now for a bit of JIT overhead… annnnnnnnnnnnnnnd

Umbraco.Core.Models.PropertyType..cctor
static PropertyType()
{
    NameSelector = ExpressionHelper.GetPropertyInfo<PropertyType, string>(x => x.Name);
    AliasSelector = ExpressionHelper.GetPropertyInfo<PropertyType, string>(x => x.Alias);
    DescriptionSelector = ExpressionHelper.GetPropertyInfo<PropertyType, string>(x => x.Description);
    DataTypeDefinitionIdSelector = ExpressionHelper.GetPropertyInfo<PropertyType, int>(x => x.DataTypeDefinitionId);
    DataTypeControlIdSelector = ExpressionHelper.GetPropertyInfo<PropertyType, Guid>(x => x.DataTypeId);
    DataTypeDatabaseTypeSelector = ExpressionHelper.GetPropertyInfo<PropertyType, DataTypeDatabaseType>(x => x.DataTypeDatabaseType);
    MandatorySelector = ExpressionHelper.GetPropertyInfo<PropertyType, bool>(x => x.Mandatory);
    HelpTextSelector = ExpressionHelper.GetPropertyInfo<PropertyType, string>(x => x.HelpText);
    SortOrderSelector = ExpressionHelper.GetPropertyInfo<PropertyType, int>(x => x.SortOrder);
    ValidationRegExpSelector = ExpressionHelper.GetPropertyInfo<PropertyType, string>(x => x.ValidationRegExp);
    PropertyGroupIdSelector = ExpressionHelper.GetPropertyInfo<PropertyType, int>(x => x.PropertyGroupId);
}

hmm bit more JIT overhead… annnnnnnnnnnnnnnd

public static PropertyInfo GetPropertyInfo<TSource, TProperty>(Expression<Func<TSource, TProperty>> propertyLambda)
{
    return PropertyInfoCache.GetOrAdd(new LambdaExpressionCacheKey(propertyLambda), delegate (LambdaExpressionCacheKey x) {
        Type type = typeof(TSource);
        MemberExpression body = propertyLambda.Body as MemberExpression;
        if (body == null)
        {
            if (propertyLambda.Body.GetType().get_Name() != "UnaryExpression")
            {
                throw new ArgumentException(string.Format("Expression '{0}' refers to a method, not a property.", propertyLambda));
            }
            UnaryExpression expression2 = propertyLambda.Body as UnaryExpression;
            if (expression2 != null)
            {
                MemberExpression operand = expression2.Operand as MemberExpression;
                if (operand == null)
                {
                    throw new ArgumentException("The type of property could not be infered, try specifying the type parameters explicitly. This can happen if you have tried to access PropertyInfo where the property's return type is a value type, but the expression is trying to convert it to an object");
                }
                body = operand;
            }
        }
        PropertyInfo member = body.Member as PropertyInfo;
        if (member == null)
        {
            throw new ArgumentException(string.Format("Expression '{0}' refers to a field, not a property.", propertyLambda));
        }
        if ((type != member.ReflectedType) && !type.IsSubclassOf(member.ReflectedType))
        {
            throw new ArgumentException(string.Format("Expresion '{0}' refers to a property that is not from type {1}.", propertyLambda, type));
        }
        return member;
    });
}

Hey were now using something in the system. Well kinda, mixed in with more usage of the above method.

In a tiny portion of application warm up 81.7% of time is spent in JIT overhead; now we can pretty much ignore that (kinda) as we know profilers disable native imaging which means the system will have to jump to JIT. Can't quite ignore the fucking horrendous pile of code it takes JUST to get a tiny bit of data.

---|| SQL SQL SQL

AND WHERE OFF >

SELECT *
FROM [cmsDocument]
INNER JOIN [cmsContentVersion]
ON [cmsDocument].[versionId] = [cmsContentVersion].[VersionId]
INNER JOIN [cmsContent]
ON [cmsContentVersion].[ContentId] = [cmsContent].[nodeId]
INNER JOIN [umbracoNode]
ON [cmsContent].[nodeId] = [umbracoNode].[id]
WHERE ([umbracoNode].[nodeObjectType] = 'c66ba18e-eaf3-4cff-8a22-41b16d66a972')
AND (umbracoNode.id = @0)
AND ([cmsDocument].[newest]='True')
ORDER BY [cmsContentVersion].[VersionDate] DESC

--

SELECT *
FROM [cmsContentTypeAllowedContentType]
WHERE ([cmsContentTypeAllowedContentType].[Id] = 1167)

--

SELECT *
FROM [cmsContentTypeAllowedContentType]
WHERE ([cmsContentTypeAllowedContentType].[Id] = 1166)

--

SELECT *
FROM [cmsPropertyTypeGroup]
LEFT JOIN [cmsPropertyType]
ON [cmsPropertyTypeGroup].[id] = [cmsPropertyType].[propertyTypeGroupId]
LEFT JOIN [cmsDataType]
ON [cmsPropertyType].[dataTypeId] = [cmsDataType].[nodeId]
WHERE ([cmsPropertyTypeGroup].[contenttypeNodeId] = 1167)
ORDER BY [cmsPropertyTypeGroup].[id]

--

SELECT *
FROM [cmsPropertyTypeGroup]
LEFT JOIN [cmsPropertyType]
ON [cmsPropertyTypeGroup].[id] = [cmsPropertyType].[propertyTypeGroupId]
LEFT JOIN [cmsDataType]
ON [cmsPropertyType].[dataTypeId] = [cmsDataType].[nodeId]
WHERE ([cmsPropertyTypeGroup].[contenttypeNodeId] = 1166)
ORDER BY [cmsPropertyTypeGroup].[id]

--

SELECT *
FROM [cmsPropertyType]
INNER JOIN [cmsDataType]
ON [cmsPropertyType].[dataTypeId] = [cmsDataType].[nodeId]
WHERE ([cmsPropertyType].[contentTypeId] = 1167)

--

SELECT *
FROM [cmsPropertyType]
INNER JOIN [cmsDataType]
ON [cmsPropertyType].[dataTypeId] = [cmsDataType].[nodeId]
WHERE ([cmsPropertyType].[contentTypeId] = 1166)

--

SELECT *
FROM [cmsDocumentType]
RIGHT JOIN [cmsContentType]
ON [cmsContentType].[nodeId] = [cmsDocumentType].[contentTypeNodeId]
INNER JOIN [umbracoNode]
ON [cmsContentType].[nodeId] = [umbracoNode].[id]
WHERE ([umbracoNode].[nodeObjectType] = 'a2cb7800-f571-4787-9638-bc48539a0efb')
AND (umbracoNode.id = @0)
ORDER BY [cmsDocumentType].[IsDefault] DESC

--

SELECT [cmsContentType2ContentType].[parentContentTypeId], [cmsContentType2ContentType].[childContentTypeId] FROM [cmsContentType2ContentType] WHERE childContentTypeId = @0

--


SELECT [cmsDocumentType].[contentTypeNodeId], [cmsDocumentType].[templateNodeId], [cmsDocumentType].[IsDefault] FROM [cmsDocumentType] WHERE contentTypeNodeId = @0

--

SELECT *
FROM [cmsContentTypeAllowedContentType]
WHERE ([cmsContentTypeAllowedContentType].[Id] = 1165)

--

SELECT *
FROM [cmsPropertyTypeGroup]
LEFT JOIN [cmsPropertyType]
ON [cmsPropertyTypeGroup].[id] = [cmsPropertyType].[propertyTypeGroupId]
LEFT JOIN [cmsDataType]
ON [cmsPropertyType].[dataTypeId] = [cmsDataType].[nodeId]
WHERE ([cmsPropertyTypeGroup].[contenttypeNodeId] = 1165)
ORDER BY [cmsPropertyTypeGroup].[id]

--

SELECT *
FROM [cmsPropertyType]
INNER JOIN [cmsDataType]
ON [cmsPropertyType].[dataTypeId] = [cmsDataType].[nodeId]
WHERE ([cmsPropertyType].[contentTypeId] = 1165)

--

SELECT *
FROM [cmsPropertyData]
INNER JOIN [cmsPropertyType]
ON [cmsPropertyData].[propertytypeid] = [cmsPropertyType].[id]
WHERE ([cmsPropertyData].[contentNodeId] = 1172)
AND ([cmsPropertyData].[versionId] = '7dbfb821-05b4-443e-a616-0cb5d126acad')


--

SELECT *
FROM [cmsPropertyData]
INNER JOIN [cmsPropertyType]
ON [cmsPropertyData].[propertytypeid] = [cmsPropertyType].[id]
WHERE ([cmsPropertyData].[contentNodeId] = 1171)
AND ([cmsPropertyData].[versionId] = '6f80d6cd-d1ac-4793-bca3-4343f34f9515')

--

SELECT *
FROM [cmsPropertyData]
INNER JOIN [cmsPropertyType]
ON [cmsPropertyData].[propertytypeid] = [cmsPropertyType].[id]
WHERE ([cmsPropertyData].[contentNodeId] = 1170)
AND ([cmsPropertyData].[versionId] = '510748fa-e6f5-4164-b237-8cea211bff31')

Ooook so less of a quick dive; more like being drowned. To add some context; Node: 1173 consists of 5 records which build up a media element in umbraco (file/width/height/etc); the query was to

To query this one bit of the required data from the db Umbraco creates this query...

SELECT  *
FROM    [cmsPropertyData]
        INNER JOIN [cmsPropertyType] ON [cmsPropertyData].[propertytypeid] = [cmsPropertyType].[id]
WHERE   ( [cmsPropertyData].[contentNodeId] = 1173 )
        AND ( [cmsPropertyData].[versionId] = '87089cf2-f3fb-48c0-904a-346c56d14f2a' )

In fairness, once started, with A LOT of the site not actually making use of umbraco it performs ok. The nodes being loaded in this instance was simply pulling the content of an advert container. Which consists of 3 adverts; an advert consisting of a dropdown (3 items in it to set the type), a media picker (1 image) and a url.

Bit of a double edged blade; if you use umbraco doctypes for everything (think blog style/comments/date filters/long trees lots of properties) it blows and flamethrows memory till there's nothing left; if you treat umbraco like a repository your reliant on the API being stable, set in stone, and eventually, and probably not as far away as you'd like it, to be able to debug it without being dropped into a mass of generic/anonymous methods/linq hell.

Bleh

Chris McKee

Chris McKee

https://chrismckee.co.uk

Software Engineer, Web Front/Backend/Architecture; all-round tech obsessed geek. I hate unnecessary optimism