SEO Tip: Submitting All of Your Blog URLs to Search Engines
Apr
9
Written by:
4/9/2009
DotNetNuke does a pretty good job of providing an XML Site Map for your to submit to the major search engines. However, it does not contain any dynamic URLs. A good example of what I am talking about is the DNN core Blog module. This module dynamically generates a new URL that will not show up in the Site Map that is generated by DNN.
You can find what is in your site map by navigating to the SiteMap.aspx page on the root of your DNN website. For example, mine would be:
http://www.willstrohl.com/SiteMap.aspx
If you look through the included pages, bugs aside, you will notice that it only contains actual “pages” or “tabs.” If will not include any Forum or Blog URLs. The result is that while your blog URLs might eventually end up in the search engine results pages (SERP), it will take a long time, and in some cases might not happen at all.
When Tom Kraak, a DNN SEO expert, was in Orlando speaking to the ODUG, he showed us a really cool site that will generate a site map for us, by crawling the entire website. A couple caveats to this though is that it will also grab URLs that may not be important to you if you have the blog calendar on the page, resulting in the actual blog URLs being deep linked in the XML file. Also, you are limited to 500 URLs on that website, unless you pay for their service. Once it reaches 500 URLs, the process stops and your XML site map is provided to you.
Tom’s presentation inspired me to provide this solution for myself. So, not being one to keep things like this to myself, I have a free solution to provide you with another site map that you can easily submit to the major search engines, to get your blog URLs included in the search index faster and easier. I will provide the download information later. Let’s talk about it first.
This only works for the DNN Blog version 3.05.00+ and on any DNN site that can run that blog version. My solution consists of two files:
- WillStrohl.Utilities.BlogSiteMap.dll
- BlogSiteMap.ashx
Here are a few instructions on how to use the files:
- Once you have download the zip file, extract the files to a local directory, such as your Desktop.
- Next, move the DLL file to the Bin directory on the website.
- The ASHX file can be moved anywhere you want. I recommend either the root of the website, or in the ~/DesktopModules/Blog/ directory.
Now, you can navigate to the handler file, passing a PortalID to it. Here is an example of mine:
http://www.willstrohl.com/DesktopModules/Blog/BlogSiteMap.ashx?portalid=1
The final step is to submit the URL to the search engines. Each search engine has a different process for doing so.
You download the two DNN blog site map files directly from this blog.
9 comment(s) so far...
Re: SEO Tip: Submitting All of Your Blog URLs to Search Engines
Hi Will, the best way to submit multiple sitemaps is to add a line to your robots.txt file:
Sitemap: www.willstrohl.com/sitemap.xml
And make the sitemap.xml a sitemap index:
www.willstrohl.com/sitemap.aspx
www.willstrohl.com/DesktopModules/Blog/BlogSiteMap.ashx?portalid=1
Of course, if you're running more than one portal then you need to make sure your robots.txt and sitemap.xml are dynamic so they can provide the right host header etc, but that's not too hard. And once you're done, any spider that hits your robots.txt will in theory automatically pick up all your URLs without you having to submit anything to anywhere!
By Mark Allan on
4/9/2009
|
Re: SEO Tip: Submitting All of Your Blog URLs to Search Engines
Great tip, thanks!
By Will on
4/9/2009
|
Re: SEO Tip: Submitting All of Your Blog URLs to Search Engines
There should have been some sample XML for the sitemap index in there, but it got stripped out instead of HtmlEncoded. Go to en.wikipedia.org/wiki/Sitemap_index instead to see an example :)
By Mark Allan on
4/9/2009
|
Re: SEO Tip: Submitting All of Your Blog URLs to Search Engines
I like your solution. I went through a similar issue months ago and ended up installing the module from this page: bitconstruct.com/Produkte/DNNSitemap.aspx and then submitted this url to the webmaster tools on Google. But your sitemap looks better.
By .Net Development on
4/9/2009
|
Re: SEO Tip: Submitting All of Your Blog URLs to Search Engines
I see very high potential for a quality module. I am glad you liked this.
By Will on
4/9/2009
|
Re: SEO Tip: Submitting All of Your Blog URLs to Search Engines
I am about to release an update to www.inspectorit.com/products/dnn-modules/dnn-sitemap.aspx which includes SiteMap generation for Blog, News Articles and a couple other providers.
Well done on this though!
BTW, do you deal with the priority issue? Google does not like it when they are all the same and generates warnings... I fixed it with a "weighting" system for each page.
By Antonio Chagoury on
4/9/2009
|
Re: SEO Tip: Submitting All of Your Blog URLs to Search Engines
A weighting system was going to be the next step with me too. I just threw this together last night.
By Will on
4/9/2009
|
Re: SEO Tip: Submitting All of Your Blog URLs to Search Engines
It seems like we are moving in the right direction. Besides a page priority weighting system, an ideal DNN xml sitemap implementation should also obey robots.txt, which would substantially cut down on "hand-massaging" the xml file after generating it.Btw, here is one way to tackle the weighting system, but I'm not convinced that's the proper approach:
http://www.eguanasolutions.com/DNN_Blog/EntryID/8.aspx
By Tom Kraak on
4/11/2009
|
Re: SEO Tip: Submitting All of Your Blog URLs to Search Engines
Thanks for the link/mention, Tom (enjoyed your recent appearances on DNNVoice, BTW). I have not seen the ODUG presentation but from Will's description I'm sure I know which sitemap generator you used for an example -- I used it myself just last week to spin up a map for a non-DNN site.
Just to clarify: As I'd mentioned in my posting, the page-weighting idea I outlined last year was put forth only as a simple stopgap solution to help improve the (non-existant) weighting routine of the default DNN sitemap. The post also outlined a simple means for providing admins the ability to manually set the weight of individual pages without altering any core DNN code. That idea was eventually implemented in only 8 lines of code by Dylan Barber here: http://codemypantsoff.com/PantsOptional/tabid/100/EntryId/11/Improve-page-priority-feature-of-DNNs-Google-SiteMap.aspx
A follow-up post (http://www.eguanasolutions.com/DNN_Blog/EntryID/20.aspx) went into the page weighting idea in a little more detail, cited some basic examples of different site content and concluded that no one approach to assigning page weights is perfect for all. My hope is that a future version of the default DNN sitemap will allow admins to pick-and-chose from a selection of page-weighting fomulas to best match their site content as well as provide a means to manually set a page's priority value. With improvements such as Will's blog-entries-as-pages, formula options could be added to give higher weighting for posts by certain users/groups or perhaps to lower weights based on the age of posts.
After having worried a bit over this myself, I can't tell you how pleased I am to see attention given to the DNN sitemap from some of the DNN community's "big hitters" like Will and Tom. I'm looking forward to playing with Will's sitemap implementation and catching more of Tom's SEO-related posts/podcasts/presentations.
Cheers!
-mamlin
By mamlin on
5/1/2009
|