Search indexes FINALLY updated

by alex 30. April 2013 18:30

About a month ago I mentioned that I was changing the titles in the sales site. It took about a month, but it is finally updated! This means I might rank better for the keywords I'm targeting.

Search results for "site:"

Broadcasting your sitemap to search engines

by alex 30. April 2013 18:10

Day 89!

I just realized that I had the sitemap for the BuildKeeper.NET blog configured incorrectly with Google. D'oh! I was using the RSS feed instead of the dynamic sitemap! After fixing this problem, my submitted-pages count jumped up!

I don't think this mistake caused terrible problems because my blog shows up in Google results when searching for my post titles and keywords. But it means that my blog would have been a step or two behind in scanning, because new blog posts could only be detected if the blog root was reindedex first.

A few months ago I manually registered a static sitemap for the sales site and the dynamic sitemap (although incorrectly) with Google Webmaster Tools. This means Google has my sitemaps, but Bing and Yahoo don't. To broadcast your sitemaps to all search engines, add them to your robots.txt file. Here's a snippet with mine:



10-Day results on new product exploration

by alex 17. April 2013 17:14

About 10 days ago, I wrote about my plans to see if traffic to one of my old websites,, was actually interested people or confused robots. In that post, I had 2 signups and 89 unique visitors. Ten days later, I have 2 signups and 95 unique visitors.

Le sigh. I guess you can't just put stuff online and expect people to flock to it, all on their own.

Metrics indicate that my most frequent visitors are bots

The spinning coins on the website are interactive - the user can click on them. Clicking on a coin will play a coin-pickup sound, and increase the coin counter at the top of the screen. The number of coins the user has is stored in a client-side cookie, so I can only grab the value of this cookie whenever the visitor interacts with my web server - by signing up or going to another page, for example.

There were only four people who both 1) understood that the coins could be clicked AND 2) also reloaded the page, went to another page or signed up for the beta.

This implies a few things:

  • The bounce rate on this page is high
  • Bots are a major source of traffic
  • Converting visitors (from visitor to sign up) don't understand that the coins are interactive
  • Non-converting visitors click the coins but don't interact with the site anymore

Actually, all of these could be true.

My bounce rate IS high, because there aren't any other calls to action or content for the visitor to navigate to; the visitor is not pulled into the site.

Bots give me a lot of traffic - there are a large number of hits to pages that were part of the old site but the new. When a visitor tries to request a page from the old site, I show them a "hey, sorry the site's gone, but here's some information, click here for more, click here to sign up" page, and I'm not seeing any traffic from these pages. There are also a few query strings that are obviously not from either the new or old site; this indicates bots trying to sniff out my server.

No converting visitor has clicked on a coin before signing up. This implies that it might not be obvious that coins can be clicked on, so perhaps non-converting visitors are also not aware that they can be clicked on.

However, the coins mean nothing. They were an experiment to see if making the site more interactive would be beneficial to the visitor's interest in the product. Kind of like an A/B test from the start.

I'm going to keep it up an running, but not much coming from (or going into) No Dice Required, for now.

Early results on exploratory signup website No Dice Required

by alex 8. April 2013 22:31

It's day 67 of the experiment. I mentioned last week that one of my old projects, No Dice Required, recently had an unexpected surge in traffic and signups. I spent a few days last week redesigning the website in ASP.NET MVC 4 and writing new copy and user tracking code.

The new website is a coming-soon page with a beta signup. The entire site consists of maybe a dozen pages: home page (with sign up control), sign up page, sign up complete page, HTTP error pages, and a this-site-has-been-redesigned page for the pages of the old site. Additionally, there are many dynamically generated landing pages.

Home page

The old home page was poor for a few reasons. There were less than 20 words available before login and there wasn't any meta data included in the headers: together, these make it hard for search engines to know anything about the site.

The old page also didn't sell the site in any way. Why does it exist? Why should the visitor care? I'm still not particularly good at writing copy that converts, but I feel like I'm improving.

The new homepage has much more text, images, and formatting variations: these should make a visitor more interested in reading the content of the webpage.

The coins at the bottom spin, and if the visitor clicks on them then a coin-pickup sound is played and the coin counter in the top menu increases. The number of collected coins is persisted in a cookie.

I pushed this to the internet a little less than a week ago. So far there are 2 signups for the beta and it has been seen by 89 unique visitors. Most visitors haven't realized that the coins were clickable. This might be because they lack imagination and I imply that they are clickable. Implying on the internet is bad for conversion! I should have come out and said that they were clickable or collectible or something a bit less subtle.

There's quite a bit of custom user tracking going on; I will write about the details of what I'm doing. The landing pages are done, except that I still need to work out the creation of dynamic sitemaps. The trouble here is that I can't easily route a single file to a controller action without also routing all files through the ASP.NET pipeline as well; doing that would be a performance nightmare.

Creating dynamic data-driven landing pages in ASP.NET MVC

by alex 8. April 2013 19:26

Creating dynamic landing pages

I mentioned in the last post that No Dice Required was updated. The real experiment with this site is that I'm playing with scaling out SEO efforts by drastically increasing the number of landing pages.

A landing page is the first page of your website that a visitor lands on. A certain amount of visitors to will be looking for "language learning tools", but another group will be looking for "Japanese food words." Ideally, the visitor should land on the home page for the first phrase but the 2nd phrase, "Japanese food words", has no place on the homepage. That's where landing pages come in.

Landing pages are kind of like search fodder to give you more indexes in Google. There are two goals: one is to grab search phrases which are related to your problem domain but not necessarily related to your product, and the second is to then route visitors into your website from those phrases. Above all else, landing pages should resolve some need the visitor has.

It would be easy to create a dozen landing pages by hand, but this has scalability problems; hand-crafting a dozen similar pages isn't what you or I should be spending our on. As a programmer, this problem has an easy programming solution!

First off, you should remember that there isn't anything particularly unique about a landing page, other than that it generally has many crosslinks to other content on your site (and maybe to other landing pages); however the rest of your site generally won't have links to landing pages.

The rest of this post is techy and related to creating landing pages in ASP.NET MVC 4. If that isn't your cup of tea, then you should probably move on to the next article now =)

Defining your data structures

You need to consider the data that you'll need to populate a page. For No Dice, I planned to have each landing page be a list of translated vocabulary words, and it will include links to other lists, and the lists are grouped by categories.

I have POCO objects to support these goals. The POCO objects may look a little wonky, but I'm using them directly with Entity Framework to build my database via Code First, so I can deal with the weird virtual properties.

public class Word
    public int ID { get; set; }

    public DateTime DateCreated { get; set; }

    public int UserID { get; set; }

    public String Text { get; set; }

    public String Language { get; set; }

    public int? WordListID { get; set; }
    public WordList WordList { get; set; }

public class WordList
    public int ID { get; set; }

    public DateTime DateCreated { get; set; }

    public int UserID { get; set; }

    public String Name { get; set; }

    public virtual IList<WordListCategory> WordListCategory { get; set; }

    public virtual IList<Word> Words { get; set; }

public class WordListCategory
    public int ID { get; set; }

    public DateTime DateCreated { get; set; }

    public int UserID { get; set; }

    public String Name { get; set; }

    public virtual IList<WordList> WordLists { get; set; }

That's pretty simple. There's a one to many relationship between WordList and Word, and a many to many between WordList and WordListCategory. Or, in other words, every Word is in one WordList; every WordList has many Words and many WordListCategorys.

The data model I'm using isn't exactly like this, but it's similar enough for this discussion.

I don't want every WordList to be a landing page, so I also have a Page object:

public class Page
    public int ID { get; set; }
    public DateTime DateCreated { get; set; }

    public String Name { get; set; }

    public String ViewName { get; set; }
    public String ViewMasterName { get; set; }

    public virtual String Stub { get; set; }

    public int WordListID { get; set; }
    public WordList WordList { get; set; }


For every landing page that I want, I create a Page object and configure it to use a WordList, give it a stub and a name, and it's good to go.

You might also notice that I added a ViewName and ViewMasterName property. This allows me to configure which .cshtml file will be used to render the landing page, in addition to a Master Page or layout, so that every page doesn't necessarily have to have the same look and feel as the others or the rest of the site.

Persisting the objects

You'll need some place to persist all these objects, probably XML files or a database. Personally, it's quickest for me to drop them into an Entity Framework DbContext and use SQL Server.

public class DiceContext : DbContext
    public DbSet<Word> Words { get; set; }
    public DbSet<WordList> WordLists { get; set; }
    public DbSet<WordListCategory> WordListCategories { get; set; }

    public DbSet<Page> Pages { get; set; }

    protected override void OnModelCreating(DbModelBuilder modelBuilder)
            .HasOptional(p => p.WordList)
            .WithMany(d => d.Words)
            .HasForeignKey(p => p.WordListID)

            .HasRequired(p => p.WordList)
            .HasForeignKey(p => p.WordListID)

Landing page Controller

Create a new LandingPageController class, which will handle page requests.

Here's mine:

public class LandingPagesController : BaseController
    public ActionResult Index(string language1, string language2, string stub)
        Page p = null;
        // [...]
        return View(page.ViewName, page.ViewMasterName); 

I need to know the name, ID, or stub of the page to be rendered. I decided to use a stub instead of the page ID because stubs are more friendly to search engines and users. I also need to know which language the visitor knows and which language they want to study, language1 and language2. From these three data points, I can find a Page, which indicates a WordList. I can find the appropriate words using language1 and language2.

Routing URLs to your Controller

I've talked about routing in a few other posts. You want to make sure that your URLs look good. By that, I mean they should be shallow and keyword-rich. Having your keywords in the URL makes your page look more useful to search engines.

Create your new routes in the Register_Routes method (this might be in global.aspx.cs or RouteConfig.cs).

My route will interpret all 3-segmented URLs as a landing page: the first value is language1 (the visitor's native language), the 2nd is language2 (the visitor's study language), and the third value is the stub of the page to be displayed.

I will probably have to make some revisions to this routing structure if I get enough interest to make me revive this project.

public static void RegisterRoutes(RouteCollection routes)

        name: "Landing Pages",
        url: "{language1}/{language2}/{stub}",
        defaults: new { controller = "LandingPages", action = "Index" }

        name: "Default",
        url: "{controller}/{action}/{id}",
        defaults: new { controller = "Home", action = "Index", id = UrlParameter.Optional }

After this, I can load URLs like and it will be routed to the LandingPageController, with the parameters properly filled in.

Finally, the easy part: create a view and go crazy in the ways you normally would to create and populate a view. Mine looked like this in the end:

Dynamic Sitemap.xml files for bots to detect the pages

You might be asking yourself, "If my homepage doesn't link to any pages that eventually link to the landing pages, how will they ever show up in search results?" Good question.

Right now, they never will. We need to create a dynamic sitemap file and submit it to our search engines of choice so that they know about the landing pages and can index them properly.

This post has gone on long enough, so I'll cover this topic later tonight or tomorrow.

If you run into any issues or have any questions, feel free to share them in the comments below!

About the author

Something about the author

Month List