Relationships between in-house marketers and developers can often be interesting to say the least, so why not sit yourself on their swivel chair for a while and try your hand at a couple of technical SEO tweaks.

Below are 2 Techie Tweaks for SEO that you can implement without bringing in the developers, so grab your insect repellent and bug box because you’re off to hunt for spiders!

Tech Tweak 1:


As with all web crawlers, spider bots have an insatiable appetite for going just about anywhere that they please and if your website has a few corners that you’d prefer were crawl-free, then knowing how to stop the bots is a useful tweak that needn’t involve dev time.

The ability to use robots.txt is a great piece of kit to have in your locker as this is the first thing that a spider will look for when visiting a web page. It gives instructions as to what it should be looking for and where it should be going.

If you’re using robots.txt for the first time then you need to know which instructions to give to the spider bot so that it understands what to do. It is a text file and not HTML based and can only be accessed from a server’s root location i.e.

Key features of robots.txt files:

  • User-agent – this is the name of the robot given by directories and search engines, for example Google’s robot is called Googlebot – sweet, no? This goes at the top of the robots.txt file and will be read first from left to right by the visiting bot.
  • Asterisk (*) – using an asterisk (*) indicates that the instruction is to be read by all.

So the wording: user-agent: * indicates that the instructions in the txt.file are for all robots whereas the wording: user-agent: googlebot explains that the instructions are only for Google’s benefit.

  • Disallow – writing disallow under the user-agent line describes where you don’t want spiders to crawl. For example if you have a page(s) or a folder(s) that you’d prefer wasn’t indexed by search engines, then write as follows:

User-agent: *
Disallow: /About us page/
Disallow: /animals pics/
And repeat…

Even if you don’t have anywhere that you don’t want bots to visit, it’s always best practice to include an empty/default robots.txt file as this will help cut down on 404 page not found errors which can occur when bots are searching for txt.file instructions.

Write empty/default as follows:

User-agent: *

Tech Tweak 2:

URL parameter handling in Webmaster tools

First thing’s first – check in Webmaster tools and find out whether you actually need to tweak your parameters. If Google says everything is fine, then don’t start tinkering just for the sake of it as this may result in pages being dropped from search results.

If you don’t find a message from Google saying that everything is fine, then it’s time to get involved.

Basically, the URL parameter handling in Webmaster tools lets you instruct Googlebot as to which pages to crawl and which should be left alone.

This cuts down on the amount of crawl time that will be spent checking over duplicate URLs, duplicate content and smaller sub-sections of your site and will result in more time being spent indexing your site for pages that matter.

In the main, there are 2 categories of URL parameters:

  • Ones which don’t change the content – These are commonly used for tracking and referring purposes
  • Ones which do change the content – These are used to separate clusters and help narrow down searches for specific items/colours/sizes etc.

Parameter handling in Webmaster tools allows you to tell Google exactly what different parameters on your site do, so they know whether to ignore them or not, and it can help reduce the risk of duplicate content issues.

For example, the URLs below all point to the same thing, but one has come from an email campaign and includes a tracking code for Google Analytics:

We’d want Google to ignore both the ?utm_medium and ?utm_source parameters, as they don’t change the page’s content at all – they just add extra information for Google Analytics.

In the case below you’ll see that all 3 URLs point to exactly the same thing – plaid trousers for men – just sorted and filtered differently.

These are just some examples of how you can improve in-house marketing without needing to utilise precious dev time, but there are many more out there and it’s never a bad idea to expand your repertoire. By developing digital skills as a marketer, you’ll have more control over your site. That can only be a positive thing in our eyes.