Sitemaps
Have you ever felt lost in a crowded theme park or museum and thought it would have been better if there had been a map with all the locations properly listed? A sitemap is the same map that lists all the website URLs in one place to make it easier for users and, most importantly, crawlers to collect information.
A sitemap is a file where the website owner provides all the information about their pages, images, videos, and other files for easy crawling.
A website can have multiple pages, some of them interlinked, while others with no links. In such a case, users can feel lost as the layout of those pages is improper or not clear. Poor navigation can be another reason for visitors to feel lost. In fact, it is not all about users, Google’s crawlers also use the information available on the sitemap for easy updates.
Let’s get deeper into understanding the importance of Sitemaps and how many types of sitemaps exist. Further, how to find, create, and submit a sitemap to Google.
What is a Sitemap and Why is it Important?
A sitemap is a file that enables crawlers and users to easily find information about all the pages and files available on the website. The big question is, “Why having a sitemap on your website is important”? There are various reasons behind creating a sitemap for your website – better discoverability, faster indexation, and improved user experience. But is it mandatory?
NO.
If you have a small website with proper internal linking, it doesn’t matter if you have a sitemap or not. But, if you have a large website or pages that are isolated or unlinked, it is recommended to create a sitemap for your site. Another case in which you must create a sitemap is if your site is new and frequently adds or updates data (for example, a blog site or news site).
Sitemaps make it easier for crawlers to discover new or important pages on your website. They are highly beneficial for large websites that have multiple orphaned pages. Secondly, if you add a new service page or blog to the website, the moment you add the sitemap to the search console, crawlers receive a request to visit the site and discover the new changes. Additionally, users can easily find the content they want to view with the HTML sitemaps.
HTML sitemap? What’s that? Are there different types of sitemap?
Yes, the next section is all about the different types of sitemap formats available.
Types of Sitemap Formats and Their Role
In most SEO interviews, you would have been asked a simple question “Name the type of sitemaps”. Generally, candidates answer “XML and HTML sitemaps”.
But, the answer is wrong.
There are three types of Sitemap Formats:
- XML Sitemap (Most Versatile)
- RSS, mRSS, and Atom 1.0 Sitemaps
- Text/HTML Sitemap (Simplest)
XML Sitemaps
XML Sitemaps are written in Extensible Markup Language (XML) for search engines like Google to easily understand which URLs the admin wants to get indexed. Website owners or admins can further generate this type of sitemap to provide additional information about your website, i.e., images, videos, news content, or localized versions of your page.
A manual XML sitemap (or sitemap.xml file) looks like:
While an auto-generated sitemap via plugin looks like:
Before we proceed with the main tags used in these sitemaps, let me clear the manual vs auto-generated sitemap use and benefits.
Manual vs auto-generated sitemaps (Static vs Dynamic Sitemaps) is used by both small and large organizations. But generally, organizations prefer auto-generated sitemaps over manual ones because they are easier to maintain and scalable. Manual (Static) sitemaps demand regular manual updates, especially if you add services or other content to your site more frequently. This can be difficult to maintain if you have a large or dynamic website.
Auto-generated (Dynamic) Sitemaps, on the other hand, automatically update with every site change. Thus, helping search engines to easily track new content URL and index faster. With these sitemaps, you do not need to worry about maintenance and regular updating. They handle your website’s structure and content changes more easily in comparison to static sitemaps.
Now, let’s discuss the main tags you can find in an XML Sitemap file.
If you are manually creating a sitemap file, it is important to ensure that each tag value is entity escaped <>. <urlset> encloses all tags for each sitemap, while <url> tag value encloses all the tags for each URL.
In <loc> tags, you must include your URLs.
While the <lastmod> tag indicates when the page was last modified represented by date and time.
<changefreq> and <priority> tag values are now ignored by Google. <changefreq> indicated how frequently the page was modified and <priority>, on the other hand, represented how important the page is in comparison to other webpages.
RSS, mRSS and Atom 1.0 Sitemaps
Another sitemap format that looks quite similar to XML sitemap and is quite easy to generate. CMS auto-generates RSS and Atom feeds disclosing information about text on the website. You can submit these recent feed URLs in your sitemap for indexing. For video content on your website, it is recommended to use mRSS (media RSS) feed.
XML sitemap can include sub-sections like Images, Videos, News, and alternate language information. However, RSS, mRSS, and Atom 1.0 Sitemaps only provide information about text and videos.
Text Sitemaps
This type of sitemap format only lists page URLs. You can simply create a text file and list all page URLs per line and submit it to Google.
In these file formats, you do not include any tag value or additional information other than URLs. Also, it is important to have a text extension – sitemap.txt
HTML sitemaps are also like text sitemaps, it’s just that they are written in HTML files and clickable links are grouped in categories. Also, their extension is sitemap.html
But why Google’s official documentation doesn’t count HTML sitemaps in the list?
Now, that you know how many sitemap formats exist, it is time to understand how to create one and make it available to Google.
How to Generate an XML Sitemap?
The simple way to check your sitemap after creation is to open your browser and type yourdomain.com/sitemap.xml
If you find your file, the sitemap already exists. But, if not, then you can either add it manually or auto-generate via plugins.
For manual XML sitemap creation, use this sitemap generator tool. They will provide you with a file and all you need to upload in your site’s root folder (ask your developer for guidance and help).
For dynamic XML sitemap, if you are using the YOAST SEO plugin, go to the Yoast SEO tool available in the dashboard after activation, under General settings, turn the XML sitemap feature ON. Similarly, if you are using other SEO plugins like RankMath SEO or All-in-One SEO, you will find the sitemap activation within your plugin settings.
How to Make Your Sitemap Available to Google?
Open your sitemap link in the browser by typing yourdomain.com/sitemap.xml.
Also, sign in to your Google Search Console Account in another tab on your browser.
Select your property and go to sitemaps under the Indexing section from the left panel.
Here, you will find the option to submit your sitemap link.
Other ways to submit a sitemap, include:
- Using Search Console API to programmatically submit a sitemap.
- Enter your sitemap link to the robots.txt file
- Use WebSub in the case of Atom 1.0 or RSS sitemap.
In case, you were using a Google Search Console account, after submitting your sitemap below you will find your search console sitemap report that provides the following information:
- Your Sitemap URL
- Type/Format of Sitemap: XML, TXT, RSS or ATOM
- Last Submission Date
- Crawl status (If there is any error, success, or couldn’t fetch).
- Number of URLs discovered in the Sitemap
One quick tip here is having a sitemap doesn’t guarantee that all your pages will be crawled and indexed by Google. It is a roadmap for crawlers to easily find new or updated data from a single source. In most cases, having a sitemap is helpful. In case your specific page is not getting indexed, you need to check for other reasons.
