Sitemap is a html page of a site or a special xml file, which reflects links to all important pages of the site. To understand what a site map is, just submit a book table of contents and immediately it becomes clear Why do I need a site map. The site map helps the visitor, whether it be a person or a search robot, quickly find any page on the site, making the minimum number of transitions. On simple, small sites, all pages can be accessed in 1-2 clicks from the main page, but what if the site is large and its structure is not simple? Such a site will be inconvenient not only to visitors, but also not obvious to search engines.
What are site maps
Search engines index the site gradually, level by level, starting from the main page. If the site has many levels of nesting of pages, then until all pages of the site are indexed, it will take many months. Means necessary create site map, on which there will be all links to all pages of the site at once, so that the robot does not waste time navigating through the levels of nesting, right? Yes and no. The answer depends on why the card is made and what it will be. Let's take a look what are site maps.
Sitemap sitemap
Site maps are divided into 2 main types or formats: sitemap html and sitemap xml file. HTML sitemap is a site pageon which the links are listed. Usually these are links to the most important sections and pages of the site. HTML sitemap is more targeted at people, rather than robots and helps to quickly navigate the main sections of the site. For a sitemap in the form of an html page, there are serious limitations on the number of links on one page. If there are too many links on the page, by no means all links can be indexed or the page of the site map can be excluded from the search for an excessive number of links, even internal links.
In order for the site map html to be properly indexed and adequately perceived by visitors, do not place more than 100 links on the page. This is more than enough to place on the page all sections and subsections that did not fit into the main menu.
Usually, HTML sitemap It has a tree structure, where detailed sections and subsections are indicated. Excessively bulky HTML site maps are often decorated with graphic elements, CSS styles and complemented by a Java script. However, the html sitemap is not a big deal for search engines.
HTML sitemap is not a full sitemap. What to do if a site has hundreds, thousands, tens of thousands of pages? To do this, you must place links to all pages in the site map in xml format.
Sitemap XML Sitemap
An XML sitemap is an xml file of the form sitemap.xml, which is usually located at the root of the site. An xml sitemap has many advantages over an html sitemap. Sitemap xml is a special sitemap format, which is determined by all popular search engines, such as Google and Yandex. You can specify up to 50,000 links in the xml sitemap. Moreover, in sitemap xml you can specify the relative priority and page refresh rate.
It is worth saying that the contents of the site map is only a recommendation for a search robot. For example, if you set an annual refresh rate for a site page, search robots will still go more often. And if you set the page refresh rate hourly, this does not mean that the robots will index the page every hour.
An xml sitemap has a specific syntax, let's look at the structure of an xml sitemap in more detail.
Sample sitemap.xml sitemap file
The correct sitemap.xml must be UTF8 encoded. The content of the sitemap.xml sitemap file is as follows:
Where are the following required tags used:
- - parent tag, it contains all urls,
- - a tag that indicates information about a specific url,
- - in this tag url is specified directly.
Next, optional tags follow:
- - this tag contains the date the page was last modified,
- - the tag is used to indicate how often the page changes: always, hourly, daily, weekly, monthly, yearly, never,
- indicates the priority of a particular page, relative to other pages of the site from 0.1 - low priority, up to 1 - high priority.
Also, the site map file in xml format must contain an indication of the XML language namespace:
If the site map file includes more than 50 thousand links or the size of sitemap.xml exceeds 10 mb, it is recommended to divide the site map into several files. In this case, it is necessary to indicate several links to different map files in the site map.
Here we use the already familiar tags and, as well as the required tags:
- - the parent tag, which contains the addresses of all site maps,
- - tag, which contains the parameters for each site map.
Sample sitemap.txt sitemap file
Another way to design a site map as a file can be a site map in txt format:
Everything is simple. The sitemap.txt file lists all the necessary links line by line. A site map in txt format is an "option for the lazy." A similar sitemap xml restriction of 50,000 links works here. However, the TXT site map is deprived of the ability to indicate the date of the last change and page priority.
How to create a sitemap
Create a sitemap an important process in which it is necessary to clearly indicate which pages of the site should be indexed and how best to index them. Depending on the type of site map in question, various ways to create a sitemap. How to create an html sitemap to discuss separately does not make sense. Consider how to make a map map in xml file format. There are several basic ways to create a sitemap, but all of them are united by where the sitemap is located and how the sitemap is determined by search engines.
As already written above - sitemap file is located in the root of the site. Search engines are able to independently detect the site map file. But there are several ways to specify a direct link to the file (s) of the sitemap (s) for faster detection by search engines. The easiest way to specify the location of the site map file is to directly indicate the link or several links to the site map files in the tools for webmasters from Yandex and Google. There you can check site map, conduct site map analysis for correctness, the correspondence of which pages from the site map are found by the search engine and how many of them are in indexing.
The second way to tell search engines the location of the sitemap file is through the Sitemap directive of the robots.txt file.
Specifying a link to a site map in robots.txt:
You can specify several sitemap files in robots.txt, after which it will automatically be added to webmaster tools. How to find a sitemap we reviewed, now let's move on to how to create a sitemap.
The main ways to create a site map
- Site map generation site management system, if the CMS has a similar built-in capability.
- Download sitemap from a third-party online service. There are many online site map generators with various features and limitations. For example, xml-sitemaps.com has the ability to configure site map parameters, but has a limit of 500 pieces in the sitemap.xml file.
- Download site map generator program. Generator programs like these are usually paid, but with their help you can regularly generate sitemap xml for one or more sites. Here are a couple of examples of such generator programs: SiteMap XML Dynamic SiteMap Generator, WonderWebWare SiteMap Generatior.
- Automatically create sitemap sitemap in Joomla (Jumla), WordPress (WordPress), Bitrix (Bitrix).
- Manual site map creation.
Site map is one of the most important SEO website optimization tools. It doesn’t matter how the site map is created. Important - which links will be listed on the site map and how often it will be updated. Sometimes, everything is downloaded to the sitemap and even those links that are prohibited in robots.txt or non-canonical links. A site map update occurs once a month or less. Such an attitude to the site map can not only make it useless, but even worse - confuse the search robot, which will negatively affect the indexation and position of the site in the search.
Create a sitemap for your resource. But be careful and think carefully about what to upload to the sitemap and what not.
One card is not enough, give two!
As mentioned above, a site map can be used not only by robots, but also by users. Therefore, most modern resources contain two maps: XML and HTML.
An XML sitemap is used exclusively by search engines. Its role is to provide the "spiders" with reliable information about the structure of the site and its updating. Thanks to the map, robots quickly index new pages, and put them in a search base. If a large site does not contain a Sitemap.xml file, many of its pages may not be indexed for months, and sometimes even for several years.
HTML map is used to visualize the structure of the site. It depends on its simplicity and clarity whether the user can find the section of interest to him or not. Creating an HTML map makes sense only when you need to organize the usability of a large Internet project, whose sections and subsections do not fit in the main menu.
Some inexperienced users quite often use an HTML map as a tool for linking. This is a very blunder. Let’s explain why. Firstly, an HTML site map cannot be represented as a full-fledged Sitemap: at best, the search engine will perceive the page with the map as a huge set of links to the internal pages of the site, at worst it will pessimize its value due to the large number of links, not even despite the fact that they are not external. Secondly, even if the search robot does not classify your map as spam, it may not take into account a certain percentage of links, which is why some pages of your resource may not be indexed.
Making an XML sitemap
There are three ways to create an XML file:
- purchase a special generator that will create a site map for you,
- use one of the online services,
- write the file manually.
Acquiring a Sitemap generator is one of the best options. You do not have to create a site map manually, which will save several hours of time. The only drawback of such generators is the need to purchase a license. Most of these programs are released under a shareware license, the cost of which varies from 20 to 30 dollars. The best representatives of the generator programs are: SiteMap XML Dynamic SiteMap Generator, WonderWebWare SiteMap Generatior and Sitemapspal.
If your site consists of only a few hundred pages, it makes sense to use free online services. You only need to specify the address of your resource and click the "Download" button. As a result, you will receive a full map of your site. Among the online services, in our opinion the best are: www.xml-sitemaps.com (site size up to 500 pages) and www.freesitemapgenerator.com (up to 5000 pages).
If you want to create Sitemap.xml manually, you will need to know the following tags:
- - this tag contains information about a specific url (page address),
- - this tag can contain many urls,
- - this tag is used to indicate url,
- - this tag indicates the date the page was last modified,
- - indicates how often the page changes (attributes: always, hourly, daily, weekly, monthly, yearly, never),
describes the priority of the page being defined, compared to others (values from 0.1 - low priority, to 1 - high priority).
Tags, and are required as well, and
may not appear in the Sitemap.xml file.
After the site map has been created, it must be added to your site. For this, the created file must be called Sitemap.xml and "filled" in the root directory of the resource. For quick detection of a map by search engines, you can use the “Webmaster Tools” functionality from Google and Yandex-webmaster from Yandex.
In the first case, the sequence of actions is as follows: log in to the “tools”, select the resource you need from the list of your sites, go to the optimization tab / Sitemap files and on the opened page click the “Download” button, then select the map file and confirm its download.
To add a Sitemap file to the Yandex database, you need to log in to Yandex Webmaster, again, select the desired site, go to the indexing / sitemap settings tab, specify the path to the map file in the appeared field and click the "Add" button.
XML map
In the official directories of Google and Yandex services, you can find a lot of useful information on creating, updating and publishing a site map. There is also data on the structure of sitemap.xml files. This article contains only the most important and useful data for the average user.
The main tasks of the file:
Providing network addresses of all pages in a convenient form for quick viewing.
Informing about the type of published content, for example text, multimedia data.
Transmission of information about the frequency and time of the last content update, priority, alternative versions of pages.
A site map may contain various information that promotes fast and correct indexing in search engines of the necessary pages of a network resource.
The importance of having a card on a site
For search engine optimization, having a site map is a must. Any help to the search engine in viewing and analyzing the content of a network resource has a positive effect on the ranking of content. So say optimizers and webmasters. Immediately after completing work on the site, they create a sitemap.xml file and show it to search engines.
For search engines, the absence of such a file does not play a significant role. Robots are stand-alone software that does not need additional instructions. They themselves will find everything you need without specially created pages with links. If you add the site to the Yandex or Google’s webmaster panel without providing a link to the site map, the service will determine the problem as recommended, and not critical or fatal.
The main reasons for creating a map:
The presence of pages that do not have inbound links from the menu or site content.
A network resource has a complex structure and a large number of pages with a variety of content.
Regular site updates.
The site has a hierarchy of pages in importance.
The site map allows you to speed up the indexing of content and simplify the process of providing new data about the site to the search engine.
Creation and connection
You can get a file in various ways, including manually compiling a list of all page addresses. Of course, most often resort to various means of automation. For example, they use server or regular software. To create and regularly update a map of a network resource in automatic mode, it is easiest to use the capabilities of CMS. If the content management system does not have a built-in tool, then an additional module is found and installed. During the existence of the search index, many programs and additions have been developed for any requirements of the user and search engines.
Find the right service on the net.
Enter the address of the network resource.
Start the software by pressing the appropriate button.
The service robot will bypass the pages of the site and form an address map.
Upon completion, the service will give the user a link to download sitemap.xml. Download the file and upload to the root directory of the site. The disadvantage of this method is to obtain a static card. After adding new pages, you will have to manually repeat the above operation, delete the obsolete file and download the current version.
Using CMS Tools with WordPress as an Example
Install and activate the appropriate plugin, for example Google XML Sitemaps.
We go to the settings page and allow: notify the search engine of the map update, add the sitemap.xml location address to the robots.txt file, automatically compress the data as necessary.
In the section "Priority of the article" we prohibit the use of automatic calculation of the priority of pages, if it is important for you to be able to independently change their priority.
We proceed to the configuration of the contents of the map and check the necessary items with daws. In the "Other Options" subsection, we allow the service to set the date of the last map update.
In the next section, the service suggests changing the frequency of checking the file for new links and any other changes. You can set any value, since search engines usually decide when to perform the next crawl.
Next, go to the “Priorities” section, which allows you to set the importance of pages by selecting the desired value from the drop-down list. The set values can also be ignored by search robots, so you can leave them as they are.
The plugin will automatically create the file and place it in the desired directory. С этого момента веб-мастер может забыть о существовании sitemap.xml, так как сервис возьмет поддержку актуального состояния файла на себя. Осталось только указать поисковику адрес расположения новой карты сайта. Делают это через панель управления веб-мастера в соответствующей поисковой системе.
Создание карты вручную
Информация дается с целью ознакомления со структурой текстового файла, сохраненного в формате xml и используемого в качестве карты сайта. It is not practical to create such voluminous files manually, as modern software very quickly and efficiently copes with the task.
Features of the test file
To create a site map manually, you can use a notepad or any other text editor. Data is written to a file using special characters and XLM markup codes. After you finish entering all the links and directions for the search engine, you need to save the file with the extension xml.
Example site map code:
A standard header reporting the version of the code used and the character encoding:.
Specification of protocol standard:.
Enumeration of website page addresses with various parameters: exact page address / date of the last change in content update frequency
Closing a file with a tag:.
The urlset, url, loc tags are required, the rest can be used as desired. Many optimizers are not limited to creating a common map of sites. Additionally make an image map, separate for video, site news and other sections. Such cards allow you to transfer information to the search engine that is not accessible in the usual way due to the peculiarities of embedding content in a page, for example, through scripts.
Several sitemap.xml
The need to use several site maps arises in the case of the simultaneous use of different content management systems, the need to add more than 50 thousand pages to the site, providing convenient conditions for tracking errors that occur in a huge number of different sections and for other reasons.
All created cards need to be combined into one. That is, create a text file and write all the cards into it using special tags.
Common sitemap code:
Protocol Standard:. Instead of the urlset tag, use sitemapindex.
Enumeration of all site maps with the necessary parameters: address of the location of the map the date of the latest changes to the file.
We list all the cards and close the code with a tag.
Thus, only regular XML maps that are not separately indexed are merged. For separate indexing of several sitemaps, they must be added to the index through the webmaster panel, indicating to the search engine the exact path to the file of each map.
We pass the address of the site map to the search engine
In order for the Google search robot to find your map, you need to go to the Google Search Console, select the "Sitemaps" section, enter the network address of the file in the corresponding field and click on the "Submit" button. After a while, the service software will check for the file at the specified address. If a site map is detected, the robot will analyze the contents and show the user the result of the work done. If errors are found, a detailed report will be prepared for each with the addition of recommendations for eliminating all problems. If there are no errors, the message “Successful” will appear in the status of the site map.
In order to transfer the sitemap.xml address to the Yandex search system, you need to enter the webmaster’s panel, in the “Indexing” section, find the “Sitemap Files” subsection, enter the path to the file in the form and click on the “Add” button. You can find out about errors in the Diagnostics section.
Limitations and Features
By creating and adding a site map, the user increases the chances of a quick and regular indexing of all pages of a network resource. Significantly reduces the likelihood of losing content for search. The robot will bypass all the addresses specified in the file, take into account all the comments of the webmaster, index the available material. In order for the site map to work properly, when creating it, the user must take into account several restrictions.
Sitemap.xml requirements:
The file should not contain more than 50 thousand page addresses.
The file size must be less than or equal to 10 MB.
Owners of popular and large network resources have found several ways to solve the problems associated with these limitations. If your site requires a card that does not fit into the established framework, use the option from the list below.
Options for solving the problem of having restrictions:
Reduce the number of links by removing the addresses of pages that have a very low priority from the file. Leave important links and page addresses that are accessible to the search engine only through the site map.
Create several sitemap files and submit their addresses to search engines.
If the problem is in size, use the archiver to compress the data.
Site map for visitors
The sitemap for users is significantly different from the map for the search robot. This is a regular site page with many links with anchors, description of content, navigation and design elements. A user site map is created to accelerate the search for information within a network resource.
It is created in a regular text or special editor for preparing HTML pages and saved in a file with any name and html extension. The finished document can be placed in any directory. It is not necessary to inform the search engine about the availability of a custom map. In the process of indexing a site, a search robot will find it and index all the links.
Features of creating an HTML map:
Use formatting. Highlight headings, subheadings, make lists, notes, highlight text with color. The card should be convenient and pleasant to use, it should not be a normal page with many links that do not inform the user about the content they are led to.
Think over the structure and add a short description to each link. This will simplify the user search for information and make the map high-quality for search engines. They do not perceive material with links without text or multimedia accompaniment.
Add anchors to links. They will inform the visitor about the results of the transition on them. Such links can have a positive impact on the trust of a network resource, so search engines better perceive links with anchors.
After creating a custom map, save it in an accessible place and make a noticeable link to it. Unlike the search engine, visitors will not crawl the entire site in search of a convenient navigation page. Therefore, place the link in a visible place, for example, in the main menu, select it with a frame, color, change the font size. The more noticeable it is, the better. Otherwise, the efforts spent on its creation will not be useful.
Some tips for creating Sitemap.xml
- Search engines today accept Sitemaps containing no more than 50,000 url. You should also pay attention to the size of Sitemap.xml. If it exceeds 10 MB, Google recommends breaking the cutter into several files. Thus, you will get rid of possible server overloads.
- When creating multiple Sitemaps, it is recommended to list them in the index file, which is created using the following tags: (parent tag located at the beginning and end of the file), (tag for each of the Sitemap files), (indicates the location of each of the Sitemap files. xml) and (optional tag indicating the dates of the latest changes to the described map files).
1 All page addresses should be specified either with the www prefix or without it.
2 File cards must have UTF8 encoding.
3 Sitemap.xml must contain an indication of the XML language namespace: xmlns = "http://www.sitemaps.org/schemas/sitemap/0.9".
Creating an HTML sitemap
Since the HTML map is used mainly by users of your resource, it should be simple, clear and at the same time provide comprehensive information about the structure of your site.
Most HTML maps have a familiar tree structure, sections and subsections in which are highlighted by various CSS styles, and possibly some graphic elements.
If you are creating an HTML map for a large portal consisting of several thousand pages, we recommend creating tab links to sections. So you get rid of the main drawback of large cards - bulkiness. Since the HTML map is of secondary importance for search engines, you can use Java-script to create it, which can significantly increase the functionality of the created page.
