Description and Robots Metadata

Metadata provides information about a document. The description can summarize the page for search result snippets, while robots directives can influence how crawlers handle a page.

<head>
  <meta name="description"
        content="A practical guide to semantic HTML, accessible page structure, and useful native elements.">
  <meta name="robots" content="index, follow">
</head>

The description should accurately summarize the page. Search engines may use it, rewrite it, or choose an excerpt from the page instead. It is not a guaranteed snippet and should not be treated as a place to repeat keywords.

The robots metadata can express directives such as noindex or nofollow, but crawler support and interpretation matter. A page blocked by robots.txt may not be crawled to see its noindex directive. Do not combine conflicting crawler instructions across HTML and HTTP headers.

Public course pages and useful articles are usually intended to be discoverable. Private previews, internal tools, and duplicate utility pages may need different handling. Make indexing decisions deliberately rather than adding a robots tag to every page without understanding its effect.