3.17 Include expected and beneficial files

Web Development

Always provide expected files and optionally offer environmentally or otherwise beneficial files to enhance the project.

Criteria

• Expected files: Machine-testable
  Include favicon.ico, robots.txt, opensearch.xml, site.webmanifest, and sitemap.xml files by default. Also ensure that any similar files defined in future web standards or specifications are included.Related Resources
  • About /robots.txt
  • AssetConfig
  • Build and submit a sitemap
  • Define a favicon to show in search results
  • Favicon Generator
  • GPF – General Policy Framework (PDF) – 1.9 – Strategy (Interoperable Technologies)
  • How Google interprets the robots.txt specification
  • OpenSearch
  • OpenSearch Protocol
  • Sitemaps Protocol
  • The Carbon Impact of Web Standards (PDF)
  • Web Application Manifest
• Beneficial files: Machine-testable
  Include beneficial files such as ads.txt, carbon.txt, humans.txt, security.txt. Also ensure that any similar files defined in future web standards or specifications are included.Related Resources
  • Ads.txt guide
  • All about humans. Humans.txt, actually
  • AssetConfig
  • Google Says LLMs.Txt Comparable To Keywords Meta Tag
  • GPF – General Policy Framework (PDF) – 1.9 – Strategy (Interoperable Technologies)
  • Meet LLMs.txt, a proposed standard for AI website content crawling
  • security.txt
  • slash pages
  • The ads.txt Standard
  • The Carbon Impact of Web Standards (PDF)
  • The carbon.txt Proposed Standard
  • The humans.txt Standard
  • The llms.txt Proposed Standard
  • The security.txt Proposed Standard
  • What Is llms.txt, and Should You Care About It?

Benefits

• Accessibility
  Integrating expected files enables the browser”s default search box to search a service, replacing any custom solution. This can increase accessibility as it encourages the use of a browser-native component and/or keyboard shortcuts, which can often better meet accessibility requirements.
• Conversion
  Configuring robots.txt appropriately can help to ensure content is correctly indexed and users are better guided to appropriate content on your project.
• Economic
  Including robots.txt and sitemap files helps search engines to discover and index your website. This can lead to more users and potentially more customers. The ads.txt file may reduce advertising fraud and could similarly benefit your business.
• Environment
  Providing files expected by search engines or browsers will reduce loading errors and may improve efficiency in how users find or interact with a site. Plain text requires no rendering. If users or search engines are able to find these files, such as carbon.txt, they can load more quickly and with less CPU/GPU impact than any formatted webpage.
• Performance
  Satisfying requests for expected files improves interactions with search engines or browsers requesting them, while also potentially avoiding additional requests once they are discovered. Plain text files contain no links, no markup, and has a low rendering impact. Including details such as site credits in such a file will reduce data transfer and have a lower rendering footprint.
• Transparency
  Providing a humans.txt file allows you to credit the people involved in the creation process, while security.txt provides critical points of contact if an issue is discovered. Both increase transparency.

GRI

• Materials: Low
• Energy: Low
• Water: Low
• Emissions: Low

View the Guideline

Tags

CompatibilityPatternsUIMarketingAssetsSecurity