# robots.txt automaticaly generated by PrestaShop e-commerce open-source solution # http://www.prestashop.com - http://www.prestashop.com/forums # This file is to prevent the crawling and indexing of certain parts # of your site by web crawlers and spiders run by sites like Yahoo! # and Google. By telling these "robots" where not to go on your site, # you save bandwidth and server resources. # For more information about the robots.txt standard, see: # http://www.robotstxt.org/robotstxt.html # --------------------------------------------------------------------------- # Onibex 2026-09-01 - Crawlers sin valor comercial que consultan precio/stock # a SAP en cada ficha de producto. Medicion en changes/ y auditoria de bots. # Googlebot, bingbot y GPTBot NO se restringen: ver nota al final. # --------------------------------------------------------------------------- # MJ12bot (Majestic): 450 fichas de producto en 12 minutos (pico 234/5 min). User-agent: MJ12bot Disallow: / # PetalBot (Huawei): rastreo sostenido, 1.212 fichas en 31 dias desde 8 IPs. User-agent: PetalBot Disallow: / # panscient.com: 132 fichas en 8 minutos. User-agent: panscient.com Disallow: / # SemrushBot: herramienta SEO de terceros. User-agent: SemrushBot Disallow: / # DotBot (Moz). User-agent: DotBot Disallow: / # AhrefsBot (indice de backlinks). User-agent: AhrefsBot Disallow: / # BLEXBot / SE Ranking: barrido de catalogo completo (162 fichas distintas). User-agent: BLEXBot Disallow: / User-agent: SEranking Disallow: / # Barkrowler (babbar.eu). No consulta robots.txt: se refuerza en el WAF. User-agent: Barkrowler Disallow: / # Crawlers de IA con historial de ignorar robots.txt. Se declaran aqui por # completitud, pero su enforcement real tiene que ser una regla de WAF. User-agent: Bytespider Disallow: / User-agent: PerplexityBot Disallow: / User-agent: * Crawl-delay: 10 # Allow Directives Allow: */modules/*.css Allow: */modules/*.js # Private pages Disallow: /*?orderby= Disallow: /*?orderway= Disallow: /*?tag= Disallow: /*?id_currency= Disallow: /*?search_query= Disallow: /*?back= Disallow: /*?n= Disallow: /*&orderby= Disallow: /*&orderway= Disallow: /*&tag= Disallow: /*&id_currency= Disallow: /*&search_query= Disallow: /*&back= Disallow: /*&n= Disallow: /*controller=addresses Disallow: /*controller=address Disallow: /*controller=authentication Disallow: /*controller=cart Disallow: /*controller=discount Disallow: /*controller=footer Disallow: /*controller=get-file Disallow: /*controller=header Disallow: /*controller=history Disallow: /*controller=identity Disallow: /*controller=images.inc Disallow: /*controller=init Disallow: /*controller=my-account Disallow: /*controller=order Disallow: /*controller=order-opc Disallow: /*controller=order-slip Disallow: /*controller=order-detail Disallow: /*controller=order-follow Disallow: /*controller=order-return Disallow: /*controller=order-confirmation Disallow: /*controller=pagination Disallow: /*controller=password Disallow: /*controller=pdf-invoice Disallow: /*controller=pdf-order-return Disallow: /*controller=pdf-order-slip Disallow: /*controller=product-sort Disallow: /*controller=search Disallow: /*controller=statistics Disallow: /*controller=attachment Disallow: /*controller=guest-tracking # Directories Disallow: */classes/ Disallow: */config/ Disallow: */download/ Disallow: */mails/ Disallow: */modules/ Disallow: */translations/ Disallow: */tools/ # Files Disallow: /*recuperacion-contraseƱa Disallow: /*direccion Disallow: /*direcciones Disallow: /*inicio-sesion Disallow: /*carrito Disallow: /*descuento Disallow: /*historial-compra Disallow: /*datos-personales Disallow: /*mi-cuenta Disallow: /*seguimiento-pedido Disallow: /*albaran Disallow: /*pedido Disallow: /*buscar Disallow: /*pedido-rapido Disallow: /*seguimiento-cliente-no-registrado Disallow: /*confirmacion-pedido # Sitemap Sitemap: https://miterza.com/1_index_sitemap.xml