If any website page is being blocked by the robots.txt file, Google won’t display the search snippet (meta description) in the search results. Google may display the page in the search results because of the linkage data it has. The instance in the image above is the only URL. Barry Schwartz have blocked from Google indexing it. I rue writing it, but also would regret erasing it, so he blocked it from Google.
I’m showing you an example with the help of Google search, if you try to match on it within Google, it turns the notification:
A description for this result is not available because of this site's robots.txt – learn more.
If you want to learn more link goes to the page named Block or remove pages using a robots.txt file in Google's help section.
I wonder this provides a much better search experience for someone who visits these blocked pages.
I do think if they show up for non site: command searches, I’d assume so.
