When I fetch my website page using GWT this is what I receive. We are not in a massive rush to complete all of this so we have the time to plan and execute this when we are fully ready.Īny help / advise would be greatly appreciated. First, go to the terminal/command line interface (hereafter referred to as terminal) on your local computer and navigate to the folder you want to work from (e.g. Or based on the data i have provided do you disagree with the plan, what would you recommend? Should I change the domain of the main site first and then merge site 2 and 3 in later? Should I merge the 3 sites into 1 and then changed the domain at a later point? I am fairly happy with the steps required (having read all the guides and migrated many sites in the past) but with the added domain name change this is a little daunting.

As you can imagine this is going to be quite a job. So the plan is to combine site 2 and 3 into site 1 and change that domain name. On top of this as the main domain has one of the product names in it they wish to change the domain to be just. The main site is an e-commerece site whereas the other 2 are not (currently). We wish to bring all the sites into one with categories for the 3 different product. (DA = 10 ranking very well for product name and little competition)ī (DA = 10 poor ranking) Our company has 3 sites, the main site and 2 sites for different product ranges:ī ( main site - DA = 22 raking well for product name) I have read so many very helpful guides and experiences from you guys that will greatly help me but I have a few questions please. Give it a few minutes and see if any of the above suggestions work. It could just be a slow connection on your end. Click the checkbox and try running it again. One thing you can try, is goto Configuration > Spider and then goto the last option Ignore robots.txt. Run through your settings and check and see if you may have turned something on inadvertently that you didn't mean to. This is shown in the Content column and should be either text/html or application/xhtml+xml.

The Content-Type header did not indicate the page is html.

The SEO Spider does not crawl the frame src attribute. These can be seen in the “Directives” tab in the “Nofollow” filter. The could be set by either a meta robots tag or an X-Robots-Tag in the HTTP header.

The page has a page level ‘nofollow’ attribute.

There is an option in Configuration->Spider under the “Basic” tab to follow ‘nofollow’ links. This allows the easy identification and evaluation of changes to the site for use in a technical SEO and website audit. The tool will retrieve critical on-site data and organize it into understandable stats and reports.

The ‘nofollow’ attribute is present on links not being crawled. The Screaming Frog SEO spider tool is a website crawler that allows you to crawl websites just as a Google bot would.

Can you view the site with cookies disabled in your browser? Licenced users can enable cookies by going to Configuration->Spider and ticking “Allow Cookies” in the “Advanced” tab. Try looking at the site in your browser with JavaScript disabled. Try changing the User Agent under Configuration->User Agent.

The site behaves differently depending on User Agent.You can configure the SEO Spider to ignore robots.txt by going to the “Basic” tab under Configuration->Spider. A count of pages blocked by robots.txt is shown in the crawl overview pane on top right hand site of the user interface. Here are a list of reasons why ScreamingFrog won't crawl your site: Reports about the page's metatags, incoming and outgoing links and keywords.īy downloading SearchStatus you also contribute towards creating statistics about visited webs, as the plug-in provides the data of visits to Alexa, PageRank and Compete with statistical purposes and in a confidential manner.ĭownload SearchStatus and get to know the importance of any website you browse.It's crawling for me.Access to web positioning services such as Google Trends.Positions on the Alexa and Compete rankings.With its different viewing modes, it is an interesting tool for both professionals and for anyone interested in this type of information. These data, that show the PageRank and its position on the Alexa ranking, also grant access to other positioning services as well as to the page's file on. SearchStatus adds to the browser's window several details that will help users to get to know the impact and influence of the website they are visiting on the Internet. Web administrators, positioning experts or anyone who wants to know about the importance on the Internet of the webpages they visit, will find in SearchStatus a very interesting add-on for Firefox.įind out the importance of any visited website with SearchStatus.

Screaming frog seo spider 4.1